{"suite_start":"2026-01-31T19:01:01.631Z","total_tests":30}
--- TEST_RESULT ---
{"id":"1.1 Hard Future - Month Year","query":"Show me LeBron James' points per game in March 2026","success":true,"confidence":0.537,"warnings":["\"Marc Gasol\" might also refer to \"Marc Jackson\"","⚠️ LeBron James is currently Day-To-Day","✅ Marc Gasol: No injury reported in SportsDB (active/healthy)"],"response_snippet":"=== DATA SHERIFF CONTEXT ===\n\n## Query Analysis\nIntent: historical_performance (0%)\nLeague: NBA\n**TIMEFRAME CONSTRAINT: 10/1/2026 to 6/30/2027**\n⚠️ ONLY provide data from this timeframe. Do NOT include data from other dates.\n\n## Resolved Entities\n- player: \"LeBron James\" (100% confidence)\n  Team: LO","rag_insight":false,"assistant_content":"","duration":2651}
--- TEST_RESULT ---
{"id":"1.2 Hard Future - Championship","query":"Backtest betting on the 2027 Super Bowl winner","success":true,"confidence":0.5755,"warnings":["Could not find \"Super Bowl\" in our sports database. This may be a misspelling or an unknown entity.","Could not find \"Backtest\" in our sports database. This may be a misspelling or an unknown entity.","Data coverage is limited (39%)"],"response_snippet":"For backtesting, please specify a **past time period** with completed games, such as:\n- \"2023 NFL season\" (Super Bowl LVII)\n- \"2022-2023 NFL season\"\n- \"Last 5 NFL seasons\"\n\nExample request: \"Backtest a strategy betting on the Super Bowl winner for the 2023 NFL season.\"\n\nIn the meantime, consider try","rag_insight":true,"assistant_content":"","duration":123146}
--- TEST_RESULT ---
{"id":"1.3 Hard Future - Specific Day","query":"What was the spread for Warriors vs Celtics on December 25, 2026?","success":true,"confidence":0.6096666666666667,"warnings":["Could not find \"December\" in our sports database. This may be a misspelling or an unknown entity.","Data coverage is limited (41%)"],"response_snippet":"Strategy analysis complete. For detailed backtesting results, please use the **Strategy Playground** tab above.\n\n---\n📊 **Data Notes:** Data coverage: 41% | Could not find \"December\" in our sports database. This may be a misspelling or an unknown entity. | Data coverage is limited (41%)\n\n---\n📊 **Da","rag_insight":true,"assistant_content":"","duration":6777}
--- TEST_RESULT ---
{"id":"1.4 Hard Future - Season Week","query":"Show me Patrick Mahomes' stats for Week 18, 2026 NFL season","success":true,"confidence":0.6970000000000001,"warnings":["\"Marquis Weeks\" might also refer to \"Rickie Weeks\"","⚠️ Patrick Mahomes is currently Injured Reserve","✅ Marquis Weeks: No injury reported in SportsDB (active/healthy)"],"response_snippet":"Strategy analysis complete. For detailed backtesting results, please use the **Strategy Playground** tab above.\n\n---\n📊 **Data Notes:** \"Marquis Weeks\" might also refer to \"Rickie Weeks\" | ⚠️ Patrick Mahomes is currently Injured Reserve\n\n---\n📊 **Data Notes:** \"Marquis Weeks\" might also refer to \"Ri","rag_insight":true,"assistant_content":"","duration":16185}
--- TEST_RESULT ---
{"id":"1.5 Hard Future - Award","query":"Backtest betting on the 2026 NBA Finals MVP","success":true,"confidence":0.583,"warnings":["Could not find \"Finals MVP\" in our sports database. This may be a misspelling or an unknown entity.","Could not find \"Backtest\" in our sports database. This may be a misspelling or an unknown entity.","Data coverage is limited (41%)"],"response_snippet":"Backtesting requires historical game scores, player stats, and bet outcomes, which aren't available for future events. \n\n- **For 2026 futures betting**: Try **paper trading** first to practice with fake money—no risk! Track odds from The Odds API for top candidates (e.g., stars from likely contender","rag_insight":true,"assistant_content":"","duration":67568}
--- TEST_RESULT ---
{"id":"1.6 Soft Future - Relative","query":"Predict the winner of next year's Stanley Cup","success":true,"confidence":0.5755,"warnings":["Could not find \"Stanley Cup\" in our sports database. This may be a misspelling or an unknown entity.","Could not find \"Predict\" in our sports database. This may be a misspelling or an unknown entity.","Data coverage is limited (39%)"],"response_snippet":"Strategy analysis complete. For detailed backtesting results, please use the **Strategy Playground** tab above.\n\n---\n📊 **Data Notes:** Data coverage: 39% | Could not find \"Stanley Cup\" in our sports database. This may be a misspelling or an unknown entity. | Could not find \"Predict\" in our sports d","rag_insight":true,"assistant_content":"","duration":14726}
--- TEST_RESULT ---
{"id":"1.7 Soft Future - Distant","query":"How will Connor McDavid do in the 2027 playoffs?","success":true,"confidence":0.657,"warnings":["✅ Connor McDavid: No injury reported in SportsDB (active/healthy)"],"response_snippet":"**Injury Status:** ✅ Connor McDavid: No injury reported (active/healthy).\n\n**Recent Games Data (4/17/2026):**  \n- Seattle Kraken @ Colorado Avalanche (TBD)  \n- Vancouver Canucks @ Edmonton Oilers (TBD)  \n- Los Angeles Kings @ Calgary Flames (TBD)  \n\n**Recent Player Stats (Connor McDavid, EDM):**  \n-","rag_insight":true,"assistant_content":"","duration":15591}
--- TEST_RESULT ---
{"id":"1.8 Soft Future - Specific Event","query":"Show me betting lines for the 2026 World Series game 7","success":true,"confidence":0.5755,"warnings":["Could not find \"World Series\" in our sports database. This may be a misspelling or an unknown entity.","Data coverage is limited (39%)"],"response_snippet":"=== DATA SHERIFF CONTEXT ===\n\n## Query Analysis\nIntent: odds_lookup (33%)\n**TIMEFRAME CONSTRAINT: 10/1/2026 to 6/30/2027**\n⚠️ ONLY provide data from this timeframe. Do NOT include data from other dates.\n\n## ⚠️ UNKNOWN ENTITIES DETECTED\nThe following names could NOT be found in our database:\n- \"World","rag_insight":false,"assistant_content":"","duration":407}
--- TEST_RESULT ---
{"id":"2.1 Fake Player - Fictional","query":"Show me Glup Shitto's career NBA stats","success":true,"confidence":0.48300000000000004,"warnings":["Could not find \"Glup Shitto\" in our sports database. This may be a misspelling or an unknown entity.","Data coverage is limited (41%)"],"response_snippet":"Glup Shitto is not found in our NBA database. No career stats available. Verify spelling or check official NBA sources like Basketball-Reference or NBA.com.\n\n---\n📊 **Data Notes:** Data coverage: 41% | Could not find \"Glup Shitto\" in our sports database. This may be a misspelling or an unknown entit","rag_insight":true,"assistant_content":"","duration":4219}
--- TEST_RESULT ---
{"id":"2.2 Fake Player - Plausible","query":"Backtest betting on teams when John Basketball plays","success":false,"warnings":[],"response_snippet":"NO_CONTENT","rag_insight":false,"assistant_content":"⚠️ **Time Period Required**\n\nTo ensure accurate backtesting results, you must specify a time period for your strategy testing.\n\nPlease specify the time period for backtesting (e.g., '2024-25 NBA season', '2024 regular season').\n\nThis helps avoid overfitting and provides transparency about when your strategy was tested.","duration":24}
--- TEST_RESULT ---
{"id":"2.3 Fake Relation","query":"What's Mike Trout's brother's baseball stats?","success":true,"confidence":0.657,"warnings":["✅ Mike Trout: No injury reported in SportsDB (active/healthy)"],"response_snippet":"No data available on Mike Trout's brother's baseball stats in SportsDB. Player not found.\n\nPossible reasons:\n- Name may be misspelled (e.g., specify full name like \"Jeff Trout\")\n- Not in our MLB database\n- May not be an active/pro player\n\nMike Trout (LAA, CF) is active/healthy per SportsDB. Recent s","rag_insight":true,"assistant_content":"","duration":9436}
--- TEST_RESULT ---
{"id":"2.4 Real Person - Wrong League","query":"Show me the career of LeBron James Jr. in the NBA","success":true,"confidence":0.557,"warnings":["Could not find \"James Jr\" in our sports database. This may be a misspelling or an unknown entity.","⚠️ LeBron James is currently Day-To-Day"],"response_snippet":"**LeBron James Jr. not found in our NBA database.** No career stats or data available. Check official NBA sources (e.g., NBA.com or ESPN) for Bronny James' information. LeBron James (LAL) recent stats (1/26/2026): 24 PTS, 5 REB, 3 AST.\n\n---\n📊 **Data Notes:** Data coverage: 59% | Could not find \"Jam","rag_insight":true,"assistant_content":"","duration":8565}
--- TEST_RESULT ---
{"id":"2.5 Impossible Timeline","query":"Analyze Michael Jordan's performance after his 2025 comeback","success":true,"confidence":0.603,"warnings":["Could not find \"Analyze Michael\" in our sports database. This may be a misspelling or an unknown entity.","⚠️ Michael Jordan is currently Day-To-Day","Data coverage is limited (41%)"],"response_snippet":"**No performance data available.**  \nSportsDB has no records of Michael Jordan playing in the NBA after a 2025 comeback (timeframe: 10/1/2025–6/30/2026). He is currently **Day-To-Day** (CHI). Check official NBA sources for any updates.\n\n---\n📊 **Data Notes:** Data coverage: 41% | Could not find \"Ana","rag_insight":true,"assistant_content":"","duration":9515}
--- TEST_RESULT ---
{"id":"2.6 Fake Team","query":"Backtest betting on the Albuquerque Roadrunners NFL team","success":false,"warnings":[],"response_snippet":"NO_CONTENT","rag_insight":false,"assistant_content":"⚠️ **Time Period Required**\n\nTo ensure accurate backtesting results, you must specify a time period for your strategy testing.\n\nPlease specify the time period for backtesting (e.g., '2024-25 NBA season', '2024 regular season').\n\nThis helps avoid overfitting and provides transparency about when your strategy was tested.","duration":20}
--- TEST_RESULT ---
{"id":"2.7 Fictional Event","query":"Show me stats for the 2025 Space Jam tournament","success":true,"confidence":0.5755,"warnings":["Could not find \"Space Jam\" in our sports database. This may be a misspelling or an unknown entity.","Data coverage is limited (39%)"],"response_snippet":"Strategy analysis complete. For detailed backtesting results, please use the **Strategy Playground** tab above.\n\n---\n📊 **Data Notes:** Data coverage: 39% | Could not find \"Space Jam\" in our sports database. This may be a misspelling or an unknown entity. | Data coverage is limited (39%)\n\n---\n📊 **D","rag_insight":true,"assistant_content":"","duration":3468}
--- TEST_RESULT ---
{"id":"2.8 Real Team - Wrong League","query":"Analyze the London Lions' 2025 NFL season","success":true,"confidence":0.543,"warnings":["Could not find \"London Lions\" in our sports database. This may be a misspelling or an unknown entity.","Could not find \"Analyze\" in our sports database. This may be a misspelling or an unknown entity."],"response_snippet":"No \"London Lions\" team exists in our NFL database or official records for the 2025 season. This may be a misspelling, fictional entity, or confusion with another league (e.g., basketball's London Lions). No stats, schedule, or performance data available.\n\nCheck official NFL sources like NFL.com for ","rag_insight":true,"assistant_content":"","duration":10645}
--- TEST_RESULT ---
{"id":"2.9 Fictional Sport","query":"Backtest betting on the Quidditch World Cup 2025","success":true,"confidence":0.5755,"warnings":["Could not find \"Quidditch World\" in our sports database. This may be a misspelling or an unknown entity.","Could not find \"World Cup\" in our sports database. This may be a misspelling or an unknown entity.","Could not find \"Backtest\" in our sports database. This may be a misspelling or an unknown entity."],"response_snippet":"I'm sorry, but Quidditch World Cup 2025 is not a real sporting event covered in our sports databases (SportsDB, BallDontLie, The Odds API, etc.). Quidditch is a fictional sport from the Harry Potter universe, and no historical or scheduled game data, odds, or results exist for backtesting.\n\n**Data L","rag_insight":true,"assistant_content":"","duration":30547}
--- TEST_RESULT ---
{"id":"3.1 Mismatch - LBJ Touchdowns","query":"Show me LeBron James' career touchdown passes","success":true,"confidence":0.557,"warnings":["⚠️ LeBron James is currently Day-To-Day"],"response_snippet":"LeBron James (LAL) is an NBA player. Touchdown passes are an NFL statistic with no data available in SportsDB.\n\nRecent NBA stats (1/26/2026 game only, timeframe 1/1/2026-1/31/2026):\n- Points: 24\n- Rebounds: 5 (5 def, 0 off)\n- Assists: 3\n- Steals: 1\n- Blocks: 0\n- Turnovers: 2\n\n**Status:** Day-To-Day ","rag_insight":true,"assistant_content":"","duration":9297}
--- TEST_RESULT ---
{"id":"3.2 Mismatch - Mahomes 3PT","query":"Analyze Patrick Mahomes' three-point percentage","success":true,"confidence":0.617,"warnings":["Could not find \"Analyze Patrick\" in our sports database. This may be a misspelling or an unknown entity.","⚠️ Patrick Mahomes is currently Injured Reserve"],"response_snippet":"Patrick Mahomes is an NFL quarterback for the Kansas City Chiefs. Three-point percentage is a basketball statistic (3PT%) and does not apply to football players or NFL games.\n\n- 12/15/2024: 159 pass yds, 2 pass TD\n- 11/24/2024: 269 pass yds, 3 pass TD, 60 rush yds\n\nCheck official NFL sources for any","rag_insight":true,"assistant_content":"","duration":60813}
--- TEST_RESULT ---
{"id":"3.3 Mismatch - McDavid HR","query":"Backtest betting on Connor McDavid's home runs","success":false,"warnings":[],"response_snippet":"NO_CONTENT","rag_insight":false,"assistant_content":"⚠️ **Time Period Required**\n\nTo ensure accurate backtesting results, you must specify a time period for your strategy testing.\n\nPlease specify the time period for backtesting (e.g., '2024-25 NBA season', '2024 regular season').\n\nThis helps avoid overfitting and provides transparency about when your strategy was tested.","duration":23}
--- TEST_RESULT ---
{"id":"3.4 Mismatch - Ohtani PIM","query":"Show me Shohei Ohtani's penalty minutes","success":true,"confidence":0.557,"warnings":["✅ Shohei Ohtani: No injury reported in SportsDB (active/healthy)"],"response_snippet":"**Shohei Ohtani (LAD, Designated Hitter)**: Penalty minutes data not available in SportsDB (not an MLB stat; applies to NHL).\n\n✅ No injury reported (active/healthy).\n\n**Recent Games Data** (timeframe 1/1/2026-1/31/2026):\n- 9/2/2026: SF @ PIT (TBD)\n- 9/2/2026: STL @ LAD (TBD)\n- 9/2/2026: NYM @ TB (TB","rag_insight":true,"assistant_content":"","duration":10526}
--- TEST_RESULT ---
{"id":"3.5 Paradox - MJ 2015","query":"Show me Michael Jordan's stats from the 2015 NBA season","success":true,"confidence":0.503,"warnings":["⚠️ Michael Jordan is currently Day-To-Day","Data coverage is limited (41%)"],"response_snippet":"**Michael Jordan (CHI) 2015 NBA Season Stats: Data not available in SportsDB.**\n\n**Current Status:** Michael Jordan is Day-To-Day.\n\n**Recent Games Data (not from 2015):**\n- 4/13/2026: New Orleans Pelicans @ Minnesota Timberwolves (TBD)\n- 4/13/2026: Golden State Warriors @ Los Angeles Clippers (TBD)\n","rag_insight":true,"assistant_content":"","duration":6431}
--- TEST_RESULT ---
{"id":"3.6 Paradox - Brady 2030","query":"Analyze Tom Brady's performance in the 2030 Super Bowl","success":true,"confidence":0.663,"warnings":["Could not find \"Analyze Tom\" in our sports database. This may be a misspelling or an unknown entity.","Could not find \"Super Bowl\" in our sports database. This may be a misspelling or an unknown entity.","⚠️ Tom Brady is currently Questionable"],"response_snippet":"Super Bowl 2030 has not occurred. Check official NFL sources for future schedules.\n\n---\n📊 **Data Notes:** Data coverage: 61% | Could not find \"Analyze Tom\" in our sports database. This may be a misspelling or an unknown entity. | Could not find \"Super Bowl\" in our sports database. This may be a mis","rag_insight":true,"assistant_content":"","duration":7267}
--- TEST_RESULT ---
{"id":"3.7 Paradox - Ruth 2025","query":"Backtest betting on Babe Ruth's 2025 home run total","success":true,"confidence":0.5755,"warnings":["Could not find \"Babe Ruth\" in our sports database. This may be a misspelling or an unknown entity.","Could not find \"Backtest\" in our sports database. This may be a misspelling or an unknown entity.","Data coverage is limited (39%)"],"response_snippet":"Could not find \"Babe Ruth\" in our sports database (SportsDB, MLB Stats API, Baseball Savant) for the 2025 timeframe (10/1/2025 to 6/30/2026). Babe Ruth was a historical MLB player active from 1914–1935 with no modern or future season data available.\n\nBacktesting requires historical game/player data,","rag_insight":true,"assistant_content":"","duration":44783}
--- TEST_RESULT ---
{"id":"3.8 Impossible Stat - 200pts","query":"Show me the game where LeBron James scored 200 points","success":true,"confidence":0.657,"warnings":["⚠️ LeBron James is currently Day-To-Day"],"response_snippet":"=== DATA SHERIFF CONTEXT ===\n\n## Query Analysis\nIntent: historical_performance (33%)\nLeague: NBA\n**TIMEFRAME CONSTRAINT: 1/1/2026 to 1/31/2026**\n⚠️ ONLY provide data from this timeframe. Do NOT include data from other dates.\n\n## Resolved Entities\n- player: \"LeBron James\" (100% confidence)\n  Team: LO","rag_insight":false,"assistant_content":"","duration":1923}
--- TEST_RESULT ---
{"id":"3.9 Impossible Score","query":"Backtest betting on NFL games ending 1-0","success":false,"warnings":[],"response_snippet":"NO_CONTENT","rag_insight":false,"assistant_content":"⚠️ **Time Period Required**\n\nTo ensure accurate backtesting results, you must specify a time period for your strategy testing.\n\nPlease specify the time period for backtesting (e.g., '2024-25 NBA season', '2024 regular season').\n\nThis helps avoid overfitting and provides transparency about when your strategy was tested.","duration":11}
--- TEST_RESULT ---
{"id":"3.10 Impossible Neg Stat","query":"Analyze the game where a team had -50 rebounds","success":true,"confidence":0.5755,"warnings":["Could not find \"Analyze\" in our sports database. This may be a misspelling or an unknown entity.","Data coverage is limited (39%)"],"response_snippet":"Strategy analysis complete. For detailed backtesting results, please use the **Strategy Playground** tab above.\n\n---\n📊 **Data Notes:** Data coverage: 39% | Could not find \"Analyze\" in our sports database. This may be a misspelling or an unknown entity. | Data coverage is limited (39%)\n\n---\n📊 **Dat","rag_insight":true,"assistant_content":"","duration":13242}
--- TEST_RESULT ---
{"id":"4.1 Era Gap - Props 1995","query":"Show me player prop bets for all NBA games in 1995","success":true,"confidence":0.583,"warnings":["Data coverage is limited (41%)"],"response_snippet":"**📊 POINTS Props for SAS @ CHA**\n\n• **KON_KNUEPPEL_1_NBA**: 4.5 (UNDER) @ +105\n• **KON_KNUEPPEL_1_NBA**: 4.5 (OVER) @ +100\n• **MILES_BRIDGES_1_NBA**: 3.5 (UNDER) @ -107\n• **BRANDON_MILLER_1_NBA**: 19.5 (UNDER) @ -106\n• **LaMelo Ball**: 3.5 (OVER) @ +100\n• **JULIAN_CHAMPAGNIE_1_NBA**: 2.5 (UNDER) @ ","rag_insight":false,"assistant_content":"","duration":37640}
--- TEST_RESULT ---
{"id":"4.2 Era Gap - Public Betting 80s","query":"Analyze public betting percentages for 1980s NFL games","success":true,"confidence":0.543,"warnings":["Could not find \"Analyze\" in our sports database. This may be a misspelling or an unknown entity."],"response_snippet":"Strategy analysis complete. For detailed backtesting results, please use the **Strategy Playground** tab above.\n\n---\n📊 **Data Notes:** Data coverage: 61% | Could not find \"Analyze\" in our sports database. This may be a misspelling or an unknown entity.\n\n---\n📊 **Data Notes:** Data coverage: 61% | C","rag_insight":true,"assistant_content":"","duration":6748}
--- TEST_RESULT ---
{"id":"4.3 Era Gap - Advanced Stats","query":"Backtest betting on NHL advanced stats (Corsi, Fenwick) in 2000","success":true,"confidence":0.583,"warnings":["Could not find \"Backtest\" in our sports database. This may be a misspelling or an unknown entity.","Could not find \"Fenwick\" in our sports database. This may be a misspelling or an unknown entity.","Could not find \"Corsi\" in our sports database. This may be a misspelling or an unknown entity."],"response_snippet":"I'm sorry, but Corsi and Fenwick stats are not available in SportsDB (or other integrated sources like NHL API, Natural Stat Trick) for the 2000 NHL season (interpreted as Oct 2000–Jun 2001). These advanced analytics (shot attempt metrics) were not systematically tracked or publicly available until ","rag_insight":true,"assistant_content":"","duration":61064}