$Ki2xdZddlZddlZddlZddlZddlZddlZddlmZmZm Z ddl Z ejjdejjejjeddlmZdZdZddiZd d d d d d ddd d dd dZdZdZdZdZdZddZdZedk(reyy)z Ingest Play-by-Play Data from ESPN Captures game flow events for pace analysis, foul tracking, turnover impact, etc. ESPN endpoint: /sports/{sport}/{league}/playbyplay?event={id} Run: Daily after games complete N)datetimetimezone timedelta)normalize_to_fullz-https://site.api.espn.com/apis/site/v2/sportsg?z User-Agentz>sCAFF G G G G ::>>/ 4 : :3 ? BB  G G    s9 B0B$6B$ B0(B$*B0$B-)B00 B<;B<c|jd|jd|jd|jdy)z!Create PlayByPlay table if neededa CREATE TABLE IF NOT EXISTS "PlayByPlay" ( id BIGSERIAL PRIMARY KEY, league VARCHAR(50) NOT NULL, "gameId" BIGINT, "espnEventId" VARCHAR(50), "gameDate" TIMESTAMP, period INT, "gameClock" VARCHAR(20), "sequenceNumber" INT, "playType" VARCHAR(100), "playText" TEXT, "homeScore" INT, "awayScore" INT, "scoringPlay" BOOLEAN DEFAULT FALSE, "teamId" VARCHAR(50), "team" VARCHAR(20), "playerId" VARCHAR(50), "playerName" VARCHAR(200), "turnover" BOOLEAN DEFAULT FALSE, "foul" BOOLEAN DEFAULT FALSE, "timeout" BOOLEAN DEFAULT FALSE, raw JSONB, "createdAt" TIMESTAMP DEFAULT NOW(), UNIQUE(league, "espnEventId", "sequenceNumber") ) zZCREATE INDEX IF NOT EXISTS "PlayByPlay_league_game_idx" ON "PlayByPlay" (league, "gameId")zQCREATE INDEX IF NOT EXISTS "PlayByPlay_playType_idx" ON "PlayByPlay" ("playType")z]CREATE INDEX IF NOT EXISTS "PlayByPlay_period_idx" ON "PlayByPlay" (league, "gameId", period)N)execute)curs r#ensure_pbp_tabler(-s<KK 6KKlmKKcdKKopc td|d|d|}|dvr|dz } tj|td}|jdk7rgS|j }g}|jdgD]}|jd }|jd ijd ijd } | d k7rK|jdg} | s`| d} | jdg} d} d}| D]M}|jdi}|jddj }|jddk(r|} L|}O|s|j|| ||jdd|S#t$r }gcYd}~Sd}~wwxYw)zGet ESPN event IDs for a date/z/scoreboard?dates=)rzcollege-footballz&groups=50&limit=300headerstimeouteventsidstatustypestatepost competitionsr competitorsNteam abbreviationrhomeAwayhomedate)r2r<awayr=) ESPN_BASErequestsrHEADERS status_codejsonupperappend Exception)r r date_strurlrespdatar1eventevent_idr3r7compr8 home_team away_team competitorr9abbreves r#get_espn_events_for_daterSOs Kqq -?z JCEE %%+||C"=   s "Iyy{XXh+ EyyHYYx,00<@@IF 99^R8L?D((="5KII) ' !~~fb1."5;;=>>*-7 &I &I  ' "%%!IIf- 5 B  s*,E& C3E&=(E&& E;/E60E;6E;ctd|d|d|} tj|td}|jdk7ry|j S#YyxYw)z-Fetch play-by-play data from summary endpointr+z/summary?event=r-r0N)r?r@rrArBrC)r r rLrHrIs r#fetch_play_by_playrVs^Kqq _XJ GC||C"=   s "yy{s+AAAc g}|s|Si}|jdijdgD]R}|jdi}t|jdd}|jddj||<T|jdg}|s|jdi}t|tri|jd g} |jd i} | | r| gngz} | D]6} t| ts| jdg} |j | 8d }|D]$}|d z }|jd i}t|tr|jddn t|}|jddj }d|vxs d|vxsd|v}d|v}d|v}|jdd }|jdd }|jdi}t|tr|jdd nd }|jdi}t|tr|jddn t|}|jdi}t|trt|jddnd}|j|d}|jdg}d}d} |rB|d jdi}!t|!jdd}|!jdd} |jid|d|d|d |d!|jddd"|d#|d$|jd%d&d'|d|d(|d)| d|d|d|d*|'|S)+z/Parse play-by-play data into structured recordsboxscoreteamsr9r2rr:playsdrivespreviouscurrentrrr4textturnoversteal interceptionfoulr/ homeScore awayScoreperiodnumberclock displayValue participantsNathlete displayName game_clocksequence play_type play_text home_score away_score scoring_play scoringPlayFteam_id player_id player_nameraw)rstrrD isinstancedictextendlowerrE)"pbp_dataleaguerZ teams_mapr9 team_infort all_playsr[previous_drives current_drive drive_listdrive drive_playsseqplayrnplay_type_text text_lower is_turnoveris_foul is_timeoutrprqre period_numrgrl team_data team_abbrevrirurvrjs" r# parse_playsrs E  I Z,00"=GHHVR( immD"-.&]]>2>DDF 'G  Wb)I h+ fd #$jjR8O"JJy"5M(}]ORTUJ# 2eT*"'))GR"8K$$[1 2 C5  qHHVR( 6@D6Qvr2WZ[dWeXXfb)//1  J.g'Z2Gg>]gKg J&*, XXk1- XXk1- (B'0:640HVZZ!,a "%6@6MUYY~r2SVW\S] HHVR( 2a SELECT id, "gameDate" FROM "SportsGame" WHERE league = %s AND "homeTeam" = %s AND "awayTeam" = %s AND "gameDate"::date BETWEEN %s AND %s + INTERVAL '1 day' LIMIT 1 Nra INSERT INTO "PlayByPlay" ( league, "gameId", "espnEventId", "gameDate", period, "gameClock", "sequenceNumber", "playType", "playText", "homeScore", "awayScore", "scoringPlay", "teamId", team, "playerId", "playerName", turnover, foul, timeout, raw ) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) ON CONFLICT (league, "espnEventId", "sequenceNumber") DO NOTHING rerlrmrnrorprqrrrtr9rurvr_rbr/rw z z : Processed  games,  playsz: Total ) LEAGUE_CONFIGprintr$psycopg2connectcursorr(commitrnowrutcr=rrDstrftimerSr&fetchonetimesleep REQUEST_DELAYrVrrrCdumpsrFclose)r~ days_backlimitconfigdb_urlconnr'end_date start_date total_games total_playsgames_processed current_daterGr1rKr}rZ home_full away_fullgamegame_id game_daterrRs r#ingest_pbp_for_leaguers ]"  )*Q'' 6 "F ]F   F #D ++-CSKKM||HLL)..0HI955J Bv||~2:,d8* MNKKOL ( "'>((2)&/6-;PRZ[F E%' KK%+& ( ||~a 1$ JJ} %)&/6-;PRWX\R]^H&1E)%-@I)%-@I KK )Y lK  M<<>D!%d1g4G#'Q\I KK !tiX\(:D^@TYftK7H$}J]Z($v,Y 4;/& 1$K+ 2 1 K q O#q(6(,.?x }TZ[\ MF P  q)) Y ( "'>\ KKMIIKJJL Bvhh{m8K= GH ; 77#!s#A1K77 L L ctjd}|jdtdd|jdtdd |jd td d |j }t d t dt dtjtjjt d |jjdDcgc] }|jj"}}d}d}|D]4}t!||j"|j$}||dz }||dz }6t dd t d|d|dt d ycc}w)NzIngest play-by-play from ESPN) descriptionz --leaguesznba,nflzComma-separated leagues)r4defaulthelpz--dayszDays to look backz--limitdzMax games per leaguez<============================================================zINGEST PLAY-BY-PLAY FROM ESPNzTime: ,r)rrrrZrzTOTAL: rr)argparseArgumentParser add_argumentrxint parse_argsrrrrr isoformatleaguesrrr|rrr)parserargslrrrr~results r#mainrYs`  $ $1P QF  #y68 sA02  S35    D (O )* F8<< -779: ;< (O*.,,*<*rs 22 277??277??8#<=>( ;  W X"% 8 6e 4#4M N   CqD1h Vrk8\< zFr)