i*< <SSKrSSKrSSKrSSKJr SSKJrJrJrJ r "SS\5r SSK J r SSK JrJrJr SrSS jrSSS SS .S \\\\\\ \\\/\4444S jjjr\R0"\5r\"5"SS55rg!\a S rN]f=f)N)Enum)AnyCallableOptionalUnionc4\rSrSrSrSrSrSrSrSr Sr S r S r g ) RequestStatusz5Status of a generation request through its lifecycle.pending prefillingprefilling_splitsplit_pending_remainderdecodingfinishedfailedN) __name__ __module__ __qualname____firstlineno____doc__PENDING PREFILLINGPREFILLING_SPLITSPLIT_PENDING_REMAINDERDECODINGFINISHEDFAILED__static_attributes__rj/home/dmtnaga/Documents/work/airagagent/rag_env/lib/python3.13/site-packages/transformers/utils/metrics.pyr r s*?GJ)7HH Fr r )metrics)Status StatusCode get_tracerTFc0^[(dS$U4SjnU$)a Decorator that attaches a tracer to a class. This decorator should be applied to classes that need OpenTelemetry tracing. It adds a tracer attribute to the class instance that can be used by the traced decorator. Args: tracer_name_template: Optional template string for the tracer name. If provided, it should contain {module} which will be replaced with the class's full module path and {class_name} for the class name. If None, a default naming scheme will be used where: - If the module already starts with "transformers.", it will use that directly - Otherwise, it will prepend "transformers." to the module name Returns: Class decorator function cU$Nr)clss r!attach_tracer..0s3r ct>^^TRm[R"T5UUU4Sj5nUTlT$)Nc>T"U/UQ70UD6 TRnTRnTc'URS5(aUSU3nOSUSU3nOTRX4S9n[ U5Ulg)N transformers..)module class_name)rr startswithformatr%tracer) selfargskwargs module_namer1 tracer_namer) original_inittracer_name_templates r!init_with_tracer:attach_tracer..decorator..init_with_tracer5s $ 0 0 0..K))J#+))/::%0M:,"?K$1+a |"LK2999d $[1DKr )__init__ functoolswraps)r)r<r:r;s` @r! decorator attach_tracer..decorator2s5  ' 2 ( 2 (  r )_has_opentelemetry)r;rAs` r! attach_tracerrDs$  . r ) span_name standaloneadditional_attributesrGc2^^^UUU4SjnUcU$U"U5$)a^ Decorator to trace function calls with OpenTelemetry. Can be used as @traced or @traced(span_name="custom_name") Args: func: The function to trace span_name: Optional custom name for the span (defaults to function name) standalone: If True, creates a parentless span additional_attributes: Optional list of additional attributes to set on the span. Each item is a tuple of (instance_attribute_name, span_attribute_key, value_or_transform_function) where: - instance_attribute_name: Name of the attribute to get from the class instance - span_attribute_key: Key to use when setting the attribute on the span - value_or_transform_function: Either a raw value to use directly, or a function to transform the attribute value before setting it on the span Returns: Decorated function with tracing ch>^[(dT$[R"T5UUUU4Sj5nU$)Nc >U(a#[TS5(aTRbUSOSnUSLnU(a[US5(a URnO%[STRSTR 35nT=(d TR nT(a UR O URnU"U5nURSTR 5 URSTR5 URSU5 U(a[U5Htup[U [[[[45(dU c URS U3[U 55 MMURS U3[[U 555 Mv U(aUR!5Htup[U [[[[45(dU c URS U 3[U 55 MMURS U 3[[U 555 Mv T(a_U(aXTHRn U upn[X-5(dM[#X-5n[%U5(a U"U5nOUnURUU5 MT T"U0UD6nUsSSS5 $![&a?nUR)[+[,R.55 UR1U5 eSnAff=f!,(df  g=f) N__self__rr4r.r/z function.namezfunction.modulezfunction.is_methodzargs.zkwargs.)hasattrrKr4r%rr start_spanstart_as_current_span set_attribute enumerate isinstancestrintfloatbooltypeitemsgetattrcallable Exception set_statusr#r$ERRORrecord_exception)r6r7instance is_methodr4namespan_fnspaniargkeyvalue attr_configinstance_attribute_namespan_attribute_keyvalue_or_transform_functionattribute_valuetransformed_valueresulterGfuncrErFs r!wrapper*traced..decorator..wrapperlsa"&GD*,E,E$--JctAwjnH ,IWXx88!#mDOO3DAdmm_$UV- D+5f''6;W;WG$""?DMMB""#4dooF""#7C"+D/%cCeT+BCCs{ ..qc{CHE ..qc{CS NK #2 &,lln %ec3t-DEE ..E K ..DK@PQ '5 )Y'< cn`/E`"8EE.5h.XO'(CDD4OP_4` 14O 1 ../ACTU(=!4262F!?@!OOF:+;+;$<=))!,As18F K".decoratorhs1!!K  .  . `r r)rorErFrGrAs ``` r!tracedrsLs 85n | T?r c\rSrSrSrS\4SjrSr\S\ S\ SS 4S j5r \S \ SS 4S j5r \SS j5r\S\S\SS 4Sj5r\S\ S\ SS 4Sj5rSrg )ContinuousBatchProcessorMetricsz0Metrics collection for ContinuousBatchProcessor.max_batch_tokensc0XlUR5 g)ztInitialize metrics for continuous batch processor. Args: max_batch_tokens: Maximum number of tokens in a batch N)rw_setup_metrics)r5rws r!r>(ContinuousBatchProcessorMetrics.__init__s !1 r c[(d[RS5 g[R"S5Ul/SQnUR R SSSUS9UlUR RS S S S 9Ul UR RS SS S 9Ul /SQnUR R SSSUS9Ul UR RSSSS 9Ul UR RSSSS 9UlUR RSSSS 9Ul/SQnUR R SSSUS9UlUR RSSS S 9UlUR RS!S"S S 9Ulg)#zIInitialize OpenTelemetry metrics and tracing if the library is available.zIOpenTelemetry is not installed. Metrics and tracing will not be recorded.Nz2transformers.generation.continuous_batch_processor) 2Kdi,i'ttft_millisecondsz#Time to first token in millisecondsms)r` descriptionunit#explicit_bucket_boundaries_advisoryactive_requests_countz3Number of active requests currently being processedrequests)r`rrwaiting_requests_countz*Number of requests waiting to be processed) r~rrrrrri Ni0ui`request_latency_millisecondsz9End-to-end latency for completed requests in millisecondsdecode_prefill_ratioz3Ratio of decode tokens to prefill tokens in a batchratioprefill_tokens_processedz"Number of prefill tokens processedtokensdecode_tokens_processedz!Number of decode tokens processed) r|(r~<FPZ_brbatch_fill_percentagez5Percentage of max_batch_tokens utilized in each batchpercentkv_cache_free_memory_bytesz/Free memory of the PagedAttentionCache in bytesbyteskv_cache_memory_bytesz0Memory usage of the PagedAttentionCache in bytes)rCloggerinfor" get_metermetercreate_histogramttft_histogram create_gaugeactive_requests_gaugewaiting_requests_gaugerequest_latency_histogramdecode_prefill_ratio_gaugecreate_counterprefill_tokens_counterdecode_tokens_counterbatch_fill_percentage_histogramkv_cache_free_memory_gaugekv_cache_memory_gauge)r5 ttft_bucketslatency_bucketsbatch_fill_bucketss r!ry.ContinuousBatchProcessorMetrics._setup_metricss"! KKc d &&'[\ _ "jj99$=0< : &*ZZ%<%<(M&=& " '+jj&=&=)D'>' #\)-)D)D/S0? *E* &+/***A*A'M+B+ ' '+jj&?&?+<'@' # &*ZZ%>%>*;&?& "R/3zz/J/J(O0B 0K0 ,+/***A*A-I+B+ ' &*ZZ%<%<(J&=& "r created_time request_idreturnNc"[(dg[R"5U- S-nURRU5 [R SUSUSS35 g![ a"n[RSU35 SnAgSnAff=f)zRecord Time to First Token (TTFT). Args: created_time: The time the request was created request_id: The ID of the request N@@zRecorded TTFT for request : .2frzFailed to record TTFT metric: )rCtimerrecordrdebugrZwarning)r5rrttft_msrns r!record_ttft_metric2ContinuousBatchProcessorMetrics.record_ttft_metrics"! 99;-7 A    & &w / LL5j\GC=PRS T A NN;A3? @ @ A8A"" B,B  Brequests_in_batchc  [(aU(dgSnSnUHpnUR[R:XaUS- nM(UR[R[R 4;dMXU[ UR5- nMr X#-nUS:aURRU5 US:aURRU5 US:aX#- nURRU5 XPR- S-nURRU5 [ R#SUSUSUSS US URS 3 5 g![$a"n[ R'S U35 SnAgSnAff=f) zRecord metrics about the batch composition including decode/prefill ratio and batch fill percentage. Args: requests_in_batch: List of request states in the current batch NrgY@zBatch metrics: z decode tokens, z prefill tokens, batch fill: rz% (/)z Failed to record batch metrics: )rCstatusr rrrlen prompt_idsraddrrsetrwrrrrrZr) r5r decode_tokensprefill_tokensstatetotal_batch_tokensrfill_percentagerns r!record_batch_metrics4ContinuousBatchProcessorMetrics.record_batch_metricssx"!):  &E||}555" -":":M>Fc h[(dgURUR-nX!RR-nS[ UR 5-UR-U-nUR5nURU- nXd-nXT-nURRU5 URRU5 [RSUS- SSUSURSXaR- S -S S 3 5 g![a"n [R!S U 35 Sn A gSn A ff=f) zRecord memory usage of the PagedAttentionCache without GPU synchronization. This calculates the theoretical memory usage based on cache configuration and the number of blocks currently in use. Args: cache: The PagedAttentionCache object to measure NzKV Cache memory: irzMB, Used blocks: rz (rz.1fz%)z*Failed to record KV cache memory metrics: )rChead_dimnum_key_value_headsdtypeitemsizer key_cache block_sizeget_num_free_blocks num_blocksrrrrrrZr) r5cache page_sizepage_mem_in_bytesblock_mem_in_bytes free_blocks used_blocksused_memory_bytesfree_memory_bytesrns r!record_kv_cache_memory_metrics>ContinuousBatchProcessorMetrics.record_kv_cache_memory_metricsBsD"!  M)B)BBI )KK,@,@ @ "#S%9!9E.>-?@"2"22S8=RA   M NNGsK L L MsC6D D1D,,D1active_requestswaiting_requestsc [(dgURRU5 URRU5 [R SUSUS35 g![ a"n[RSU35 SnAgSnAff=f)zRecord metrics about active and waiting requests. Args: active_requests: Number of active requests waiting_requests: Number of waiting requests NzQueue metrics: z active requests, z waiting requestsz Failed to record queue metrics: )rCrrrrrrZr)r5rrrns r!record_queue_metrics4ContinuousBatchProcessorMetrics.record_queue_metricsjs"!  C  & & * *? ;  ' ' + +,< = LL??*;;MN^M__pq r C NN=aSA B B CsAA!! B +BB c"[(dg[R"5U- S-nURRU5 [R SUSUSS35 g![ a"n[RSU35 SnAgSnAff=f)zRecord metrics about a completed request. Args: created_time: The time the request was created request_id: The ID of the request Nrz Recorded request completion for rrrz,Failed to record request completion metric: )rCrrrrrrZr)r5rr latency_msrns r!record_request_completion9ContinuousBatchProcessorMetrics.record_request_completion|s"! iikL0F:  O  * * 1 1* = LL;JryrsrTrRrlistrrrrrrr r!rurus:O b AuA#A$A A$ &Cd&Ct&C &CP %M %MN CCC3CSWC C" OeOOQUO Or rur()r?loggingrenumrtypingrrrrr opentelemetryr"opentelemetry.tracer#r$r%rC ImportErrorrDrtuplerRrs getLoggerrrrurr r!rs 11 D %BB ,` U_c U $DsCsHcUTWZDX?X9Y/Y)Z$[\ Up   8 $fOfOfO_sBBB