9i_ Z d Z ddlZddlZddlmZ ddlmZmZmZmZ ddl Z
ddl m Z ddl
mZmZ ddlmZ dd lmZ ej( e Ze G d
d Ze G d d
Z G d d Z G d de Z G d de Z G d de Z G d de Z G d de Z y)zJ
Callbacks to use with the Trainer class and customize the training loop.
N) dataclass)DictListOptionalUnion)tqdm )IntervalStrategy
has_length)TrainingArguments)loggingc T e Zd ZU dZdZee ed< dZe ed< dZ
e ed< dZ e ed< dZ e ed < dZ
e ed
< dZe ed < dZeed < dZeeeef ed
< dZee ed< dZee ed< dZeed< dZeed< dZeed< dZeed< dZeee eee ef f ed< d Z defdZ e defd Z y) TrainerStateaN
A class containing the [`Trainer`] inner state that will be saved along the model and optimizer when checkpointing
and passed to the [`TrainerCallback`].
In all this class, one step is to be understood as one update step. When using gradient accumulation, one update
step may require several forward and backward passes: if you use `gradient_accumulation_steps=n`, then one update
step requires going through *n* batches.
Args:
epoch (`float`, *optional*):
Only set during training, will represent the epoch the training is at (the decimal part being the
percentage of the current epoch completed).
global_step (`int`, *optional*, defaults to 0):
During training, represents the number of update steps completed.
max_steps (`int`, *optional*, defaults to 0):
The number of update steps to do during the current training.
logging_steps (`int`, *optional*, defaults to 500):
Log every X updates steps
eval_steps (`int`, *optional*):
Run an evaluation every X steps.
save_steps (`int`, *optional*, defaults to 500):
Save checkpoint every X updates steps.
total_flos (`float`, *optional*, defaults to 0):
The total number of floating operations done by the model since the beginning of training (stored as floats
to avoid overflow).
log_history (`List[Dict[str, float]]`, *optional*):
The list of logs done since the beginning of training.
best_metric (`float`, *optional*):
When tracking the best model, the value of the best metric encountered so far.
best_model_checkpoint (`str`, *optional*):
When tracking the best model, the value of the name of the checkpoint for the best model encountered so
far.
is_local_process_zero (`bool`, *optional*, defaults to `True`):
Whether or not this process is the local (e.g., on one machine if training in a distributed fashion on
several machines) main process.
is_world_process_zero (`bool`, *optional*, defaults to `True`):
Whether or not this process is the global main process (when training in a distributed fashion on several
machines, this is only going to be `True` for one process).
is_hyper_param_search (`bool`, *optional*, defaults to `False`):
Whether we are in the process of a hyper parameter search using Trainer.hyperparameter_search. This will
impact the way data will be logged in TensorBoard.
Nepochr global_step max_stepsi
logging_steps
eval_steps
save_stepsnum_train_epochs
total_flos log_history best_metricbest_model_checkpointTis_local_process_zerois_world_process_zeroFis_hyper_param_search
trial_name trial_paramsc , | j g | _ y y N)r selfs g/var/www/html/backtest/airagagent/rag_env/lib/python3.12/site-packages/transformers/trainer_callback.py
__post_init__zTrainerState.__post_init__d s # !D $ json_pathc t j t j | dd dz }t |dd 5 }|j | ddd y# 1 sw Y yxY w) zDSave the content of this instance in JSON format inside `json_path`. T)indent sort_keys
wutf-8encodingN)jsondumps dataclassesasdictopenwrite)r# r' json_stringfs r$ save_to_jsonzTrainerState.save_to_jsonh sT jj!3!3D!9!tTW[[
)S7
3 !q
GGK ! ! !s AA!c t |dd 5 }|j }ddd | di t j S # 1 sw Y $xY w)z3Create an instance from the content of `json_path`.rr. r/ N )r5 readr1 loads)clsr' r8 texts r$ load_from_jsonzTrainerState.load_from_jsonn sI )S7
3 q668D &TZZ%&& s AA )!__name__
__module__ __qualname____doc__r r float__annotations__r intr r r r r r r r r strr r r boolr r r r r r% r9 classmethodrA r< r& r$ r r " s
-^ "E8E?!KIs M3 JJ c J*.Kd3:&'.#'K% '+/8C=/"&4&"&4&"'4'J<@L$sE#uc4"7 889@"!c ! 's ' 'r& r c j e Zd ZU dZdZeed< dZeed< dZeed< dZ eed< dZ
eed< d Z d Z d
Z
y ) TrainerControlaA
A class that handles the [`Trainer`] control flow. This class is used by the [`TrainerCallback`] to activate some
switches in the training loop.
Args:
should_training_stop (`bool`, *optional*, defaults to `False`):
Whether or not the training should be interrupted.
If `True`, this variable will not be set back to `False`. The training will just stop.
should_epoch_stop (`bool`, *optional*, defaults to `False`):
Whether or not the current epoch should be interrupted.
If `True`, this variable will be set back to `False` at the beginning of the next epoch.
should_save (`bool`, *optional*, defaults to `False`):
Whether or not the model should be saved at this step.
If `True`, this variable will be set back to `False` at the beginning of the next step.
should_evaluate (`bool`, *optional*, defaults to `False`):
Whether or not the model should be evaluated at this step.
If `True`, this variable will be set back to `False` at the beginning of the next step.
should_log (`bool`, *optional*, defaults to `False`):
Whether or not the logs should be reported at this step.
If `True`, this variable will be set back to `False` at the beginning of the next step.
Fshould_training_stopshould_epoch_stop should_saveshould_evaluate
should_logc d| _ y)zInternal class that just calls the list of callbacks in order.c g | _ |D ] }| j | || _ || _ || _ || _ d | _ d | _ t d | j D s#t j d| j z y y )Nc 3 < K | ] }t |t y wr! )
isinstanceDefaultFlowCallback.0cbs r$ z+CallbackHandler.__init__..: s P2:b"56Ps zThe Trainer will not work properly if you don't have a `DefaultFlowCallback` in its callbacks. You
should add one before training with `trainer.add_callback(DefaultFlowCallback). The current list ofcallbacks is
:) callbacks add_callbackmodel tokenizer optimizer lr_schedulertrain_dataloadereval_dataloaderanyloggerwarning
callback_list)r# r r r r r r s r$ __init__zCallbackHandler.__init__/ s "B b ! "
"" ( $ # P PP NN$ $$%
Qr& c P t |t r | n|}t |t r|n |j }|| j D cg c] }|j c}v r)t j d| ddz | j z | j j | y c c}w )NzYou are adding a zH to the callbacks of this Trainer, but there is already one. The currentzlist of callbacks is
:)r type __class__r r r r append)r# callbackr cb_classcs r$ r z CallbackHandler.add_callbackB s %h5XZ8 )(D 98x?Q?Q T^^< < < NN#H:-uv+,$$%
b!
=s B#c t |t r=| j D ]- }t || s| j j | |c S y | j D ]& }||k( s | j j | |c S y r! r r r remover# r r s r$ pop_callbackz CallbackHandler.pop_callbackM ss h %nn
b(+NN))"- I
nn
>NN))"- I
r& c t |t r;| j D ]+ }t || s| j j | y y | j j | y r! r r s r$ remove_callbackz CallbackHandler.remove_callbackY sR h %nn
b(+NN))"-
NN ! !( +r& c F dj d | j D S )Nr, c 3 H K | ] }|j j y wr! )r rB r s r$ r z0CallbackHandler.callback_list..d s H2 ..Hs ")joinr r" s r$ r z CallbackHandler.callback_listb s yyH HHHr& r[ r\ r] c * | j d||| S )Nra
call_eventr# r[ r\ r] s r$ ra zCallbackHandler.on_init_endf }dE7CCr& c 8 d|_ | j d||| S )NFrd )rN r r s r$ rd z CallbackHandler.on_train_begini s ',$ /ugFFr& c * | j d||| S )Nrf r r s r$ rf z CallbackHandler.on_train_endm ~tUGDDr& c 8 d|_ | j d||| S )NFrh )rO r r s r$ rh z CallbackHandler.on_epoch_beginp s $)! /ugFFr& c * | j d||| S )Nrj r r s r$ rj z CallbackHandler.on_epoch_endt r r& c T d|_ d|_ d|_ | j d||| S )NFrl )rR rQ rP r r s r$ rl z CallbackHandler.on_step_beginw s/ ""' #eWEEr& c * | j d||| S )Nro r r s r$ ro z CallbackHandler.on_substep_end} s /ugFFr& c * | j d||| S )Nrq r r s r$ rq zCallbackHandler.on_step_end r r& c < d|_ | j d|||| S )NFrs ru )rQ r r# r[ r\ r] ru s r$ rs zCallbackHandler.on_evaluate s# "' }dE7GTTr& c . | j d|||| S )Nrv r r r s r$ rv zCallbackHandler.on_predict s |T5'7SSr& c 8 d|_ | j d||| S )NFrx )rP r r s r$ rx zCallbackHandler.on_save s #y$w??r& c < d|_ | j d|||| S )NFrz )logs)rR r )r# r[ r\ r] r s r$ rz zCallbackHandler.on_log s# "xugDIIr& c * | j d||| S )Nr| r r s r$ r| z"CallbackHandler.on_prediction_step s 3T5'JJr& c | j D ]_ } t || |||f| j | j | j | j
| j | j d|}|^|}a |S )N)r r r r r r )r getattrr r r r r r )r# eventr[ r\ r] r` r results r$ r zCallbackHandler.call_event s !H-WXu- jj .. .. ! . .!%!6!6 $ 4 4 F ! ! r& N)rB rC rD rE r r r r propertyr r r rM ra rd rf rh rj rl ro rq rs rv rx rz r| r r< r& r$ r~ r~ , s H& "
, I ID 1 D, DQ_ DG#4 G\ GTb GE!2 E< ER` EG#4 G\ GTb GE!2 E< ER` EF"3 FL FSa F G#4 G\ GTb GD 1 D, DQ_ DU 1 U, UQ_ UT 0 T TP^ T@ - @l @^ @J, J\ JN JK'8 K KXf Kr& r~ c 8 e Zd ZdZdededefdZdededefdZy)r zx
A [`TrainerCallback`] that handles the default flow of the training loop for logs, evaluation and checkpoints.
r[ r\ r] c Z |j dk( r|j rd|_ |j t j
k( r#|j |j z dk( rd|_ |j t j
k( r<|j |j z dk( r |j |j k rd|_
|j t j
k( r2|j dkD r#|j |j z dk( rd|_
|j |j k\ rd|_ |S )Nr Tr )r logging_first_steprR logging_strategyr
STEPSr evaluation_strategyr
eval_delayrQ
save_strategyr rP r rN r_ s r$ rq z DefaultFlowCallback.on_step_end s !d&=&=!%G $4$:$: :u?P?PSXSfSf?fjk?k!%G
$ $(8(>(> >!!E$4$4495#4#44&*G #
"2"8"8 8 1$!!E$4$449"&G /+/G (r& c |j t j k( rd|_ |j t j k( r |j
|j k rd|_ |j t j k( rd|_ |S )NT)
r r
EPOCHrR r r r rQ r rP r_ s r$ rj z DefaultFlowCallback.on_epoch_end st $4$:$: :!%G # #'7'='= =$//UZU`U`B`&*G # !1!7!7 7"&G r& N) rB rC rD rE r r rM rq rj r< r& r$ r r s? 1 , Q_ :
!2
<
R`
r& r c D e Zd ZdZd Zd Zd Zd dZd Zd Z d d Z
d
Z y) ProgressCallbackzU
A [`TrainerCallback`] that displays the progress of training or evaluation.
c d | _ d | _ y r! ) training_barprediction_barr" s r$ r zProgressCallback.__init__ s "r& c b |j r t |j d | _ d| _ y )NT)total
dynamic_ncolsr )r r r r current_stepr_ s r$ rd z ProgressCallback.on_train_begin s& & & $5??$ OD r& c |j rD| j j |j | j z
|j | _ y y r! )r r updater r r_ s r$ rq z ProgressCallback.on_step_end sC & & $ $U%6%69J9J%J K % 1 1D 'r& Nc |j r\t | rP| j (t t | | j
d u d | _ | j j
d y y y )NT)r leaver r )r r r r lenr r )r# r[ r\ r] r r` s r$ r| z#ProgressCallback.on_prediction_step sb & &:o+F""*&* o.d6G6G46O_c'#
& &q ) ,G &r& c x |j r.| j | j j d | _ y y r! r r closer_ s r$ rs z ProgressCallback.on_evaluate 6 & &"".##))+"&D 'r& c x |j r.| j | j j d | _ y y r! r r_ s r$ rv zProgressCallback.on_predict r r& c |j rD| j 7|j dd }| j j t | y y y Nr )r r popr6 rI r# r[ r\ r] r r` _s r$ rz zProgressCallback.on_log sG & &4+<+<+H t,A # #CI . ,I &r& c ` |j r"| j j d | _ y y r! )r r r r_ s r$ rf z ProgressCallback.on_train_end s* & & # # % $D 'r& r! ) rB rC rD rE r rd rq r| rs rv rz rf r< r& r$ r r s/ #
2
*' ' /
%r& r c e Zd ZdZddZy)PrinterCallbackz?
A bare [`TrainerCallback`] that just prints the logs.
Nc X |j dd }|j r t | y y r )r r printr s r$ rz zPrinterCallback.on_log s' HH\4 ( & & $K 'r& r! )rB rC rD rE rz r< r& r$ r r
s r& r c : e Zd ZdZd dedee fdZd Zd Z d Z
y)
EarlyStoppingCallbacka1
A [`TrainerCallback`] that handles early stopping.
Args:
early_stopping_patience (`int`):
Use with `metric_for_best_model` to stop training when the specified metric worsens for
`early_stopping_patience` evaluation calls.
early_stopping_threshold(`float`, *optional*):
Use with TrainingArguments `metric_for_best_model` and `early_stopping_patience` to denote how much the
specified metric must improve to satisfy early stopping conditions. `
This callback depends on [`TrainingArguments`] argument *load_best_model_at_end* functionality to set best_metric
in [`TrainerState`]. Note that if the [`TrainingArguments`] argument *save_steps* differs from *eval_steps*, the
early stopping will not occur until the next save step.
early_stopping_patienceearly_stopping_thresholdc . || _ || _ d| _ y )Nr )r r early_stopping_patience_counter)r# r r s r$ r z EarlyStoppingCallback.__init__) s '>$(@%/0,r& c |j rt j nt j }|j 8 |||j r-t ||j z
| j kD rd| _ y | xj dz
c_ y )Nr r )greater_is_betternpgreaterlessr absr r )r# r[ r\ r] metric_valueoperators r$ check_metric_valuez(EarlyStoppingCallback.check_metric_value/ sl !%!7!72::RWW $ \5#4#4 5L5#4#4458U8UU34D 0 0 0A 5 0r& c |j sJ d |j J d |j t j k7 sJ d y )Nzr s ! . . 7 ,
H % P' P' P'f . . . bA
A
Hzo zz// /d/% /%do :0O :0r&