;iLSSKJr SSKrSSKrSSKrSSKrSSKrSSKJr SSK J r SSK J r J r SSKrSSKJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJ r J!r!J"r"J#r#J$r$J%r%J&r&J'r'J(r(J)r) SSK*J+r+ \$"5(aSSK,J-s J.r/ SSK0J1r2 \"S S 9(aSSK3r3\#"S S 9(aSSK4r4\!"S S 9(aSSK5r5\""S S 9(aSSK6r6\Rn"\85r9SS jr:S r;"S S\Rx5r=\$"5(d\>O\=r?"SS5r@"SS5rA"SS5rBg)) annotationsN)contextmanager)partial)AnyCallable)DistributedType DynamoBackendGradientAccumulationPlugincheck_cuda_fp8_capabilitycheck_cuda_p2p_ib_supportdeepspeed_requiredget_cpu_distributed_informationget_int_from_envis_ccl_availableis_datasets_availableis_deepspeed_availableis_fp8_availableis_habana_gaudi1is_hpu_availableis_ipex_availableis_mlu_availableis_mps_availableis_musa_availableis_npu_availableis_sdaa_availableis_torch_xla_availableis_xccl_availableis_xpu_availableparse_choice_from_envparse_flag_from_envset_numa_affinity)SageMakerDistributedTypeF) check_devicec([R0:g$)z Checks if the `AcceleratorState` has been initialized from `Accelerator`. Same as `AcceleratorState.initialized`, but works as a module method. )AcceleratorState _shared_state`/home/dmtnaga/Documents/work/airagagent/rag_env/lib/python3.13/site-packages/accelerate/state.pyis_initializedr+Os  ) )R //r)cgNr()argskwargss r* do_nothingr0Xs r)c6\rSrSrSrSS SjjrS SjrSrSrg) ThreadLocalSharedDict\a Descriptor that holds a dict shared between instances of a class in the same thread. Note: Descriptors have slightly different semantics than just a dict field on its own. `PartialState(...)._shared_state` and `PartialState._shared_state` (instance vs class) give the same value: the underlying _storage dict. Likewise, `PartialState(...)._shared_state = {...}` overrides the _storage dict inside the descriptor as you would expect. However, `PartialState._shared_state = {}` actually replaces the descriptor object with a dict instead Thus, you should modify the _storage dict in-place (e.g. `_shared_state.clear()`). See Python documentation for an explanation of descriptors: https://docs.python.org/3/howto/descriptor.html This is required for using PyTorch/XLA with PJRT in multithreaded mode (required for TPU v2 and v3). See https://github.com/pytorch/xla/blob/r2.0/docs/pjrt.md#multithreading-on-tpu-v2v3 c0Ulgr-_storage)self thread_locals r*__init__ThreadLocalSharedDict.__init__ms  r)NcUR$r-r5)r7objobjtypes r*__get__ThreadLocalSharedDict.__get__ps }}r)cX lgr-r5)r7r<values r*__set__ThreadLocalSharedDict.__set__ss r)r5F)r8boolr-) __name__ __module__ __qualname____firstlineno____doc__r9r>rB__static_attributes__r(r)r*r2r2\s r)r2c\rSrSrSr\"5r/SQrSSSjjrS Sjr \ S5r \ S!Sj5r \ S5r\ S!S j5r\ S!S j5r\ S!S j5rS rS"S jr\SS#Sjj5r\S5r\S5rS$S%SjjrS$S%SjjrS&SjrS'S(SjjrS'S)SjjrSr\ S*Sj5rS+S,SjjrSr S$Sjr!S-Sjr"Sr#g). PartialState|a Singleton class that has information about the current training environment and functions to help with process control. Designed to be used when only process control and device execution states are needed. Does *not* need to be initialized from `Accelerator`. Args: cpu (`bool`, *optional*): Whether or not to force the script to execute on CPU. Will ignore any accelerators available if set to `True` and force the execution on the CPU. kwargs (additional keyword arguments, *optional*): Additional keyword arguments to pass to the relevant `init_process_group` function. Valid `kwargs` can be found in [`utils.InitProcessGroupKwargs`]. See the example section for detailed usage. **Available attributes:** - **device** (`torch.device`) -- The device to use. - **distributed_type** ([`~accelerate.state.DistributedType`]) -- The type of distributed environment currently in use. - **local_process_index** (`int`) -- The index of the current process on the current server. - **mixed_precision** (`str`) -- Whether or not the current script will use mixed precision, and if so the type of mixed precision being performed. (Choose from 'no','fp16','bf16 or 'fp8'). - **num_processes** (`int`) -- The number of processes currently launched in parallel. - **process_index** (`int`) -- The index of the current process. - **is_last_process** (`bool`) -- Whether or not the current process is the last one. - **is_main_process** (`bool`) -- Whether or not the current process is the main one. - **is_local_main_process** (`bool`) -- Whether or not the current process is the main one on the local node. - **debug** (`bool`) -- Whether or not the current script is being run in debug mode. Example: ```python from accelerate.utils import InitProcessGroupKwargs # To include `InitProcessGroupKwargs`, init then call `.to_kwargs()` kwargs = InitProcessGroupKwargs(...).to_kwargs() state = PartialState(**kwargs) ``` ) _cpu_mixed_precisionr'backenddebugdevicedistributed_type fork_launchedlocal_process_index num_processes process_indexc 8URUlUR(GdfXlSUl[ R RSS5nUb[R"U5OSUl [S5Ul URSS5nSnUch[ R RSS5R5S:H=(a0 [ R RS5[R:gnURSS5nUR!XU5upxUbXv:wa[#S US U35eXplXlS n U(GdURS :wGaz['[ R RS S55S:wGaL[ R RSS5R5S:Xa[)5(d [+S5eSSKJn U R15(dqURS:XaB[ R RS S5n [R2R5SU 35 U R6"S4URS S.UD6 Sn GOpUR$[8R:[8R<4;GaA[R>R15(GdURS:XaB[ R RS S5n [R2R5SU 35 URS:Xa[ R RSS5R5S:Xa][ R RSS5R5S:Xd$[ R RSS5S:XaSUl[R>R@"S4SUR0UD6 UR$[8R:[8R<4;Ga1[C5n[EURF5[ R S'[EURH5[ R S'[EURJ5[ R S '[EURL5[ R S '[ R RS!S5(dS"[ R S!'[ R RS#S5(d5URLURH:waURS$:wa [#S%5eURFUS&'URHUS''UR$[8R<:Xar[OS(/S5S:XaaSSK(n ['U RSS S)9URL- 5n U S:XaS*n [RT"U 5 [VRX"S+U S,35 [R>R15(d,[R>R@"S4SUR0UD6 URc,[8RUlS*Ul-SUl.SUl/GOcURS :XaUR55 [`Rb"UR[`Rd"55 [fRH"5Ul-[fRh"5Ul.[kSS-9(a[`Rl"5Ul/O['[ R RS S55Ul/O[R>Ro5Ul-[R>Rq5Ul.Uc)['[ R RS S55O URJUl/UR55 U (a[8RrUl[S.S 5(a[uUR^5 URRvS/:XaB[y5(d3S0[ R ;dS1[ R ;a [{S25e[S3S5Ul>g)5NACCELERATE_TORCH_DEVICEACCELERATE_DEBUG_MODE_use_sagemaker_dpACCELERATE_USE_SAGEMAKERfalsetrue%ACCELERATE_SAGEMAKER_DISTRIBUTED_TYPErQzYour assigned backend z is not available, please use Fxla LOCAL_RANKACCELERATE_USE_DEEPSPEEDz_DeepSpeed is not available => install it using `pip3 install deepspeed` or build it from sourcer)commtcclzsdaa:) dist_backendauto_mpi_discoveryTncclACCELERATE_USE_FSDPFSDP_OFFLOAD_PARAMSFSDP_STATE_DICT_TYPESHARDED_STATE_DICTFULL_STATE_DICTzcuda:nccl,cpu:glooRANK WORLD_SIZELOCAL_WORLD_SIZE MASTER_PORT29500 MASTER_ADDRmpizTried to launch on distributed with multinode, but `MASTER_ADDR` env was not set, please try exporting rank 0's hostname as `MASTER_ADDR`rank world_sizeOMP_NUM_THREADS)logicalrz4OMP_NUM_THREADS/MKL_NUM_THREADS unset, we set it at z to improve oob performance. check_is_tpuACCELERATE_CPU_AFFINITYcudaNCCL_P2P_DISABLENCCL_IB_DISABLEzUsing RTX 4000 series doesn't support faster communication broadband via P2P or IB. Please set `NCCL_P2P_DISABLE="1"` and `NCCL_IB_DISABLE="1" or use `accelerate launch` which will do this automatically. FORK_LAUNCHEDr()?r'__dict__ initializedrOrQosenvirongettorchrSr!rRpoplowerr#NO_prepare_backend ValueErrorrTintr ImportError deepspeedrer+sdaa set_deviceinit_distributedr MULTI_XPU MULTI_CPU distributedinit_process_grouprstrrvrw local_ranklocal_world_sizerpsutil cpu_countset_num_threadswarningswarnrWrXrVxmset_replicationget_xla_supported_devicesxrglobal_ordinalrget_local_ordinalget_world_sizeget_rank DEEPSPEEDr"typer NotImplementedErrorrU)r7cpur/ env_deviceuse_sagemaker_dpdist_informationoriginal_backendrQrT use_deepspeeddistrrnum_cpu_threads_per_processs r*r9PartialState.__init__s** IDL(A4HJ6@6L%,,z2RVDK,-DEDJ%zz*=tD # 'JJNN#=wGMMOSYYo 'NOSkSnSnn! &zz)T: (,(=(=cUe(f %G+0K #9:J9KKijqir!stt"L$4 !!M4<<50rzz~~lB78B>zz~~&@'JPPRV\\577"-!B#;#2244#||v5-/ZZ^^L"-M % 5 5j\6J K 11pt||`epiop(, --o6O6OQ`QjQj5kk % 1 1 @ @ B B<<61)+ b)IJ!JJ11E*2FG LLF2 " /Dg N T T VZ` ` " /Dg N T T VZ` `#%::>>2HJ^#_ct#t,@DL))<<\T\\\U[\$$)B)BOD]D](^^#B#D %()9)>)>%? 6"+./?/J/J+K <(+./?/J/J+K <(145E5V5V1W -.zz~~mT::07BJJ}- }d;;(99=M=X=XX -$R"2!6!6v'7'B'B|$))_-F-FF(*;)>,PR3S/TD,%*%6%6%E%E%G"%*%6%6%?%?%A"=M=UC |R89[k[v[v( OO (7(A(A%##aeamam PTP\P\@\su>vw"0012"0012$$($<$<#=>{{m2  ' r)c@[RR5 gzCResets `_shared_state`, is used internally and should not be calledN)rMr'clearr(r)r* _reset_statePartialState._reset_stateKs ""((*r)c UR0:g$)z7Returns whether the `PartialState` has been initialized)r'rs r*rPartialState.initializedPs!!R''r)chUR[R:g=(a URS:$)@ Whether the Accelerator is configured for distributed training r)rTr rrWrs r*use_distributedPartialState.use_distributedUs, $$(:(::Ut?Q?QTU?UUr)c:URURS- :H$)3Returns whether the current process is the last oner)rXrWrs r*is_last_processPartialState.is_last_process\s !!T%7%7!%;;;r)ctUR[R:waURS:H$UR$)7Returns whether the current process is the main processr)rTr MEGATRON_LMrXrrs r*is_main_processPartialState.is_main_processas8(,'<'<@[@['[D  ! # aeauau r)ctUR[R:waURS:H$UR$)IReturns whether the current process is the main process on the local noder)rTr rrVrrs r*is_local_main_process"PartialState.is_local_main_processhs= $$(C(CC  $ $ ) %% r)c  UR[R[R[R[R [R [R[R[R[R[R4 ;a)[RRUR/S9 gUR[R :Xa["R$"S5 gg)aT Will stop the execution of the current process until every other process has reached that point (so this does nothing when the script is only run in one process). Useful to do before saving a model. Example: ```python >>> # Assuming two GPU processes >>> import time >>> from accelerate.state import PartialState >>> state = PartialState() >>> if state.is_main_process: ... time.sleep(2) >>> else: ... print("I'm waiting for the main process to finish its sleep...") >>> state.wait_for_everyone() >>> # Should print on every process at the same time >>> print("Everyone is here") ``` ) device_idsz"accelerate.utils.wait_for_everyoneN)rTr MULTI_GPU MULTI_MLU MULTI_SDAA MULTI_MUSA MULTI_NPUrr MULTI_HPUrFSDPrrbarrierrVXLAr rendezvousrs r*wait_for_everyonePartialState.wait_for_everyoneqs,   % %  % %  & &  & &  % %  % %  % %  % %  % %   %     % %$2J2J1K % L  " "o&9&9 9 MM> ?:r)c#r# U(dUR5 Sv U(aUR5 gg7fr-)r)r7is_mains r* _goes_firstPartialState._goes_firsts+  " " $   " " $ s57c#^^^^^^# TRS:XaUv g[U5m[U[5(a[[U[ UR 55S5m[ U4SjUR555(d [S5e[TTR5ummTRT-[TRT5-nUT-TRT:aSOS-nUUUUU4SjmT"XU5v g7f)a Splits `input` between `self.num_processes` quickly and can be then used on that process. Useful when doing distributed inference, such as with different prompts. Note that when using a `dict`, all keys need to have the same number of elements. Args: inputs (`list`, `tuple`, `torch.Tensor`, `dict` of `list`/`tuple`/`torch.Tensor`, or `datasets.Dataset`): The input to split between processes. apply_padding (`bool`, `optional`, defaults to `False`): Whether to apply padding by repeating the last element of the input so that all processes have the same number of elements. Useful when trying to perform actions such as `gather()` on the outputs or passing in less inputs than there are processes. If so, just remember to drop the padded elements afterwards. Example: ```python # Assume there are two processes from accelerate import PartialState state = PartialState() with state.split_between_processes(["A", "B", "C"]) as inputs: print(inputs) # Process 0 ["A", "B"] # Process 1 ["C"] with state.split_between_processes(["A", "B", "C"], apply_padding=True) as inputs: print(inputs) # Process 0 ["A", "B"] # Process 1 ["C", "C"] ``` rNrc3@># UHn[U5T:Hv M g7fr-)len).0vlengths r* 7PartialState.split_between_processes..sAAs1v'sz6All values in the dictionary must have the same lengthc8>[U[[[R45(aU[ U5:aUSSnOXUnT (ag[U[R5(a&SSKJnJn U"UTR5nU"X`SS9nU$X3S/T T S:aSOS-[ U5- -- nU$[U[5(a&UR5HnT "XX5X'M U$[5(aSSK Jn [X5(aU[ U5:a[ U5S- nU[ U5:a [ U5n[[X55n T (a"XS- /T T S:aSOS-[ U 5- -- n UR!U 5$U$)Nrcr)pad_across_processessend_to_device) pad_indexr)Dataset) isinstancelisttuplerTensorraccelerate.utilsrrrSdictkeysrdatasetsrrangeselect)inputs start_index end_indexresultrrtensorized_resultkeyr result_idcs _split_values apply_padding num_extrasnum_samples_per_processr7s r*r;PartialState.split_between_processes.._split_valuess&4 "=>>#f+-#BC[F# :F !&%,,77Y-;64;;,O)!56GZ\S]!^ ":,2IR\_`R`Qfg2hknoukv2v"ww FD))!;;=C"/ ["TFK) (**0!&22&#f+5*-f+/K$s6{2(+F I&*5+H&I ('M? 7 Q1TU VY\]hYi i,K &}}[99 r)) rWrrrrrallvaluesrdivmodrXmin) r7rrrrrrrrs ` ` @@@@r*split_between_processes$PartialState.split_between_processessN    "L V fd # #V[[] 3A 678FAAAA !YZZ.4VT=O=O.P+((+BBSI[I[]gEhh "99$BTBTWaBaQghi # # JF;;sDDc#V# URUR5ShvN gN7f)a Lets the main process go first inside a with block. The other processes will enter the with block after the main process exits. Example: ```python >>> from accelerate import Accelerator >>> accelerator = Accelerator() >>> with accelerator.main_process_first(): ... # This will be printed first by process 0 then in a seemingly ... # random order by the other processes. ... print(f"This will be printed by process {accelerator.process_index}") ``` N)rrrs r*main_process_firstPartialState.main_process_firsts!&##D$8$8999 )')c#V# URUR5ShvN gN7f)a Lets the local main process go inside a with block. The other processes will enter the with block after the main process exits. Example: ```python >>> from accelerate.state import PartialState >>> state = PartialState() >>> with state.local_main_process_first(): ... # This will be printed first by local process 0 then in a seemingly ... # random order by the other processes. ... print(f"This will be printed by process {state.local_process_index}") ``` N)rrrs r*local_main_process_first%PartialState.local_main_process_firsts!&##D$>$>???rNcUR(d [S5eUR(dUR(dU$[$)a Decorator that only runs the decorated function on the main process. Args: function (`Callable`): The function to decorate. Example: ```python >>> from accelerate.state import PartialState >>> state = PartialState() >>> @state.on_main_process ... def print_something(): ... print("This will be printed by process 0 only.") >>> print_something() "This will be printed by process 0 only" ``` zUThe `PartialState` or `Accelerator` must be initialized before calling this function.)rrrrr0r7functions r*on_main_processPartialState.on_main_process%s50tu u   t';';Or)cVUR(dUR(dU$[$)a Decorator that only runs the decorated function on the local main process. Args: function (`Callable`): The function to decorate. Example: ```python # Assume we have 2 servers with 4 processes each. from accelerate.state import PartialState state = PartialState() @state.on_local_main_process def print_something(): print("This will be printed by process 0 only on each server.") print_something() # On server 1: "This will be printed by process 0 only" # On server 2: "This will be printed by process 0 only" ``` )rrr0rs r*on_local_main_process"PartialState.on_local_main_processCs 6  % %T-A-AOr)cVUR(dUR(dU$[$)a Decorator that only runs the decorated function on the last process. Args: function (`Callable`): The function to decorate. Example: ```python # Assume we have 4 processes. from accelerate.state import PartialState state = PartialState() @state.on_last_process def print_something(): print(f"Printed on process {state.process_index}") print_something() "Printed on process 3" ``` )rrr0rs r*on_last_processPartialState.on_last_processbs 0   t';';Or)cUc[URUS9$URU:XdUR(dU$[$)a" Decorator that only runs the decorated function on the process with the given index. Args: function (`Callable`, `optional`): The function to decorate. process_index (`int`, `optional`): The index of the process on which to run the function. Example: ```python # Assume we have 4 processes. from accelerate.state import PartialState state = PartialState() @state.on_process(process_index=2) def print_something(): print(f"Printed on process {state.process_index}") print_something() "Printed on process 2" ``` )rX)r on_processrXrr0)r7rrXs r*rPartialState.on_process~s<6  4??-H H   - /9M9MOr)cUc[URUS9$URU:XdUR(dU$[$)a Decorator that only runs the decorated function on the process with the given index on the current node. Args: function (`Callable`, *optional*): The function to decorate. local_process_index (`int`, *optional*): The index of the local process on which to run the function. Example: ```python # Assume we have 2 servers with 4 processes each. from accelerate import Accelerator accelerator = Accelerator() @accelerator.on_local_process(local_process_index=2) def print_something(): print(f"Printed on process {accelerator.local_process_index}") print_something() # On server 1: "Printed on process 2" # On server 2: "Printed on process 2" ``` )rV)ron_local_processrVrr0)r7rrVs r*r!PartialState.on_local_processs@<  400FYZ Z  $ $(; ;TEYEYOr)c>UR(a [U0UD6 ggr-)rprintr7r.r/s r*r$PartialState.prints  % % 4 "6 " &r)c[5(a)S[RS'[R"S5$[ 5(a[R"S5$[ 5(a[R"S5$[5(a[R"S5$[5(a[R"S5$[5(a[R"S5$[RR5(a[R"S 5$[5(a[R"S 5$[R"S 5$) aY Returns the default device which is: - MPS if `torch.backends.mps.is_available()` and `torch.backends.mps.is_built()` both return True. - CUDA if `torch.cuda.is_available()` - MLU if `is_mlu_available()` - SDAA if `is_sdaa_available()` - MUSA if `is_musa_available()` - NPU if `is_npu_available()` - HPU if `is_hpu_available()` - CPU otherwise 1PYTORCH_ENABLE_MPS_FALLBACKmpsmlurmusanpuhpur}xpur) rrrrrSrrrrrr} is_availablerrs r*default_devicePartialState.default_devices   8;BJJ4 5<<& &   <<& &  <<' '  <<' '  <<& &   <<& & ZZ $ $ & &<<' '   <<& &<<& &r)c SnU(aSSKnSn[RnGOp[5(aSn[RnGOM[ [ RRSS55S:wGaU(Gd[5(aSn[Rn[5(aSn[RnO[5(aS n[RnO[5(aS n[R nO[#S S 9(aUcS n[R$nOl[&R(R+5(aUcS n[RnO3[-5(a$[/5(aUcSn[R0nUc[ [ RRSS55S:wd[3/SQS5S:aU(d [-5(a[R0nO[R4nUS;a;[75(a,[3S/S5S:dU[R0:XaSSKnSnO.US;a&[&R:R=5(aSnOSnUc[R>nX44$)zhPrepares any imports needed before initializing the distributed backend and sets `self.backend` properlyNrsmddprarbrccnclrfmcclhcclT) init_hcclrixccl)PMI_SIZEOMPI_COMM_WORLD_SIZEMV2_COMM_WORLD_SIZErpr)NcclCCL_WORKER_COUNTr=)Nrurugloo) ,smdistributed.dataparallel.torch.torch_smddpr rrrrrrrrrrrrrrrrrrr}r0rrrrrroneccl_bindings_for_pytorchris_mpi_availabler)r7r sagemaker_dprQrT smdistributedrAs r*rPartialState._prepare_backends   ?G.88  # % %G.22   b1 2b 8!! #2#<#<  "" #2#=#= "$$ #2#=#= "## #2#<#< !D1?$G#2#<#< ((**?$G#2#<#< !##(9(;(;?$G#2#<#<  #  |R0 1R 7 iklmpqq+--#2#<#< #2#<#< =($&&%'9&:A>BFVZiZsZsFs2M)e.?.?.P.P.R.R   #.11 ((r)c2URbgUR[R:Xa9UR(a[ R"S5O UR Ulg[UR5RS5SRSS5R5nUS;a[SURS US 35eUS :Xa[R"5UlgUS :Xa9[ R"S [ RR55UlgUS :XaSn[![ U5nUR"UR%5-n[ R"X5UlUR'UR5 g)zJ Sets the device in `self.device` to the current distributed environment. Nr.rcMULTI_r) rgpur+r,r-r/rar.rzCan't set device for z (z6), verify we should be calling `_set_device()` for it!rar.rIr})rSrTr rrOrr1rsplitreplacerrr xla_devicer.current_devicegetattrrV device_countr)r7rS device_module device_indexs r*rPartialState.set_device.s; ;; "   O$6$6 615%,,u-@S@SDK T**+11#6r:BB8RPVVX Z Z'(=(='>bH~  U?--/DK u_,,ueii.F.F.HIDK#E62M33m6P6P6RRL,,v8B"H# ' 'FLPB)B)>HB) $B)H24; Qr)rMc\rSrSrSr\"5r\R/SQ-rSSSjjr \ SSj5r Sr SSjr \ S 5r\S S!S jj5rS"S jr\ S 5r\ S 5r\ SSj5r\ SSj5r\ SSj5r\ SSj5rSr\S S#Sjj5r\S5r\S5r\ S5r\S$Sj5r\S"S%Sjj5r Sr!S$Sjr"Sr#g)&r&i_a Singleton class that has information about the current training environment. **Available attributes:** - **device** (`torch.device`) -- The device to use. - **distributed_type** ([`~accelerate.state.DistributedType`]) -- The type of distributed environment currently in use. - **parallelism_config** ([`~accelerate.utils.ParallelismConfig`]) -- The parallelism configuration for the current training environment. This is used to configure the distributed training environment. - **initialized** (`bool`) -- Whether or not the `AcceleratorState` has been initialized from `Accelerator`. - **local_process_index** (`int`) -- The index of the current process on the current server. - **mixed_precision** (`str`) -- Whether or not the current script will use mixed precision, and if so the type of mixed precision being performed. (Choose from 'no','fp16','bf16 or 'fp8'). - **num_processes** (`int`) -- The number of processes currently launched in parallel. - **process_index** (`int`) -- The index of the current process. - **is_last_process** (`bool`) -- Whether or not the current process is the last one. - **is_main_process** (`bool`) -- Whether or not the current process is the main one. - **is_local_main_process** (`bool`) -- Whether or not the current process is the main one on the local node. - **debug** (`bool`) -- Whether or not the current script is being run in debug mode. )deepspeed_pluginuse_ipex fsdp_pluginmegatron_lm_plugin dynamo_pluginNc  URUl[S5(aSn[R0:Xa [U40U D6 URR [R5 UR X5 UR (GdSUlSUlX`l Xl SUl Uc [SS5OUR5nUS:Xa[5(d [S5e[ R"R%5(aG['5(d8[(R+S[ R"R-5S35 S nO&[/5(a[(R+S 5 S nX0lU (d [S 5eUR2[4R6:XaUS:waSUlOXlUR2[4R::Xa[=SS 9(aUS :Xa[>R@RCS5(aA[ES5[>R@S'[ES5[>R@S'SUl#GO[ES5[>R@S'[ES5[>R@S'SUl#GO[>R@RCSS5R5S:XaU(d[4R6Ul[IU[J5(d!URMU5 UROSS9 OYURQ5Hn U RMU5 M [S[UURQ555n U ROSS9 X@lGOUR2[4RV[4RX[4RZ[4R\[4R^[4R`[4Rb4;Ga[>R@RCSS5R5S:Xd}UR(a)URRd(aUc [S5eURb6URRd(aURfS:Xa [S5e[>R@RCSS5R5S:Xd+Uc(URbdURRd(aI[4RhUlUR8S:waUbURMUR85 XPl5[>R@RCSS5R5S:XaUUR2[4R`4;a6[4RlUlURMUR85 Xpl7OcUR2[4Rp[4R`[4Rr4;a&[u5(a[SSS9UlOSUlUR0Rv[xRr:waSUR8S:XaCURzR|S:Xa)S[ R~R"RlAUR0Rv[xRr:waSUR8S:XaCURzR|S :Xa)S[ R~RRlAUR2[RS!'gg)"NACCELERATE_USE_CPUTACCELERATE_MIXED_PRECISIONnofp8zPUsing `fp8` precision requires `transformer_engine` or `MS-AMP` to be installed.z-The current device has compute capability of z which is insufficient for FP8 mixed precision training (requires a GPU Hopper/Ada Lovelace or higher, compute capability of 8.9 or higher). Will use FP16 instead.fp16zThe current HPU device is Gaudi1 which does not support FP8 mixed precision training (requires Gaudi2 or higher). Will use BF16 instead.bf16zPlease make sure to properly initialize your accelerator via `accelerator = Accelerator()` before using any functionality from the `accelerate` library.rzACCELERATE_DOWNCAST_BF16r XLA_USE_BF16rXLA_DOWNCAST_BF16Frdr^r__from_accelerator_stateACCELERATE_ALLOW_CP_STANDALONEz`cp_size > 1` specified in the `parallelism_config`, but no `fsdp_plugin` was provided. We need a `fsdp_plugin` to use context parallelism, as we also shard the model across the device mesh to save more memoryzQUsing `cp_size>1` requires FSDP2, but the provided `fsdp_plugin` is using FSDP1. rjACCELERATE_USE_MEGATRON_LMACCELERATE_USE_IPEXdefaultr}r,rT)Cr'rr!rMupdate_check_initializedrdeepspeed_pluginsrntorch_tp_pluginparallelism_config device_meshr rrrrr}r0r loggerwarningget_device_capabilityrrqrTr rrPrrrrrrdowncast_bfloatrrset_mixed_precisionrrnextiterrrrrrrr cp_enabled fsdp_versionrrorrprrrrQr rSrbackendsmatmul allow_tf32r,) r7mixed_precisionrrqrmrorrpr_from_acceleratorr/plugin first_plugins r*r9AcceleratorState.__init__s** 3 4 4C  % % +  ' ' \778 5%)D " DM#2 &8 ##D #*&&BDI$**,  %''))$jZZ,,..7P7R7RNNG HhHhHjGklbb '-O%''NND'-O!. $ T $$(A(AAoY^F^(,%(7%$$(;(;;@Vdh@i"f,zz~~&@AA58V >2:=a& #67/3,58V >2:=a& #67/4, :GDJJLPVV_b(7(A(A%!"2D99$88I$++D+I"2"9"9";22?C#<$(-=-D-D-F(G#HL '''E)9&&&))))****))))))+zz~~&FPVVX\bb..43J3J3U3UZeZm(p//; 33>>'449(oJJNN#8'BHHJfTXcXo++7D>0'%'V$(,(=(=#--F)-<,G,GD)&::4;P;PQ.@+&&?+D+DoF_F_apasas*tt$&&$78MW[$\DM$)DM""**m.>.>>))T1KK$$.8<##**5""**m.>.>>))T1KK$$.8<##**5=A=R=RL & &'9 :u r)c<UR[R:g$r-)r'rMrs r*rAcceleratorState.initializeds!!\%?%???r)c[5R5SURS3-nUR[R :XaUSUR RS3- nU$)Nz Mixed precision type: rz ds_config: )rMrrrTr rrmdeepspeed_config)r7reprs r*rAcceleratorState.__repr__sb~&&(-EdFZFZE[[]+^^  O$=$= = k$"7"7"H"H!IL LD r)c:UR(aSnU(a2URRS:wa[UR SS95eUbKXR :wa;UR [R:wa[UR SUS3S95egggg)zeChecks if a modification is trying to be made and the `AcceleratorState` has already been initializedzAcceleratorState has already been initialized and cannot be changed, restart your runtime completely and pass `{flag}` to `Accelerator()`.rzcpu=True)flagNzmixed_precision='rY) rrSrrformatrPrTr r)r7rrerrs r*r#AcceleratorState._check_initializeds   _Ct{{''50 !<==+#'<'<<))_-F-FF 3D_DUUV1W!XYYG=, r)c\UR[R:XaURS:waqURR nUR S05R SS5(aSnU$UR S05R SS5(aSnU$SnU$URnU$)NrvrwenabledFrxru)rTr rrPrmrr)r7configrs r*r AcceleratorState.mixed_precision&s  O$=$= =$BWBW[`B`**;;Fzz&"%)))U;;"( FB'++Iu=="( #'#33Or)cx[RR5 U(a[R 5 ggr)r&r'rrMr)reset_partial_states r*rAcceleratorState._reset_state4s* &&,,.   % % ' r)c6[5RU5 g)z Destroys the process group. If one is not specified, the default process group is destroyed. If `self.fork_launched` is `True` and `group` is `None`, nothing happens. N)rMrTrUs r*rT&AcceleratorState.destroy_process_group;s ,,U3r)c*[5R$r-)rMrUrs r*rUAcceleratorState.fork_launchedCs~+++r)c*[5R$)r)rMrrs r*r AcceleratorState.use_distributedGs ~---r)c|UR[R:H=(a URRS:H$)N)rTr rrorrs r*is_fsdp2AcceleratorState.is_fsdp2Ns0$$(<(<<cAQAQA^A^bcAccr)c*[5R$)r)rMrrs r*r AcceleratorState.is_last_processR~---r)c*[5R$)r)rMrrs r*r AcceleratorState.is_main_processWrr)c*[5R$)r)rMrrs r*r&AcceleratorState.is_local_main_process\s~333r)c4[5R5 gr-)rMrrs r*r"AcceleratorState.wait_for_everyoneas((*r)c#v# [5RXS9nUv SSS5 g!,(df  g=f7f)a Splits `input` between `self.num_processes` quickly and can be then used on that process. Useful when doing distributed inference, such as with different prompts. Note that when using a `dict`, all keys need to have the same number of elements. Args: inputs (`list`, `tuple`, `torch.Tensor`, or `dict` of `list`/`tuple`/`torch.Tensor`): The input to split between processes. apply_padding (`bool`, `optional`, defaults to `False`): Whether to apply padding by repeating the last element of the input so that all processes have the same number of elements. Useful when trying to perform actions such as `gather()` on the outputs or passing in less inputs than there are processes. If so, just remember to drop the padded elements afterwards. Example: ```python # Assume there are two processes from accelerate.state import AcceleratorState state = AcceleratorState() with state.split_between_processes(["A", "B", "C"]) as inputs: print(inputs) # Process 0 ["A", "B"] # Process 1 ["C"] with state.split_between_processes(["A", "B", "C"], apply_padding=True) as inputs: print(inputs) # Process 0 ["A", "B"] # Process 1 ["C", "C"] ``` )rN)rMr )r7rrs r*r (AcceleratorState.split_between_processesds/N^ 3 3F 3 X\bLY X Xs9( 9 69c#x# [5R5 Sv SSS5 g!,(df  g=f7f)z Lets the main process go first inside a with block. The other processes will enter the with block after the main process exits. N)rMr rs r*r #AcceleratorState.main_process_firsts#^ . . 0 1 0 0:) : 7:c#x# [5R5 Sv SSS5 g!,(df  g=f7f)z Lets the local main process go inside a with block. The other processes will enter the with block after the main process exits. N)rMrrs r*r)AcceleratorState.local_main_process_firsts#^ 4 4 6 7 6 6rc\UR[R:wagSSKJn U"U5$)zX Returns the currently active DeepSpeedPlugin. If not using deepspeed, returns `None`. Nr)get_active_deepspeed_plugin)rTr raccelerate.utils.deepspeedr)r7rs r*rm!AcceleratorState.deepspeed_plugins)  O$=$= =J*400r)c URU$)z8 Returns the DeepSpeedPlugin with the given plugin_key. )rr]s r*get_deepspeed_plugin%AcceleratorState.get_deepspeed_plugins %%d++r)cURR5Hup#X!:wdM UR5 M URURSS9 g)zZ Activates the DeepSpeedPlugin with the given `name`, and will disable all other plugins. Tr|N)ritems _unselectr)r7r^rrs r*select_deepspeed_plugin(AcceleratorState.select_deepspeed_pluginsN  11779KC{  ": t$++D+Ir)c8[5R"U0UD6 gr-)rMr$r%s r*r$AcceleratorState.printsd-f-r)c\XR;a[SUS35e[SUS35e)Nz,`AcceleratorState` object has no attribute `z}`. This happens if `AcceleratorState._reset_state()` was called and an `Accelerator` or `PartialState` was not reinitialized.z,'AcceleratorState' object has no attribute 'rYrZr]s r*r_AcceleratorState.__getattr__sI $$ $ >tfELL  KD6QRSTTr)) rrPrrrTrrqrorprrrn) NFNNNNNNF)rrgrrErrErbrerD)rrEr-rcrh)r^rg)$rFrGrHrIrJrir'rMr[r9rkrrrrrjrrTrUrrrrrrrr r rrmrrrr$r_rKr(r)r*r&r&_s,LM,,0L'+"'NS#NSNS NS`@@ Z  (( 4,,.. dd....44+''R 1 1,, JJ. Ur)r&c`\rSrSrSr\"5rSSSjjr\SSj5r \SSj5r \SSj5r \SSj5r \SS j5r \SS j5rS r\S 5r\R"S 5rSrSrSr\S5r\S5r\R"S5r\SSj5r\S5rSrg) GradientStateia Singleton class that has information related to gradient synchronization for gradient accumulation **Available attributes:** - **end_of_dataloader** (`bool`) -- Whether we have reached the end the current dataloader - **remainder** (`int`) -- The number of extra samples that were added from padding the dataloader - **sync_gradients** (`bool`) -- Whether the gradients should be synced across all devices - **active_dataloader** (`Optional[DataLoader]`) -- The dataloader that is currently being iterated over - **dataloader_references** (`List[Optional[DataLoader]]`) -- A list of references to the dataloaders that are being iterated over - **num_steps** (`int`) -- The number of steps to accumulate over - **adjust_scheduler** (`bool`) -- Whether the scheduler should be adjusted to account for the gradient accumulation - **sync_with_dataloader** (`bool`) -- Whether the gradients should be synced at the end of the dataloader iteration and the number of total steps reset - **is_xla_gradients_synced** (`bool`) -- Whether the XLA gradients have been synchronized. It is initialized as false. Once gradients have been reduced before the optimizer step, this flag is set to true. Subsequently, after each step, the flag is reset to false. FSDP will always synchronize the gradients, hence is_xla_gradients_synced is always true. NcURUlUR(d0SUlS/UlUbUR 5O0UlSUlUb5UR UR 5:waUR 5Ulggg)NTF)r'rrsync_gradients_dataloader_references_ref to_kwargs plugin_kwargs_is_xla_gradients_synced)r7gradient_accumulation_plugins r*r9GradientState.__init__s** "&D /3fD +r*s# % 88))"'%(%('   8 $0IOO8011T7L ` Q` QFoUoUd L,L,r)