;iSSKrSSKrSSKJr SSKJrJr SSKrSSK r SSK J r SSjr Sr SrS\\\R 4S \\\ R$44S jr"S S \5r"S S\5rS \\\ R$4S\\4Sjrg)N)Mapping)OptionalUnion) safe_opencSn[UR5S:Xa!UR[R5nSnUR 5R 5n[RRX!S35nUb2Uc[UR5nU[UR5S.X1'URS:XaUSn[R"XeRSURS9nUSSUSS&UR5 U$)Nztorch.bfloat16bfloat16.dat)dtypeshaperzw+)r moder )strr viewtorchint16cpunumpyospathjoinlistr ndimnpmemmapflush)weight weight_nameoffload_folderindexr array tensor_file file_arrays h/home/dmtnaga/Documents/work/airagagent/rag_env/lib/python3.13/site-packages/accelerate/utils/offload.pyoffload_weightr#s E 6<<,,U[[) JJL   E'',,~d/CDK  = $E',tEKK7HI zzQd ;kkEKKXJ!HJqM Lc"[US5nUS:XaSnUSnUS:XaSn[R"XUSS9n[US5S :XaUS n[R "U5nUSS:XaUR [R5nU$) Nr )r rrr)r r r r)tuplerrlenrtensorrr) weight_file weight_infor r rs r"load_offloaded_weightr..s +g& 'E {  E  YY{u3 GF ;w  A% \\& !F7z)U^^, Mr$cUb[U5S:Xag[RRUS5n[RR U5(a*[ USS9n[ R"U5nSSS5 O0nWRU5 [ USSS9n[ R"XCSS9 SSS5 g!,(df  NI=f!,(df  g=f)Nr index.jsonzutf-8)encodingw)indent) r*rrrisfileopenjsonloadupdatedump)rroffload_index_filef current_indexs r"save_offload_indexr>Ds }E anlC ww~~()) $w 71 IIaLM8 7   # 8A -1- 9 8 8 7 9 8s"B? C? C  Csave_dir state_dictc[R"USS9 0nUR5Hup4[XCXS9nM [ X 5 g)z Offload a state dict in a given folder. Args: save_dir (`str` or `os.PathLike`): The directory in which to offload the state dict. state_dict (`Dict[str, torch.Tensor]`): The dictionary of tensors to offload. T)exist_ok)rN)rmakedirsitemsr#r>)r?r@rname parameters r"offload_state_dictrGUsBKK4( E%++-yF.u'r$c<\rSrSrSrS\S\4SjrSrSr Sr S r g ) PrefixedDatasethz Will access keys in a given dataset by adding a prefix. Args: dataset (`Mapping`): Any map with string keys. prefix (`str`): A prefix to add when trying to access any element in the underlying dataset. datasetprefixcXlX lgNrKrL)selfrKrLs r"__init__PrefixedDataset.__init__qs   r$c<URURU3$rNrOrPkeys r" __getitem__PrefixedDataset.__getitem__us||t{{mC5122r$c[URVs/sH&oRUR5(dM$UPM( sn5$s snfrN)iterrK startswithrLrTs r"__iter__PrefixedDataset.__iter__xs0DLLPLSNN4;;4OSLPQQPs #AAc,[UR5$rN)r*rKrPs r"__len__PrefixedDataset.__len__{s4<<  r$rON) __name__ __module__ __qualname____firstlineno____doc__rr rQrVr[r___static_attributes__r&r$r"rIrIhs*3R!r$rIc\rSrSrSrS S\\\\R4S\\ \\ R4S\\ 4SjjrS\4S jrS rS rS rg)OffloadedWeightsLoadera A collection that loads weights stored in a given state dict or memory-mapped on disk. Args: state_dict (`Dict[str, torch.Tensor]`, *optional*): A dictionary parameter name to tensor. save_folder (`str` or `os.PathLike`, *optional*): The directory in which the weights are stored (by `offload_state_dict` for instance). index (`Dict`, *optional*): A dictionary from weight name to their information (`dtype`/ `shape` or safetensors filename). Will default to the index saved in `save_folder`. Nr@ save_folderrcUcUcUc [S5eUc0OUUlX lUcKUbH[[R R US55n[R"U5nSSS5 Uc0OUUl [URR55Ul URRURVs/sHofUR;dMUPM sn5 X@lg!,(df  N=fs snf)NzWNeed either a `state_dict`, a `save_folder` or an `index` containing offloaded weights.r0) ValueErrorr@rjr6rrrr7r8rrkeysall_keysextenddevice)rPr@rjrrpr<rUs r"rQOffloadedWeightsLoader.__init__s  +"5%-vw w * 2" & =[4bggll; =>! ! ? =Re T__1134  TZZTZcdmm;ScZTU ?>UsC7 D#D7 DrUc2XR;aURU$URUnURS5bURcSO URnSn[ USSUS9nUR URSU55nSSS5 SU;a"UR[[US55nUR[R"U5:waURU5nU$[RRURUS35n[Xb5$!,(df  N=f![ aN [ USSSS9nUR URSU55nSSS5 M!,(df  N=ff=f)Nsafetensors_filerpt) frameworkrprr r )r@rgetrpr get_tensor TypeErrortogetattrrrrrrjr.)rPrUr-rpr+r<r,s r"rV"OffloadedWeightsLoader.__getitem__s\ // !??3' 'jjo ??- . :"kk1Ut{{FF O{+=>$W]^bc\\+//-*MNF_+%75+g2F#GH}} V 446*Mggll4#3#3uD\B $[>>_^ O{+=>$W\]ab\\+//-*MNF^]]] OsHD>,"D-D>- D;7D>;D>>F"F8F F FFc,[UR5$rN)rYrnr^s r"r[OffloadedWeightsLoader.__iter__sDMM""r$c,[UR5$rN)r*rnr^s r"r_OffloadedWeightsLoader.__len__s4==!!r$)rnrprrjr@)NNNN)rarbrcrdrerdictr rTensorrrPathLikerrQrVr[r_rfr&r$r"rhrhsw 9=9=#' T#u||"345eC$456 (?s?4#"r$rhsubmodule_namesc 0nUHWnURUR5VVs0sH'upEXC:XdURUS-5(dM%XE_M) snn5 MY U$s snnf)z Extract the sub state-dict corresponding to a list of given submodules. Args: state_dict (`Dict[str, torch.Tensor]`): The state dict to extract from. submodule_names (`List[str]`): The list of submodule names we want to extract. .)r9rDrZ)r@rresult module_namerUparams r"extract_submodules_state_dictrsoF&   #-"2"2"4 "4JC% c8I)J "4  ' M  s $A" A" rN)r7rcollections.abcrtypingrrrrr safetensorsrr#r.r>r rrrrGrIrhrrr&r$r"rs #" !*,."(sBKK'7!8(d3PUP\P\K\F](&!g!.@"W@"Fd3 3D.EX\]`Xar$