;iSSKrSSKrSSKrSSKrSr"SS5rSSjrS\S\\\44Sjr SS jr S r S r SS S SS.r \ R5VVs0sHupX_M snnrS\R 4Sjrgs snnf)Nc ~[U[RR5(de[R 5nUR UR5nX2R;Ga[UR5nURRU5 URRU5 [UR5S:axURSR5S:a:URSR5URR S:deURSURU'gUSR#SUSR S5n[USSS9n[R$"US5S:R'[)[+[USR 5S- 55S9n[R,"US:5Sn[R."XG/5R15nXBRU'gUR2HnUR55 M g)Nr) reduction_dimzscore)dim) isinstancetorchnnLinear OutlierTracer get_instance get_hvalueweighthvalue2outlier_idxfind_outlier_dimsoutliersappendhvalueslennumelmaxshapeviewabssumlistrangewherecatuniquehooksremove) moduleinputtracerhvalue outlier_idxmergeddims outlier_idx2hooks b/home/dmtnaga/Documents/work/airagagent/rag_env/lib/python3.13/site-packages/bitsandbytes/utils.py outlier_hookr0s fehhoo . .. .  ' ' )F   v}} -F ...' 6 {+f% v ! #r"((*Q.r*..06==3F3Fq3IIII060CF % %f -1X]]2uQx~~b'9:F+F!ANKIIeAh'!+00T%E!HNN@SVW@W:X5Y0ZD ;;tax03L))[$?@GGIK0; % %f -LLD KKM!cF\rSrSrSrSrSrSrSrSr \ S5r S r g) r)Nc[S5e)NzCall get_instance() instead) RuntimeErrorselfs r/__init__OutlierTracer.__init__,s899r1cJSUlSUl/Ul/Ul0UlSUl/UlUR5H^up#[U[RR5(dM0UR RUR[55 M` g)NT)last_wcurrent_outlier_dimsrrr initializedr$ named_modulesr r r rrregister_forward_pre_hookr0)r7modelnms r/ initializeOutlierTracer.initialize/s} $(!  "$ '')DA!UXX__-- !!!"="=l"KL*r1c[USS5$)Nr=F)getattrr6s r/is_initializedOutlierTracer.is_initialized<st]E22r1cRURR5R5$N)datastoragedata_ptr)r7rs r/rOutlierTracer.get_hvalue?s{{""$--//r1cUR5(d [S5 gURU5nX R;aURU$g)Nz$Outlier tracer is not initialized...)rGprintrr)r7rr)s r/ get_outliersOutlierTracer.get_outliersBsK""$$ 8 9( ,, ,**62 2r1c`URcURU5UlUR$rJ) _instance__new__)clss r/rOutlierTracer.get_instanceLs& == KK,CM}}r1)r<r$rrr=r;r) __name__ __module__ __qualname____firstlineno__rTr8rCrGrrQ classmethodr__static_attributes__r1r/rr)s5I: M30r1rcU(a=[R"SURSU4URS9R 5$UR U5nUR 5nUR 5nXV- U- nUb([R"UR5USS9upU $[R"X:5Sn U $)Nrr)sizedevice)kr ) r randintrralongstdmeantopkrr!) rrrrgrdmrestdmstdstdzstdvalidxs r/rrSs }}Q QtgfmmTYY[[ **] #C 88:D WWYF J& D ::cggi4Q7 Jkk$-(+ Jr1command_stringreturnc0^SmU4SjnU"U5up#X#4$)Nc&[SU55$)Nc3^# UH#oRS5R5v M% g7f)zUTF-8N)decodestrip).0 to_decodes r/ 6execute_and_return.._decode..gs'aH`9%%g.4466H`s+-)tuple)subprocess_err_out_tuples r/_decode#execute_and_return.._decodefsaH`aaar1c>T"[R"[R"U5[R[RS9R 55$)N)stdoutstderr) subprocessPopenshlexsplitPIPE communicate)rnr{s r/&execute_and_return_decoded_std_streamsBexecute_and_return..execute_and_return_decoded_std_streamsisA    N+!! km   r1r^)rnrstd_outstd_errr{s @r/execute_and_returnres&b >nMG  r1cVUR5GHupV[[UR555S:a [ XaX#U5 [ U[ RR5(dMdXR;dMkURUnU"URURURSL5URU'U(a<URURUl URURUl UcM[XdS5nUcGM U"U5 GM U$)a Replace linear modules with a new Linear module. Parameters: model (`torch.nn.Module`): Input model or `torch.nn.Module` as the function is run recursively. linear_replacement (`torch.nn.Module`): The linear module that replaces the old one. Only expects standard arguments. If other arguments need to be passed, use a lambda. skip_modules (`List[str]`, *optional*, defaults to `lm_head`): List of modules names not to convert. Defaults to `lm_head`. copy_weights (`bool`): Copy the weights from the old linear module to the new one post_processing_function (`str`): A function name of the replacement linear class that is called after processing. rN)named_childrenrrchildrenreplace_linearr r r r_modules in_features out_featuresbiasrrF) r@linear_replacement skip_modules copy_weightspost_processing_functionnamer& old_modulefuncs r/rrvs.,,.  tFOO%& '! + 6|Sk l fehhoo . .43K-J#5""## 4'$ENN4 .8.?.?t$+,6OOt$)'3vF#L%/& Lr1c[R"U5nURS5n[R"[ U5[R S9nU$)z Pack a dictionary into a torch tensor for storing quant_state items in state_dict. Parameters: - source_dict: The dictionary to be packed. Returns: A torch tensor containing the packed data. utf-8)dtype)jsondumpsencoder tensorruint8) source_dictjson_str json_bytes tensor_datas r/pack_dict_to_tensorrs?zz+&H)J,,tJ/u{{CK r1c[UR5R55nURS5n[R "U5nU$)z Unpack a torch tensor into a Python dictionary. Parameters: - tensor_data: The torch tensor containing the packed data. Returns: A Python dictionary containing the unpacked data. r)bytescpunumpyrsrloads)rrr unpacked_dicts r/unpack_tensor_to_dictrsB{(..01J  )HJJx(M r1rr)rowcol32 col_turing col_amperetcURRS:Xa[RR 5 gURRS:Xa[R R 5 gg)Ncudaxpu)ratyper r synchronizer)rs r/sync_gpursGxx}}  %   r1)rg@NF))lm_headFN)rrrr r0rrstrryrrrr"LINEAR_8BIT_WEIGHTS_FORMAT_MAPPINGitems*INVERSE_LINEAR_8BIT_WEIGHTS_FORMAT_MAPPINGTensorr)rrls00r/rs  B''T$suS#X(! *Z""./![\%]"KmKsKsKu-vKuKTciKu-v*   .wsA=