9iXdZddlmZmZddlmZmZerddlmZ dde fdZ y) z;AWQ (Activation aware Weight Quantization) integration file)is_auto_awq_availableis_torch_available)AwqBackendPackingMethodAWQLinearVersionNreturnc |g}|j}ts td|tjk(r ddlm}m}n|tjk(rddl m }|tjk(r"|jtjk(rn} n} |jD]\} } gj| t!| t"j$r| |vrt'fd|Ds| j(} | j*} | |j,|j.| | | j0du| j2j4|j6| <d}|j6| j9d t;t=| j?dkDrtA| ||| \}}jCd  ||fS) a Public method that recursively replaces the Linear layers of the given model with AWQ quantized layers. `accelerate` is needed to use this method. Returns the converted model and a boolean that indicates if the conversion has been successfull or not. During the module replacement, we also infer the backend to use through the `quantization_config` object. Args: model (`torch.nn.Module`): The model to convert, can be any `torch.nn.Module` instance. quantization_config (`AwqConfig`): The quantization config object that contains the quantization parameters. modules_to_not_convert (`list`, *optional*): A list of modules to not convert. If a module name is in the list (e.g. `lm_head`), it will not be converted. current_key_name (`list`, *optional*): A list that contains the current key name. This is used for recursion and should not be passed by the user. has_been_replaced (`bool`, *optional*): A boolean that indicates if the conversion has been successful or not. This is used for recursion and should not be passed by the user. NzAWQ (either `autoawq` or `llmawq`) is not available. Please install it with `pip install autoawq` or check out the installation guide in https://github.com/mit-han-lab/llm-awqr) WQLinear_GEMM WQLinear_GEMV)WQLinearc3DK|]}|djvyw).N)join).0keycurrent_key_names g/var/www/html/backtest/airagagent/rag_env/lib/python3.12/site-packages/transformers/integrations/awq.py z*replace_with_awq_linear..Ns [Sschh'788[s )w_bit group_size in_features out_featuresbiasdevTF)modules_to_not_convertrquantization_confighas_been_replaced)"backendr ValueErrorrAUTOAWQawq.modules.linearr r LLMAWQawq.quantize.qmoduler versionrGEMMnamed_childrenappend isinstancennLinearanyrrbitsrrweightdevice_modulesrequires_grad_lenlistchildrenreplace_with_awq_linearpop)modelrrrrrr r r target_clsnamemodulerr_s ` rr5r5s8%!#!))G " ~  )111CC +22 21)111&9&A&AEUEZEZ&Z]`m  ,,. ! f  #! % fbii (T9O-O[DZ[[$00 %22 '1-222== +!-D0 ,, (t$%)!t$33E: tFOO%& '! +#:'=!1$7"3 $ A  R A !B # ##)NNNF) __doc__utilsrrutils.quantization_configrrtorch.nnr*boolr5r<rrCs:>=Q   Q$  Q$r<