;iRSSKJr "SS\5r"SS\5r"SS\5rg) )Optimizer2StatecF^\rSrSrSU4SjjrSrU=r$)LAMBc6>[TU]SUUUUUU U U U U SS9 g)a Base LAMB optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-3): The learning rate. bias_correction (`bool`, defaults to `True`): Whether to apply bias correction to the first and second-order moments. betas (`tuple(float, float)`, defaults to (0.9, 0.999)): The beta values are the decay rates of the first and second-order moment of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 1e-2): The weight decay value for the optimizer. amsgrad (`bool`, defaults to `False`): Whether to use the [AMSGrad](https://hf.co/papers/1904.09237) variant of Adam that uses the maximum of past squared gradients instead. adam_w_mode (`bool`, defaults to `True`): Whether to use the AdamW variant. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. max_unorm (`float`, defaults to 1.0): The maximum gradient norm. lamb? max_unormNsuper__init__)selfparamslrbias_correctionbetaseps weight_decayamsgrad adam_w_mode optim_bitsargs min_8bit_sizepercentile_clipping block_wiser __class__s g/home/dmtnaga/Documents/work/airagagent/rag_env/lib/python3.13/site-packages/bitsandbytes/optim/lamb.pyr LAMB.__init__ s<d              ) MbP?Tg?g+?:0yE>rFT NdFr __name__ __module__ __qualname____firstlineno__r__static_attributes__ __classcell__rs@rrrs6   ? ? r rcD^\rSrSrSU4SjjrSrU=r$)LAMB8bitKc6>[TU]SUUUUUSU U U U SS9 g)a@ 8-bit LAMB optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-3): The learning rate. bias_correction (`bool`, defaults to `True`): Whether to apply bias correction to the first and second-order moments. betas (`tuple(float, float)`, defaults to (0.9, 0.999)): The beta values are the decay rates of the first and second-order moment of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 1e-2): The weight decay value for the optimizer. amsgrad (`bool`, defaults to `False`): Whether to use the [AMSGrad](https://hf.co/papers/1904.09237) variant of Adam that uses the maximum of past squared gradients instead. adam_w_mode (`bool`, defaults to `True`): Whether to use the AdamW variant. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. max_unorm (`float`, defaults to 1.0): The maximum gradient norm. rrr r Nr rrrrrrrrrrrrrr rs rrLAMB8bit.__init__Ls<^             r r! r"Tr#r$rFTNr&r'Fr r(r/s@rr1r1K3   < < r r1cD^\rSrSrSU4SjjrSrU=r$) LAMB32bitc6>[TU]SUUUUUSU U U U SS9 g)aA 32-bit LAMB optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-3): The learning rate. bias_correction (`bool`, defaults to `True`): Whether to apply bias correction to the first and second-order moments. betas (`tuple(float, float)`, defaults to (0.9, 0.999)): The beta values are the decay rates of the first and second-order moment of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 1e-2): The weight decay value for the optimizer. amsgrad (`bool`, defaults to `False`): Whether to use the [AMSGrad](https://hf.co/papers/1904.09237) variant of Adam that uses the maximum of past squared gradients instead. adam_w_mode (`bool`, defaults to `True`): Whether to use the AdamW variant. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. max_unorm (`float`, defaults to 1.0): The maximum gradient norm. rr%r r Nr r4s rrLAMB32bit.__init__s<^              r r!r6r(r/s@rr9r9r7r r9N)bitsandbytes.optim.optimizerrrr1r9r!r rr>s2 9@ ?@ F= = @= = r