;i2>SSKJr "SS\5r"SS\5r"SS\5r"SS \5r"S S \5r"S S \5rg))Optimizer2StatecB^\rSrSrSU4SjjrSrU=r$)Adam c 6>[T U]SUUUUUUUU U U U S9 g)a Base Adam optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-3): The learning rate. betas (`tuple(float, float)`, defaults to (0.9, 0.999)): The beta values are the decay rates of the first and second-order moment of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 0.0): The weight decay value for the optimizer. amsgrad (`bool`, defaults to `False`): Whether to use the [AMSGrad](https://hf.co/papers/1904.09237) variant of Adam that uses the maximum of past squared gradients instead. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. is_paged (`bool`, defaults to `False`): Whether the optimizer is a paged optimizer or not. adamis_pagedNsuper__init__selfparamslrbetaseps weight_decayamsgrad optim_bitsargs min_8bit_sizepercentile_clipping block_wiser __class__s g/home/dmtnaga/Documents/work/airagagent/rag_env/lib/python3.13/site-packages/bitsandbytes/optim/adam.pyr Adam.__init__ s<X              gMbP?)g?g+?g:0yE>rF NidTF__name__ __module__ __qualname____firstlineno__r __static_attributes__ __classcell__rs@rrr 0   9 9 rrcB^\rSrSrSU4SjjrSrU=r$)Adam8bitFc |>U(a [S5eUS:wa [S5e[T U] SUUUUUSUU U U U S9 g)a 8-bit Adam optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-3): The learning rate. betas (`tuple(float, float)`, defaults to (0.9, 0.999)): The beta values are the decay rates of the first and second-order moment of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 0.0): The weight decay value for the optimizer. amsgrad (`bool`, defaults to `False`): Whether to use the [AMSGrad](https://hf.co/papers/1904.09237) variant of Adam that uses the maximum of past squared gradients instead. Note: This parameter is not supported in Adam8bit and must be False. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. Note: This parameter is not used in Adam8bit as it always uses 8-bit optimization. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. is_paged (`bool`, defaults to `False`): Whether the optimizer is a paged optimizer or not. z&Adam8bit does not support amsgrad=Truer!zFAdam8bit only supports optim_bits=32 (default value for compatibility)rr N ValueErrorr r rs rr Adam8bit.__init__Gsa^ EF F  ef f             rrr r#r*s@rr-r-F2   D D rr-cB^\rSrSrSU4SjjrSrU=r$) Adam32bitc 6>[T U]SUUUUUSUU U U U S9 g)a 32-bit Adam optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-3): The learning rate. betas (`tuple(float, float)`, defaults to (0.9, 0.999)): The beta values are the decay rates of the first and second-order moment of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 0.0): The weight decay value for the optimizer. amsgrad (`bool`, defaults to `False`): Whether to use the [AMSGrad](https://hf.co/papers/1904.09237) variant of Adam that uses the maximum of past squared gradients instead. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. is_paged (`bool`, defaults to `False`): Whether the optimizer is a paged optimizer or not. rr!r Nr rs rr Adam32bit.__init__s<X              rrr r#r*s@rr6r6r+rr6cB^\rSrSrSU4SjjrSrU=r$) PagedAdamc 6>[T U]SUUUUUUUU U U SS9 g)a Paged Adam optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-3): The learning rate. betas (`tuple(float, float)`, defaults to (0.9, 0.999)): The beta values are the decay rates of the first and second-order moment of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 0.0): The weight decay value for the optimizer. amsgrad (`bool`, defaults to `False`): Whether to use the [AMSGrad](https://hf.co/papers/1904.09237) variant of Adam that uses the maximum of past squared gradients instead. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. is_paged (`bool`, defaults to `False`): Whether the optimizer is a paged optimizer or not. rTr Nr rs rr PagedAdam.__init__s<X              rrr r#r*s@rr;r;r+rr;cB^\rSrSrSU4SjjrSrU=r$) PagedAdam8bitic |>U(a [S5eUS:wa [S5e[T U] SUUUUUSUU U U SS9 g) a 8-bit paged Adam optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-3): The learning rate. betas (`tuple(float, float)`, defaults to (0.9, 0.999)): The beta values are the decay rates of the first and second-order moment of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 0.0): The weight decay value for the optimizer. amsgrad (`bool`, defaults to `False`): Whether to use the [AMSGrad](https://hf.co/papers/1904.09237) variant of Adam that uses the maximum of past squared gradients instead. Note: This parameter is not supported in PagedAdam8bit and must be False. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. Note: This parameter is not used in PagedAdam8bit as it always uses 8-bit optimization. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. is_paged (`bool`, defaults to `False`): Whether the optimizer is a paged optimizer or not. z+PagedAdam8bit does not support amsgrad=Truer!zKPagedAdam8bit only supports optim_bits=32 (default value for compatibility)rr0Tr Nr1rs rr PagedAdam8bit.__init__ sa^ JK K  jk k             rrr r#r*s@rr@r@r4rr@cB^\rSrSrSU4SjjrSrU=r$)PagedAdam32bitiPc 6>[T U]SUUUUUSUU U U SS9 g)a Paged 32-bit Adam optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-3): The learning rate. betas (`tuple(float, float)`, defaults to (0.9, 0.999)): The beta values are the decay rates of the first and second-order moment of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 0.0): The weight decay value for the optimizer. amsgrad (`bool`, defaults to `False`): Whether to use the [AMSGrad](https://hf.co/papers/1904.09237) variant of Adam that uses the maximum of past squared gradients instead. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. is_paged (`bool`, defaults to `False`): Whether the optimizer is a paged optimizer or not. rr!Tr Nr rs rr PagedAdam32bit.__init__Qs<X              rrr r#r*s@rrDrDPr+rrDN)bitsandbytes.optim.optimizerrrr-r6r;r@rDrrrrHs[ 9: ?: zE E P: : z: : zE OE P: _: r