;iL$|SSKrSSKJr SSKJr "SS\5r"SS\5r"SS \5r"S S \5rg) N) Optimizer)Optimizer1Statec>^\rSrSrSU4SjjrSrU=r$)LARS c Z>US:Xa [S5e[T U] SUUX44SUUUU U U SS9 g)a Base LARS optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`): The learning rate. momentum (`float`, defaults to 0): The momentum value speeds up the optimizer by taking bigger steps. dampening (`float`, defaults to 0): The dampening value reduces the momentum of the optimizer. weight_decay (`float`, defaults to 1e-2): The weight decay value for the optimizer. nesterov (`bool`, defaults to `False`): Whether to use Nesterov momentum. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. max_unorm (`float`, defaults to 0.02): The maximum gradient norm. r'LARS without momentum is not supported!larsF max_unorm block_wiseNNotImplementedErrorsuper__init__) selfparamslrmomentum dampening weight_decaynesterov optim_bitsargs min_8bit_sizepercentile_clippingr __class__s g/home/dmtnaga/Documents/work/airagagent/rag_env/lib/python3.13/site-packages/bitsandbytes/optim/lars.pyr LARS.__init__ sRR q=%&OP P      !        ) rrrF Nd{Gz?__name__ __module__ __qualname____firstlineno__r__static_attributes__ __classcell__rs@rrr s*  8 8 r!rc<^\rSrSrSU4SjjrSrU=r$)LARS8bitGc Z>US:Xa [S5e[T U] SUUX44SUSUUU U SS9 g) a% 8-bit LARS optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`): The learning rate. momentum (`float`, defaults to 0): The momentum value speeds up the optimizer by taking bigger steps. dampening (`float`, defaults to 0): The dampening value reduces the momentum of the optimizer. weight_decay (`float`, defaults to 1e-2): The weight decay value for the optimizer. nesterov (`bool`, defaults to `False`): Whether to use Nesterov momentum. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. max_unorm (`float`, defaults to 0.02): The maximum gradient norm. rr r r Fr Nr rrrrrrrrrrr rs rrLARS8bit.__init__HsRL q=%&OP P      !       r!r"rrrFNr$r%r&r'r.s@rr0r0G'  5 5 r!r0c<^\rSrSrSU4SjjrSrU=r$) LARS32bitc Z>US:Xa [S5e[T U] SUUX44SUSUUU U SS9 g) a& 32-bit LARS optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`): The learning rate. momentum (`float`, defaults to 0): The momentum value speeds up the optimizer by taking bigger steps. dampening (`float`, defaults to 0): The dampening value reduces the momentum of the optimizer. weight_decay (`float`, defaults to 1e-2): The weight decay value for the optimizer. nesterov (`bool`, defaults to `False`): Whether to use Nesterov momentum. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. max_unorm (`float`, defaults to 0.02): The maximum gradient norm. rr r r r#Fr Nrr4s rrLARS32bit.__init__sRL q=%&OP P      !        r!r"r6r'r.s@rr9r9r7r!r9cv^\rSrSrSU4SjjrU4Sjr\R"5SSj5rSr U=r $) PytorchLARSc >US:a[SU35eUS:a[SU35eUS:a[SU35e[UUUUUUS9nU(aUS::dUS:wa [S5e[T U] X5 g)Nr zInvalid learning rate: zInvalid momentum value: zInvalid weight_decay value: )rrrrrr rz8Nesterov momentum requires a momentum and zero dampening) ValueErrordictrr) rrrrrrrr defaultsrs rrPytorchLARS.__init__s 86rd;< < c>7zBC C # ;L>JK K%   Q)q.WX X *r!cn>[TU]U5 URHnURSS5 M g)NrF)r __setstate__ param_groups setdefault)rstategrouprs rrFPytorchLARS.__setstate__s0 U#&&E   Z /'r!czSnUb%[R"5 U"5nSSS5 URGHmnUSnUSnUSnUSnUSnUSn USGHAn U RcMURU n U Rn US :waU R XS 9n US :waqU R S S5n U c)[R"U 5R5n XS 'O"U RU5RU S U- S 9 U(aXU--nOU nS nUS:akU R[R:Xde[R"U R55n[R"W5nUUU-:aUU-U- nU RWU *U-S 9 GMD GMp U$!,(df  GN=f)zPerforms a single optimization step. Args: closure (callable, optional): A closure that reevaluates the model and returns the loss. Nrrrrr rrr)alphamomentum_bufferg?r )torch enable_gradrGgradrIaddgetclonedetachmul_add_dtypefloat32norm)rclosurelossrJrrrrr rprId_pbufupdate update_scalepnormunorms rstepPytorchLARS.steps  ""$y%&&E 0LZ(Hk*IZ(Hk*ItB8_66> 1 ff1$''!'8Cq=))$5tr>s; +:0  ]]_55r!r>) rP torch.optimrbitsandbytes.optim.optimizerrrr0r9r>r"r!rrjsE  !89 ?9 x6 6 r6 6 rY)Yr!