ߔ9i?ddlZddlZddlmZddlmZmZmZmZm Z m Z ddl Z ddl m Z mZddlmZmZddlmZddlmZmZmZerddlZej2eZe d d ZGd d ZGd deZy)N)Path)DictListOptionalTypeTypeVarUnion) CONFIG_NAMEPYTORCH_WEIGHTS_NAME)hf_hub_downloadis_torch_available)HfApi)SoftTemporaryDirectoryloggingvalidate_hf_hub_argsT ModelHubMixin)boundceZdZdZdddddeeefdeedeede d eef d Z ded dfd Z e e dddddddd d eedeeefde de deedeeee fdeeeefde deed efdZe d eededeedeeeefde deede de deeee fd efdZe ddddddddddd dedeedede deedeedeedee d eeeeefd!eeeeefd"eeeeefd efd#Zy)$ra A generic mixin to integrate ANY machine learning framework with the Hub. To integrate your framework, your model class must inherit from this class. Custom logic for saving/loading models have to be overwritten in [`_from_pretrained`] and [`_save_pretrained`]. [`PyTorchModelHubMixin`] is a good example of mixin integration with the Hub. Check out our [integration guide](../guides/integrations) for more instructions. NF)configrepo_id push_to_hubsave_directoryrrrreturnc Nt|}|jdd|j|t|tr+|t z j tj||r9|j}|||d<| |j}|jdd|i|Sy)a Save weights in local directory. Args: save_directory (`str` or `Path`): Path to directory in which the model weights and configuration will be saved. config (`dict`, *optional*): Model configuration specified as a key/value dictionary. push_to_hub (`bool`, *optional*, defaults to `False`): Whether or not to push your model to the Huggingface Hub after saving it. repo_id (`str`, *optional*): ID of your repository on the Hub. Used only if `push_to_hub=True`. Will default to the folder name if not provided. kwargs: Additional key word arguments passed along to the [`~ModelHubMixin._from_pretrained`] method. T)parentsexist_okNrr) rmkdir_save_pretrained isinstancedictr write_textjsondumpscopynamer)selfrrrrkwargss c/var/www/html/backtest/airagagent/rag_env/lib/python3.12/site-packages/huggingface_hub/hub_mixin.pysave_pretrainedzModelHubMixin.save_pretrained s2n-TD9 n- fd # k ) 5 5djj6H I [[]F!#)x (--#4##>G>v> >ct)aD Overwrite this method in subclass to define how to save your model. Check out our [integration guide](../guides/integrations) for instructions. Args: save_directory (`str` or `Path`): Path to directory in which the model weights and configuration will be saved. NotImplementedError)r)rs r+r!zModelHubMixin._save_pretrainedLs "!r-)force_downloadresume_downloadproxiestoken cache_dirlocal_files_onlyrevisionclspretrained_model_name_or_pathr1r2r3r4r5r6r7c |} d} tjj| rvttj| vr%tjj | t} nht jtdt| jn2t| tr" tt| t||||||| } | ?t!| dd5} t#j$| } ddd| j'd i|j(d t| |||||||d | S#tjj$rt jtdYwxYw#1swYxYw) a Download a model from the Huggingface Hub and instantiate it. Args: pretrained_model_name_or_path (`str`, `Path`): - Either the `model_id` (string) of a model hosted on the Hub, e.g. `bigscience/bloom`. - Or a path to a `directory` containing model weights saved using [`~transformers.PreTrainedModel.save_pretrained`], e.g., `../path/to/my_model_directory/`. revision (`str`, *optional*): Revision of the model on the Hub. Can be a branch name, a git tag or any commit id. Defaults to the latest commit on `main` branch. force_download (`bool`, *optional*, defaults to `False`): Whether to force (re-)downloading the model weights and configuration files from the Hub, overriding the existing cache. resume_download (`bool`, *optional*, defaults to `False`): Whether to delete incompletely received files. Will attempt to resume the download if such a file exists. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on every request. token (`str` or `bool`, *optional*): The token to use as HTTP bearer authorization for remote files. By default, it will use the token cached when running `huggingface-cli login`. cache_dir (`str`, `Path`, *optional*): Path to the folder where cached files are stored. local_files_only (`bool`, *optional*, defaults to `False`): If `True`, avoid downloading the file and return the path to the local cached file if it exists. model_kwargs (`Dict`, *optional*): Additional kwargs to pass to the model during initialization. Nz not found in rfilenamer7r5r1r3r2r4r6z not found in HuggingFace Hub.rzutf-8)encodingr)model_idr7r5r1r3r2r6r4r)ospathisdirr listdirjoinloggerwarningrresolver"strr requests exceptionsRequestExceptionopenr%loadupdate_from_pretrained)r8r9r1r2r3r4r5r6r7 model_kwargsr? config_filefrs r+from_pretrainedzModelHubMixin.from_pretrainedWs`X1%) 77== "bjj22 ggll8[A +nT(^=S=S=U;?;?DD D  D  DsmD}D DD>D!tCy#~!67D"%S 3"78D"%S 3"78D DDr-ceZdZdZdeddfdZedddd ed eed ee eefd e d ee de de de ee dfdede fdZ y)PyTorchModelHubMixinaL Implementation of [`ModelHubMixin`] to provide model Hub upload/download capabilities to PyTorch models. The model is set in evaluation mode by default using `model.eval()` (dropout modules are deactivated). To train the model, you should first set it back in training mode with `model.train()`. Example: ```python >>> import torch >>> import torch.nn as nn >>> from huggingface_hub import PyTorchModelHubMixin >>> class MyModel(nn.Module, PyTorchModelHubMixin): ... def __init__(self): ... super().__init__() ... self.param = nn.Parameter(torch.rand(3, 4)) ... self.linear = nn.Linear(4, 5) ... def forward(self, x): ... return self.linear(x + self.param) >>> model = MyModel() # Save model weights to local directory >>> model.save_pretrained("my-awesome-model") # Push model weights to the Hub >>> model.push_to_hub("my-awesome-model") # Download and initialize weights from the Hub >>> model = MyModel.from_pretrained("username/my-awesome-model") ``` rrNct|dr |jn|}tj|j |t z y)z7Save weights from a Pytorch model to a local directory.moduleN)hasattrrptorchsave state_dictr )r)r model_to_saves r+r!z%PyTorchModelHubMixin._save_pretrainedGs3'.tX'> D  =++-~@T/TUr-cpuF) map_locationstrictr?r7r5r1r3r2r6r4rwrxc ~tjj|r0tdtjj |t } nt |t ||||||| } |di| } tj| tj| }| j|| | j| S)zrsl ==8>HH   H % C'JJZN=Nr-