9io5SSKrSSKrSSKrSSKrSSKJr SSKrSSKJ r SSK r SSK J r J r SSKJrJrJrJr \R&"\5rS\S\4Sjr"S S 5r"S S \5rS\\4S jrS\S\4Sjr"SS\5rS\4Sjr\"5rSr Sr!\ RDRF(aSr Sr!O?\ RHRK5(aSr!O\ RLRN(aSr!\"5r(g!\)a@r*\!S;a \"S5r(Sr*C*g\"\*5r+\RYS\+3SS9 \"\+5r(Sr*C*gSr*C*ff=f)N)Path)Optional)DYNAMIC_LIBRARY_SUFFIX PACKAGE_DIR) CUDASpecsget_cuda_specsget_cuda_version_tupleget_rocm_gpu_arch cuda_specsreturnc[RR(aSOSnSUUR[3n[ R RS5nU(ad[R"SSU3USS9n[RR(a[SUS 35e[RS US US 35 [U- $) z Get the disk path to the CUDA BNB native library specified by the given CUDA specs, taking into account the `BNB_CUDA_VERSION` override environment variable. The library is not guaranteed to exist at the returned path. rocmcudalibbitsandbytes_BNB_CUDA_VERSIONzcuda\d+)countzBNB_CUDA_VERSION=zM detected for ROCm!! Clear the variable and retry: export BNB_CUDA_VERSION= zWARNING: BNB_CUDA_VERSION=z( environment variable detected; loading z. This can be used to load a bitsandbytes version built with a CUDA version that is different from the PyTorch CUDA version. If this was unintended set the BNB_CUDA_VERSION variable to an empty string: export BNB_CUDA_VERSION= )torchversionhipcuda_version_stringrosenvirongetresub RuntimeErrorloggerwarningr)r prefix library_nameoverride_values g/home/dmtnaga/Documents/work/airagagent/rag_env/lib/python3.13/site-packages/bitsandbytes/cextension.pyget_cuda_bnb_library_pathr$s}}((VfF%fXj.L.L-MNdMefLZZ^^$67NvvjD0@*A[STS35e)NzMethod 'zm' not available in CPU-only version of bitsandbytes. Reinstall with GPU support or use CUDA-enabled hardware.)r)argskwargsnames r# throw_on_call3BNBNativeLibrary.__getattr__..throw_on_call;s#4&!KK r%)getattrr))r.r5fnr6s ` r# __getattr__BNBNativeLibrary.__getattr__4s, TYYd + >I  r%c$URU5$r,r:)r.items r# __getitem__BNBNativeLibrary.__getitem__C%%r%r-N)__name__ __module__ __qualname____firstlineno__ctCDLL__annotations__compiled_with_cudar/ functoolscacher:r?__static_attributes__r%r#r'r'-s> ''MBGG__  &r%r'cH^\rSrSrSrS\R 4U4SjjrSrU=r $)CudaBNBNativeLibraryGTr*c>[TU]U5 [RURl[RUR l[RURlgr,)superr/rFc_void_p get_contextrestype get_cusparsecget_managed_ptr)r.r* __class__s r#r/CudaBNBNativeLibrary.__init__JsD "$++#%;; '){{$r%rM) rBrCrDrErIrFrGr/rL __classcell__)rXs@r#rOrOGs3BGG33r%rOcS[R5S[3n/n[[5R R U5H}n[R5S3n[R"X2R5nU(dMC[URS55nUS-nUS-nURUSU35 M [U5$)zNGet formatted CUDA versions from existing library files using cuda_specs logicr*z(\d+)r .) BNB_BACKENDlowerrr__file__parentglobrsearchr5intgroupappendsorted) lib_patternversionsr*patternmatchver_codemajorminors r#"get_available_cuda_binary_versionsrpQs$[%6%6%8$9;Q:RSKHH~$$))+6!'')*%0 '88, 55;;q>*HNErME OOugQug. /7 ( r% version_strcJUR5(a USSSUS3$U$)zWConvert raw version string (e.g. '118' from env var) to formatted version (e.g. '11.8')Nr^)isdigit)rqs r#parse_cuda_versionru`s5cr"#1[_$566 r%c r\rSrSrSrS\4SjrSS\\S\S\S \\S \4 S jjr S \4S jr S r Sr Sr g) ErrorHandlerMockBNBNativeLibraryga Mock library handler that defers errors until native methods are called. This class serves as a fallback when the native bitsandbytes library fails to load. It captures the original error and generates detailed troubleshooting guidance. Key behaviors: - Allows attribute access and method assignment without immediate errors - Throws a RuntimeError with diagnostic information only when a native method is called, as otherwise it would error out on import, breaking backward compatibility - Handles both missing CUDA dependencies and version mismatch scenarios Error scenarios covered: 1. Missing shared library dependencies (e.g., libcudart.so not in LD_LIBRARY_PATH or through PyTorch CUDA installation) 2. CUDA version mismatch between PyTorch and available pre-compiled binaries 3. Completely missing pre-compiled binaries when CUDA is detected 4. Custom BNB_CUDA_VERSION override but mismatch 5. CPU-only installation attempts when GPU functionality is requested error_msgcXl[5Ul[5Ul[ R RS5UlUR(a[UR5O3UR(a!URSSURS3OSUl SU;aUR5Ul gURURUR(a!URSSURS3OSUR(aSURS3OS URS 9Ul g) Nrrr^runknownzcannot open shared object filezOriginal error:  )available_versionsuser_cuda_versionoriginal_errorrequested_version)ryr rrpr~rrrr"rur_format_dependency_errorformatted_error_format_lib_error_message)r.rys r#r/)ErrorHandlerMockBNBNativeLibrary.__init__|s""!7!9"D"F jjnn-?@"" t22 3%%**1-.a0F0Fq0I/JK  ,y 8#'#@#@#BD #'#A#A#'#:#:))&*%;%;A%>$?qAWAWXYAZ@["\HL!1$..1AD\^"&"8"8 $B$D r%Nr~rrrr c SnSU;n[S3U;nU(aSnONU(aGU(aSSRU5-OSnU(a!S[S[S US [S US US 3 OSnSn U(a S[S3O S[S3n U(aS[S[S3OSn U(dSO[(dSOSn S[R5S3n UU U U U USU 3$)z:Format detailed error message for library loading failuresr}z#libbitsandbytes_cpu.so: cannot openz binary not foundu9 🚨 Failed to load CPU-only bitsandbytes library 🚨 z - NONE 🚨 u! VERSION MISMATCH 🚨 Requested z version: z Detected PyTorch z version: z" Available pre-compiled versions: zU This means: The version you're trying to use is NOT distributed with this package u 🚨 Forgot to compile the bitsandbytes library? 🚨 1. You're not using the package but checked-out the source code 2. You MUST compile from source zTAttempted to use bitsandbytes native library functionality but it's not available. z^This typically happens when: 1. bitsandbytes doesn't ship with a pre-compiled binary for your zR version 2. The library wasn't compiled properly during installation from source zlThis typically happens when you checked the code out from source and your torch installation doesn't detect z on your machine. zVTo make bitsandbytes work, the compiled library version MUST exactly match the linked z version. If your zL version doesn't have a pre-compiled binary, you MUST compile from source. zPCOMPILE FROM SOURCE for CPU-only: `cmake -DCOMPUTE_BACKEND=cpu -S . && make` aNYou have two options: 1. COMPILE FROM SOURCE (required if no binary exists): https://huggingface.co/docs/bitsandbytes/main/en/installation#cuda-compile 2. Use BNB_CUDA_VERSION to specify a DIFFERENT CUDA version from the detected one, which is installed on your machine and matching an available pre-compiled version listed above zYou can COMPILE FROM SOURCE as mentioned here: https://huggingface.co/docs/bitsandbytes/main/en/installation?backend=AMD+ROCm#amd-gpu u🔍 Run this command for detailed diagnostics: python -m bitsandbytes If you've tried everything and still have issues: 1. Include ALL version info (operating system, bitsandbytes, pytorch, , python) 2. Describe what you've tried in detail 3. Open an issue with this information: https://github.com/bitsandbytes-foundation/bitsandbytes/issues r|)r_joinHIP_ENVIRONMENTr`)r.r~rrranalysisno_cpu_lib_foundno_cuda_lib_foundversion_list_strbase_msgtroubleshootingnotecompile_instructions diagnosticss r#r:ErrorHandlerMockBNBNativeLibrary._format_lib_error_messages@NR*m+<=O UH Oax(--8J*KKgm &k]+!!, -@AR@ST((3}LAR@ST88H7IJ``6 l! TT_S`a^_ @AL@MMbc !iithuv&-'uw %c#? Hn "UVaUfUfUhTijT U H:o%6tf=Q[TRSTS35e)Nz*Native code method attempted to call: lib.z())rr)r3r4r5r.s r#r6CErrorHandlerMockBNBNativeLibrary.__getattr__..throw_on_calls&$"6"6!77abfaggijk kr%rM)r.r5r6s`` r#r:,ErrorHandlerMockBNBNativeLibrary.__getattr__s lr%c$URU5$r,r=)r.r5s r#r?,ErrorHandlerMockBNBNativeLibrary.__getitem__ rAr%)r~ryrr"rr)r}N)rBrCrDrE__doc__strr/listrrrr:r?rLrMr%r#rwrwgs{(#<!+/ Os IOsOs Os $C= Os Osb # 8&r%rwc[5n[S[3- nU(a7[U5nUR 5(d[ S[ SU35eUn[RR(a[S[3- n[RSU35 [RR[U55n[!US5(a [#U5$[%U5$)zV Load CUDA library XOR CPU, as the latter contains a subset of symbols of the former. libbitsandbytes_cpuz Configured z binary not found at libbitsandbytes_xpuz*Loading bitsandbytes native library from: rT)rrrr$existsrr_r_C_has_xpurdebugrFcdll LoadLibraryrhasattrrOr')r binary_pathcuda_binary_pathdlls r#get_native_libraryrs !J"56L5M NNK4Z@&&((9NO_N`ab b&  xx!&9:P9Q$RR  LL=k]KL ''  c+. /CsM""#C(( C  r%FCPUTROCmCUDAXPU)rrz'XPU/CPU can run without native library.z!bitsandbytes library load error: )exc_info)-ctypesrFrJloggingrpathlibrrtypingrrbitsandbytes.constsrrbitsandbytes.cuda_specsrrr r getLoggerrBrr$r'rOrrrprurwr ROCM_GPU_ARCHrr_rrr is_availablerrr* ExceptioneryerrorrMr%r#rsX   Chh   8 $&)&&8&&43+3 DI CCe&'7e&P!,!:"#  ==OK ZZK XXK :  C :n$./XYF  / { ;  /y9 :s9DEE#EE