;i{:SrSSKJr SSKrSSKrSSKJrJrJrJ r J r J r J r SSK Jr SSK Jr SSK Jr SSK Jr \(a SS KJr SS KJr "S S 5r\"5r"S S5r"SS5r\"5rS/rg)aMake approximate assertions as "expectations" on test results. This module is designed to be used within test cases decorated with the `@pytest.mark.decorator` decorator It allows you to log scores about a test case and optionally make assertions that log as "expectation" feedback to LangSmith. Example usage: .. code-block:: python import pytest from langsmith import expect @pytest.mark.langsmith def test_output_semantically_close(): response = oai_client.chat.completions.create( model="gpt-3.5-turbo", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Say hello!"}, ], ) response_txt = response.choices[0].message.content # Intended usage expect.embedding_distance( prediction=response_txt, reference="Hello!", ).to_be_less_than(0.9) # Score the test case matcher = expect.edit_distance( prediction=response_txt, reference="Hello!", ) # Apply an assertion and log 'expectation' feedback to LangSmith matcher.to_be_less_than(1) # You can also directly make assertions on values directly expect.value(response_txt).to_contain("Hello!") # Or using a custom check expect.value(response_txt).against(lambda x: "Hello" in x) # You can even use this for basic metric logging within tests expect.score(0.8) expect.score(0.7, key="similarity").to_be_greater_than(0.7) ) annotationsN) TYPE_CHECKINGAnyCallableLiteralOptionalUnionoverloadclient) run_helpers) run_trees)utils)EditDistanceConfig)EmbeddingConfigc,\rSrSrSrSSjrSSjrSrg) _NULL_SENTRYKzA sentinel singleton class used to distinguish omitted keyword arguments from those passed in with the value None (which may have different behavior). cg)NFselfs a/home/dmtnaga/Documents/work/airagagent/rag_env/lib/python3.13/site-packages/langsmith/_expect.py__bool___NULL_SENTRY.__bool__Pscg)N NOT_GIVENrrs r__repr___NULL_SENTRY.__repr__SsrrN)returnzLiteral[False])r!str)__name__ __module__ __qualname____firstlineno____doc__rr__static_attributes__rrrrrKsrrc\rSrSrSrSSSjjrSSSjjrSSjrSSjrSSjr SS jr SSS jjr SS jr SS jr SS jrSSjrSrg)_MatcherZz4A class for making assertions on expectation values.NcXlX lX0lU=(d [R"SS9Ul[ R"5nU(aURUl gUUl g)N max_workers) _clientkeyvaluels_utilsContextThreadPoolExecutor _executorrhget_current_run_treetrace_id_run_id)rr r1r2r5run_idrts r__init___Matcher.__init__]sL  "Wh&H&HUV&W  $ $ &&(r{{ f rc[R"5(dgUR(d[R"5UlUR R URRURSUUS9 gg)N expectation)r:r1scorecomment) r3test_tracking_is_disabledr0r;get_cached_clientr5submitcreate_feedbackr9)rr@messages r_submit_feedback_Matcher._submit_feedbacklsa1133<<!335 NN ! ! ,,||! " 4rcU(dU5eURSSURSU3S9 g![a#nURS[U55 USeSnAff=f)Nz Success: .)rFr)rGr1AssertionErrorrepr)r conditionrF method_namees r_assert_Matcher._assertxsd  %g %9  ! !!y !K=-Q ! R   ! !!T!W -  s.1 AAAc ~URURU:SURSUSUR3S5 g)zAssert that the expectation value is less than the given value. Args: value: The value to compare against. Raises: AssertionError: If the expectation value is not less than the given value. Expected z to be less than , but got to_be_less_thanNrQr2r1rr2s rrV_Matcher.to_be_less_thans> JJ z!25'DJJ< P  rc ~URURU:SURSUSUR3S5 g)zAssert that the expectation value is greater than the given value. Args: value: The value to compare against. Raises: AssertionError: If the expectation value is not greater than the given value. rTz to be greater than rUto_be_greater_thanNrWrXs rr[_Matcher.to_be_greater_thans> JJ z!5eWJtzzl S  rc URXRs=:=(a U:Os SURSUSUSUR3S5 g)aAssert that the expectation value is between the given min and max values. Args: min_value: The minimum value (exclusive). max_value: The maximum value (exclusive). Raises: AssertionError: If the expectation value is not between the given min and max. rTz to be between z and rU to_be_betweenNrW)r min_value max_values rr^_Matcher.to_be_betweensP  . .Y .z 5 L | %   rc UR[URU5[X5:HSURSUSUR3S5 g)a3Assert that the expectation value is approximately equal to the given value. Args: value: The value to compare against. precision: The number of decimal places to round to for comparison. Raises: AssertionError: If the rounded expectation value does not equal the rounded given value. rTz to be approximately rUto_be_approximatelyN)rQroundr2r1)rr2 precisions rrc_Matcher.to_be_approximatelysJ $**i (E%,C Cz!6ugZ | T ! rc ~URURU:HSURSUSUR3S5 g)zAssert that the expectation value equals the given value. Args: value: The value to compare against. Raises: AssertionError: If the expectation value does not exactly equal the given value. rTz to be equal to rUto_equalNrWrXs rrh_Matcher.to_equals> JJ% z!1% 4::, O  rcvURURSLSURSUR3S5 g)znAssert that the expectation value is None. Raises: AssertionError: If the expectation value is not None. NrTz to be None, but got to_be_nonerWrs rrk_Matcher.to_be_nones8 JJ$ z!6tzzl C  rcdURXR;SURSUS3S5 g)zAssert that the expectation value contains the given value. Args: value: The value to check for containment. Raises: AssertionError: If the expectation value does not contain the given value. rTz to contain z, but it does not to_containNrWrXs rrn_Matcher.to_contains4 ZZ zeW4E F  rc[R"U5nURU"UR5SUSUR3S5 g)zAssert the expectation value against a custom function. Args: func: A custom function that takes the expectation value as input. Raises: AssertionError: If the custom function returns False. z Assertion z failed for againstN)inspect signaturerQr2r1)rfuncfunc_signatures rrq_Matcher.againstsD!**40  ( TXXJ ?  r)r0r5r9r1r2)NN) r Optional[ls_client.Client]r1r"r2rr5z,Optional[ls_utils.ContextThreadPoolExecutor]r: Optional[str]N)r@intrFrxr!None)rNboolrFr"rOr"r!r{)r2floatr!r{)r_r}r`r}r!r{))r2r}rerzr!r{)r!r{)r2rr!r{)rtrr!r{)r#r$r%r&r'r<rGrQrVr[r^rcrhrkrnrqr(rrrr*r*Zsw>CG $ 5* 5 5 5 @ 5  5     $ "     rr*c\rSrSrSrSS.SSjjrSS.SSjjrSS.SSjjrSS jrS SSS .SS jjr \ SS j5r \ SSj5r \ S4SSjjr SSjr Srg)_Expectz1A class for setting expectations on test results.Nr cXl[R"SS9Ul[R "URR SS9 g)Nr-r.T)wait)r0r3r4executoratexitregistershutdownrr s rr<_Expect.__init__s1  ::qI  ..T:rconfigc (SSKJn U=(d 0nURS5(aSOSnU"US9nURXS9nXVRS.nUR S UUS US UR3S .5 [ URS XpRS 9$)aCompute the embedding distance between the prediction and reference. This logs the embedding distance to LangSmith and returns a `_Matcher` instance for making assertions on the distance value. By default, this uses the OpenAI API for computing embeddings. Args: prediction: The predicted string to compare. reference: The reference string to compare against. config: Optional configuration for the embedding distance evaluator. Supported options: - `encoder`: A custom encoder function to encode the list of input strings to embeddings. Defaults to the OpenAI API. - `metric`: The distance metric to use for comparison. Supported values: "cosine", "euclidean", "manhattan", "chebyshev", "hamming". Returns: A `_Matcher` instance for the embedding distance value. Examples: >>> expect.embedding_distance( ... prediction="hello", ... reference="hi", ... ).to_be_less_than(1.0) r)EmbeddingDistanceencodercustomopenair prediction reference)rmetricembedding_distanceUsing z , Metric: r@ source_inforAr5) 'langsmith._internal._embedding_distancergetevaluatedistancerGr*r0r) rrrrr encoder_func evaluatorr@src_infos rr_Expect.embedding_distancesF N2#)::i#8#8xh %V4 ""j"N+7I7IJ  '#L>I>> expect.edit_distance("hello", "helo").to_be_less_than(1) r) EditDistancerdamerau_levenshteinnormalize_scoreTrr)r normalize edit_distancerz , Normalize: rr)"langsmith._internal._edit_distancerrrrGr*r0r) rrrrrrrrr@rs rr_Expect.edit_distance=s< D2H%>)>JJ0$7  / ""j"N$=  '#F8= D   LL  mm   rc@[URSXRS9$)zCreate a `_Matcher` instance for making assertions on the given value. Args: value: The value to make assertions on. Returns: A `_Matcher` instance for the given value. Examples: >>> expect.value(10).to_be_less_than(20) r2r)r*r0rrXs rr2 _Expect.valuers gu NNrr@)r1 source_run_idrAcrURUUSS0UUS.5 [URX!URS9$)a?Log a numeric score to LangSmith. Args: score: The score value to log. key: The key to use for logging the score. Defaults to "score". Examples: >>> expect.score(0.8) # doctest: +ELLIPSIS >>> expect.score(0.8, key="similarity").to_be_greater_than(0.7) methodz expect.score)r@rrrAr)rGr*r0r)rr@r1rrAs rr@ _Expect.scoresD(   (.9!."    cDMMJJrcgryrrXs r__call___Expect.__call__s36rcgryrrs rrrsCFrcL[US9nU[LaURU5$U$)Nr )rrr2)rr2r expecteds rrrs* &)  !>>%( (rcJ[R"5nU(a UROSn[R"5(d_UR (d[ R"5UlURR"UR R4XAS.UD6 gg)N)r:r1) r6r7r8r3rBr0r;rCrrDrE)rr1results current_runr:s rrG_Expect._submit_feedbacksy--/ )4%%$1133<<!335 MM  ,, 5; HO 4r)r0r)r rw)rr"rr"rzOptional[EmbeddingConfig]r!r*)rr"rr"rzOptional[EditDistanceConfig]r!r*)r2rr!r*) r@zUnion[float, int, bool]r1r"rzOptional[ls_client.ID_TYPE]rArxr!r*)r zls_client.Clientr!r)r rwr2z Optional[Any]r!zUnion[_Expect, _Matcher])r1r"rdict)r#r$r%r&r'r<rrr2r@r rrrGr(rrrrrs;?C;-1 4 4 4 * 4  4 v04 3 3 3 - 3  3 j O$59!% K&K K 3 K  K KB66 FF )-1 +   " rrexpect) r' __future__rrrrtypingrrrrrr r langsmithr ls_clientr r6rr;rr3rrrrrrr*rr__all__rrrrsy/b# *'%'EG   N b b J{{|  *r