h"dZddlmZddlZddlZddlZddlmZddlm Z m Z ddl m Z m Z mZmZmZmZddlmZddlmZdd lmZ dd lmZmZmZmZn#e$rdd lmZmZmZmZYnwxYwddlZdd l m!Z!dd l"m#Z#m$Z$m%Z%m&Z&ej'e(Z)Gd deZ*GddedZ+GddeZ,GddedZ-GddZ.ee,e-e/fZ0GddeZ1ee1e/fZ2Gdde.Z3d4dZ4d Z5d5d#Z6Gd$d%Z7d6d(Z8d7d+Z9d8d-Z:d9d1Z;ee e ej&e ej%gee,e-ffe edS);z?This module contains the evaluator classes for evaluating runs.) annotationsN)abstractmethod) AwaitableSequence)AnyCallableLiteralOptionalUnioncast) TypedDict run_helpers)schemas) BaseModelFieldValidationError validator)wraps) SCORE_TYPE VALUE_TYPEExampleRunc*eZdZUdZded< ded<dS)Categoryz$A category for categorical feedback.Optional[Union[float, int]]valuestrlabelN__name__ __module__ __qualname____doc____annotations__k/var/www/html/web-builder-api.evdpl.com/venv/lib/python3.11/site-packages/langsmith/evaluation/evaluator.pyrr/s0..&&&&CJJJ&&r'rcBeZdZUdZded< ded< ded< ded<d S) FeedbackConfigziConfiguration to define a type of feedback. Applied on on the first creation of a feedback_key. z0Literal['continuous', 'categorical', 'freeform']typerminmaxz%Optional[list[Union[Category, dict]]] categoriesNr r&r'r(r*r*8sW ;:::$$$$;$$$$A555555r'r*F)totalc eZdZUdZded< dZded< dZded< dZd ed < dZd ed < e e Z ded< dZ ded< dZ ded< dZded< dZd ed< GddZedddZdS)EvaluationResultzEvaluation result.rkeyNrscorerrz Optional[str]commentzOptional[dict] correction)default_factorydictevaluator_infoz%Optional[Union[FeedbackConfig, dict]]feedback_configOptional[Union[uuid.UUID, str]] source_run_id target_run_idextraceZdZdZdZdS)EvaluationResult.ConfigzPydantic model configuration.FN)r!r"r#r$ allow_extrar&r'r(Configr?bs++ r'rAT)precd|vs|d9t|ttfrtd||S)z$Check that the value is not numeric.r3NzJNumeric values should be provided in the 'score' field, not 'value'. Got: ) isinstanceintfloatloggerwarning)clsvvaluess r(check_value_non_numericz(EvaluationResult.check_value_non_numericgs] & F7O$;!c5\** !!! r')r!r"r#r$r%r3rr4r5rr7r8r9r;r<r=rArrLr&r'r(r1r1Gs@ HHH@E0E8!G!!!!2!%J%%%%: 5666N66665=AOAAAA;59M9999659M9999!E    ) YwD!!!  "!   r'r1ceZdZUdZded<dS)EvaluationResultszqBatch evaluation results. This makes it easy for your evaluator to return multiple metrics at once. zlist[EvaluationResult]resultsNr r&r'r(rNrNvs* $###!!r'rNc>eZdZdZe d dd Z d dd ZdS) RunEvaluatorzEvaluator interface class.NrunrexampleOptional[Example]evaluator_run_idOptional[uuid.UUID]return*Union[EvaluationResult, EvaluationResults]cdS)zEvaluate an example.Nr&)selfrRrSrUs r( evaluate_runzRunEvaluator.evaluate_runsr'cKtjfd}tjd|d{VS)z#Evaluate an example asynchronously.ctjdi5cdddS#1swxYwYdS)Nr&)rhtracing_contextr[)current_contextrUrSrRrZsr(_run_with_contextz5RunEvaluator.aevaluate_run.._run_with_contexts#66o66 I I((g7GHH I I I I I I I I I I I I I I I I I Is 7;;N)r^get_tracing_contextasyncioget_running_looprun_in_executor)rZrRrSrUrar`s```` @r( aevaluate_runzRunEvaluator.aevaluate_runs022 I I I I I I I I I-//??FWXXXXXXXXXr'NNrRrrSrTrUrVrWrX)r!r"r#r$rr[rfr&r'r(rQrQsm$$&*04 ####^#&*04 Y Y Y Y Y Y Yr'rQcJeZdZUdZded< ded< dZded< dZd ed <dS) ComparisonEvaluationResultzFeedback scores for the results of comparative evaluations. These are generated by functions that compare two or more runs, returning a ranking or other feedback. rr2z'dict[Union[uuid.UUID, str], SCORE_TYPE]scoresNr:r;z6Optional[Union[str, dict[Union[uuid.UUID, str], str]]]r4)r!r"r#r$r%r;r4r&r'r(rjrjsb  HHH@3333459M99996FJGJJJJ::r'rjceZdZdZ d$d%dZ d&d'dZd(dZd)dZed*dZ d+d,dZ d+d-fd Z d$d.d!Z d/d#Z xZS)0DynamicRunEvaluatoraA dynamic evaluator that wraps a function and transforms it into a `RunEvaluator`. This class is designed to be used with the `@run_evaluator` decorator, allowing functions that take a `Run` and an optional `Example` as arguments, and return an `EvaluationResult` or `EvaluationResults`, to be used as instances of `RunEvaluator`. Attributes: func (Callable): The function that is wrapped by this evaluator. NfuncXCallable[[Run, Optional[Example]], Union[_RUNNABLE_OUTPUT, Awaitable[_RUNNABLE_OUTPUT]]]afuncIOptional[Callable[[Run, Optional[Example]], Awaitable[_RUNNABLE_OUTPUT]]]ct|\}|rt|\}d fd }t||ddlm}|2||||_t |d d |_tj |rE|td ||||_t |d d |_dS|tttttgt f|||_t |d d |_dS) zInitialize the DynamicRunEvaluator with a given function. Args: func (Callable): A function that takes a `Run` and an optional `Example` as arguments, and returns a dict or `ComparisonEvaluationResult`. inputsr7rWc||S|d|d\}}}|S)NrRrSgetrs_ traced_inputsprepare_inputss r(process_inputsz4DynamicRunEvaluator.__init__..process_inputssK% $2N 5!!6::i#8#8%% !Q=! r'rrNr{r!rmFunc was provided as a coroutine function, but afunc was also provided. If providing both, func should be a regular function to avoid ambiguity.rsr7rWr7)_normalize_evaluator_funcr langsmithrensure_traceablerpgetattr_nameinspectiscoroutinefunction TypeErrorr rrr r_RUNNABLE_OUTPUTrnrZrnrpr{rrzs @r(__init__zDynamicRunEvaluator.__init__s(";4!@!@~  G&?&F&F #UN ! ! ! ! ! ! d D))))))  $55n6DJ! 4IJJDJ  &t , , J 3 %55^6DJ!z3HIIDJJJ#44XsHW$568HHI4PP-5DI!z3HIIDJJJr'FresultUnion[EvaluationResult, dict]r; uuid.UUID allow_no_keyboolrWr1c\ttrjs|_S stddvr |r |jd<t fddDrtdtdid|iS#t $r}td|d}~wwxYw) NziExpected an EvaluationResult object, or dict with a metric 'key' and optional 'score'; got empty result: r2c3 K|]}|vV dSNr&).0krs r( z@DynamicRunEvaluator._coerce_evaluation_result.. s'JJq1F?JJJJJJr')r3rr4zrExpected an EvaluationResult object, or dict with a metric 'key' and optional 'score' or categorical 'value'; got r;z[Expected an EvaluationResult object, or dict with a metric 'key' and optional 'score'; got r&)rDr1r; ValueErrorrallr)rZrr;res ` r(_coerce_evaluation_resultz-DynamicRunEvaluator._coerce_evaluation_results6 f. / / ' 5'4$M   OFLOOF""|" $ u JJJJ,IJJJJJ  XOUXX$QQ &P&PQQ Q   =4:==  sA B B+B&&B+rOUnion[dict, EvaluationResults]rXcd|vr8|}fd|dD|d<tdi|Stt|dS)NrOc>g|]}|S))r;)r)rrrZr;s r( zBDynamicRunEvaluator._coerce_evaluation_results..s<..q .NNr'T)r;rr&)copyrNrr r7)rZrOr;cps` ` r(_coerce_evaluation_resultsz.DynamicRunEvaluator._coerce_evaluation_resultss   B +ByM%**r** *-- w  }4.   r'MUnion[EvaluationResult, EvaluationResults, dict, str, int, bool, float, list]ct|tr|js||_|St|}|||Sr)rDr1r;_format_evaluator_resultr)rZrr;s r(_format_resultz"DynamicRunEvaluator._format_result)sR f. / / ' 5'4$M)&11..v}EEEr'c"t|dSzCheck if the evaluator function is asynchronous. Returns: bool: True if the evaluator function is asynchronous, False otherwise. rphasattrrZs r(is_asynczDynamicRunEvaluator.is_async7tW%%%r'rRrrSrTrUrVct|ds_tj}|rt d||||S|tj}d|j i}t|ddrt|j |d<| ||||d}|||S) aEvaluate a run using the wrapped function. This method directly invokes the wrapped function with the provided arguments. Args: run (Run): The run to be evaluated. example (Optional[Example]): An optional example to be used in the evaluation. Returns: Union[EvaluationResult, EvaluationResults]: The result of the evaluation. rntCannot call `evaluate_run` on an async run evaluator from within an running event loop. Use `aevaluate_run` instead.Nr< session_id experimentrun_idmetadatalangsmith_extra)rrcget_event_loop is_running RuntimeErrorrun_until_completerfuuiduuid4idrrrrnr)rZrRrSrU running_looprrs r(r[z DynamicRunEvaluator.evaluate_run@s"tV$$ Y"133L&&(( Y"R $66t7I7I#w7W7WXXX  ##z|| $3SV#< 3 d + + 9%(%8%8H\ "  '7XNN   ""6+;<<rrs r(__repr__zDynamicRunEvaluator.__repr__s4tz4444r'r)rnrorprq)F)rrr;rrrrWr1)rOrr;rrWrX)rrr;rrWrXrWrrgrh)rRrrSrTrUrV)rRrrSrTrWrXrWr)r!r"r#r$rrrrpropertyrr[rfrr __classcell__)rs@r(rmrms7, 8J8J8J8J8J|# <    " F F F F&&&X&&*04 $=$=$=$=$=R&*04 =======D6://///"55555555r'rmrnroc t|S)zmCreate a run evaluator from a function. Decorator that transforms a function into a `RunEvaluator`. )rmrns r( run_evaluatorrs t $ $$r'i'objrc~t|}t|tkr|dtdz dz}|S)Nz...))reprlen_MAXSIZE)rss r( _maxsize_reprrs: S A 1vv n1 n  & Hr'ceZdZdZ dddZedd Z dddZ dddZ dddZ d dZ e d!dZ d"dZ dS)#DynamicComparisonRunEvaluatorz4Compare predictions (as traces) from 2 or more runs.NrnfCallable[[Sequence[Run], Optional[Example]], Union[_COMPARISON_OUTPUT, Awaitable[_COMPARISON_OUTPUT]]]rpUOptional[Callable[[Sequence[Run], Optional[Example]], Awaitable[_COMPARISON_OUTPUT]]]ct|\}|rt|\}d fd }t||ddlm}|2||||_t |d d |_tj |rE|td ||||_t |d d |_dS|tttttt gt"f|||_t |d d |_dS) zInitialize the DynamicRunEvaluator with a given function. Args: func (Callable): A function that takes a `Run` and an optional `Example` as arguments, and returns an `EvaluationResult` or `EvaluationResults`. rsr7rWc||S|d|d\}}}|S)NrunsrSrurws r(r{z>DynamicComparisonRunEvaluator.__init__..process_inputssK% $2N 6""FJJy$9$9%% !Q=! r'rrNr|r!rmr}r~)$_normalize_comparison_evaluator_funcrrrrrprrrrrr rrrr r_COMPARISON_OUTPUTrnrs @r(rz&DynamicComparisonRunEvaluator.__init__s("Fd!K!K~  R&J5&Q&Q #UN ! ! ! ! ! ! d D))))))  $55n6DJ! 4IJJDJ  &t , , J 3 %55^6DJ!z3HIIDJJJ#44!#(9:*,  .5  DI!z3HIIDJJJr'rWrc"t|dSrrrs r(rz&DynamicComparisonRunEvaluator.is_asyncrr'r Sequence[Run]rSrTrjct|ds_tj}|rt d||||Stj}| |}| ||||d}| |||S)zCompare runs to score preferences. Args: runs: A list of runs to compare. example: An optional example to be used in the evaluation. rnrrtagsr) rrcrrrr acompare_runsrr _get_tagsrn_format_results)rZrrSrr;rrs r( compare_runsz*DynamicComparisonRunEvaluator.compare_runsstV$$ "133L&&(( "R $66&&tW55 ~~d##  '4dCC   ##FM4@@@r'cKt|ds|||Stj}||}|||||dd{V}||||S)aEvaluate a run asynchronously using the wrapped async function. This method directly invokes the wrapped async function with the provided arguments. Args: runs (Run): The runs to be evaluated. example (Optional[Example]): An optional example to be used in the evaluation. Returns: ComparisonEvaluationResult: The result of the evaluation. rprrN)rrrrrrpr)rZrrSr;rrs r(rz+DynamicComparisonRunEvaluator.acompare_runss tW%% 4$$T733 3 ~~d##zz  '4dCC"         ##FM4@@@r'c.|||S)aMake the evaluator callable, allowing it to be used like a function. This method enables the evaluator instance to be called directly, forwarding the call to `evaluate_run`. Args: run (Run): The run to be evaluated. example (Optional[Example]): An optional example to be used in the evaluation. Returns: ComparisonEvaluationResult: The result of the evaluation. )r)rZrrSs r(rz&DynamicComparisonRunEvaluator.__call__:s  w///r'rcd|jdS)rz>>>>r' list[str]cg}|D]g}|dt|jzt|ddr*|dt|jzh|S)zExtract tags from runs.zrun:rNz experiment:)appendrrrr)rrrRs r(rz'DynamicComparisonRunEvaluator._get_tagsOsu A AC KKSV, - - -sL$// A MC,?,??@@@ r'r-Union[dict, list, ComparisonEvaluationResult]r;rct|tr|js||_|St|tr$dt ||D|j|d}n8t|t rd|vr |j|d<nd|}t| tdid|i|S#t$r}td||d}~wwxYw)Nc$i|] \}}|j|Sr&)r)rrRr3s r( zADynamicComparisonRunEvaluator._format_results..fs MMMZS%365MMMr')rkr2r;r2zXExpected 'dict', 'list' or 'ComparisonEvaluationResult' result object. Received: result=r;zExpected a dictionary with a 'key' and dictionary of scores mappingrun IDs to numeric scores, or ComparisonEvaluationResult object, got r&) rDrjr;listziprr7rr)rZrr;rmsgrs r(rz-DynamicComparisonRunEvaluator._format_resultsZs: f8 9 9 "' 5'4$M  % % "MM3tV;L;LMMMz!.FF  % % "F"" $ u /%+// S// ! -"M> >J>J>J>J>J@&&&X&AEAAAAA@AEAAAAA:AE00000"????\""""""r'rrrWc t|S)z.Create a comaprison evaluator from a function.)rrs r(comparison_evaluatorrs ) . ..r'rtuple[Union[Callable[[Run, Optional[Example]], _RUNNABLE_OUTPUT], Callable[[Run, Optional[Example]], Awaitable[_RUNNABLE_OUTPUT]]], Optional[Callable[..., dict]]]cdtjdjD}djD|r;t fd|Ds4t fd|Ddkrdd}t |t fd |Dr|d d gkrdfStjr=dfd dfd }tdrtdn|j |_ |fSdfd dfd }tdrtdn|j |_ |fS)N)rRrSrsoutputsreference_outputs attachmentsc8g|]\}}|j|jk|Sr&kind VAR_KEYWORDrpnameps r(rz-_normalize_evaluator_func..*XXX(%!-@W@W@W@W@Wr'cHg|]\}}|jtjju| Sr&defaultr Parameteremptyrs r(rz-_normalize_evaluator_func..9 E1 9G-3 3 3  3 3 3r'c3(K|] }|vp|vV dSrr&rrargs_with_defaultssupported_argss r(rz,_normalize_evaluator_func..D  GLE^ # Bu0B'B      r'cg|]}|v| Sr&r&rar s r(rz-_normalize_evaluator_func..$DDDq1C(C(C(C(C(Cr'UInvalid evaluator function. Must have at least one argument. Supported arguments are . Please see https://docs.smith.langchain.com/evaluation/how_to_guides/evaluation/evaluate_llm_application#use-custom-evaluatorsc3(K|] }|vp|vV dSrr&r s r(rz,_normalize_evaluator_func..DCH>5,>#>r'rRrSrrTrWtuple[list, dict, dict]c~|||r|jni|jpi|r |jpini|r |jpinid}i}g}i}jD]i\}}||vr`|j|j|jfvr|||n ||||<|dvrt||n||||<j|||fSN)rRrSrsrrr)rRrS rsrr parametersitemsrPOSITIONAL_OR_KEYWORDPOSITIONAL_ONLYrr rRrSarg_mapkwargsargsry param_nameparamsigs r(_prepare_inputsz2_normalize_evaluator_func.._prepare_inputs"&07?gnnR"{0b@G#O7#6#<"RBI)Q)>Br  " ),)=)=)?)?%J!W,, :!7!1*!KK (;<<<<181DF:. *-???*'**=>>>!(!4&j1 V]22r'rcDK||\}}}|i|d{VSrr&rRrSr$r#rxr(rns r(awrapperz+_normalize_evaluator_func..awrappersH%4OC$A$A!vq!T426222222222r'r!c~|||r|jni|jpi|r |jpini|r |jpinid}i}g}i}jD]i\}}||vr`|j|j|jfvr|||n ||||<|dvrt||n||||<j|||fSrrr!s r(r(z2_normalize_evaluator_func.._prepare_inputsr)r'c4||\}}}|i|Srr&r+s r(wrapperz*_normalize_evaluator_func..wrappers0$3OC$A$A!vqtT,V,,,r')rRrrSrTrWr)rRrrSrTrWr r signaturerrrrrrrrr! rnall_argsrr,r/r(r r'rs ` @@@@r(rrsN  D ! !CXXcn&:&:&<&<XXXH,,.. o.      PX     o. DDDDHDDD E E J J G1? G G G ooLT`.    Tz  &t , ,X . 3 3 3 3 3 3> 3 3 3 3 3 3 34,,'j)))&   o. . 3 3 3 3 3 3> - - - - - - - 4,,&j)))%   _- -r'tuple[Union[Callable[[Sequence[Run], Optional[Example]], _COMPARISON_OUTPUT], Callable[[Sequence[Run], Optional[Example]], Awaitable[_COMPARISON_OUTPUT]]], Optional[Callable[..., dict]]]cdtjdjD}djD|r;t fd|Ds4t fd|Ddkrdd}t |t fd |Dr|d d gkrdfStjr=dfd dfd }tdrtdn|j |_ |fSdfd dfd }tdrtdn|j |_ |fS)NrrSrsrrc8g|]\}}|j|jk|Sr&rrs r(rz8_normalize_comparison_evaluator_func..rr'cHg|]\}}|jtjju| Sr&rrs r(rz8_normalize_comparison_evaluator_func..r r'c3(K|] }|vp|vV dSrr&r s r(rz7_normalize_comparison_evaluator_func..%rr'cg|]}|v| Sr&r&rs r(rz8_normalize_comparison_evaluator_func..(rr'rrrc3(K|] }|vp|vV dSrr&r s r(rz7_normalize_comparison_evaluator_func..3rr'rrSrrTrWrcl|||r|jnid|D|r |jpinid}i}g}i}jD]i\}}||vr`|j|j|jfvr|||n ||||<|dvrt||n||||<j|||fS)Nc g|] }|jpi Sr&rrrRs r(rzQ_normalize_comparison_evaluator_func.._prepare_inputs..DBBBc 1rBBBr'r6rrS rsrrrrrr rr rrSr"r#r$ryr%r&r's r(r(z=_normalize_comparison_evaluator_func.._prepare_inputs=!&07?gnnRBBTBBBBI)Q)>Br  " ),)=)=)?)?%J!W,, :!7!1*!KK (;<<<<181DF:. *-@@@*'**=>>>!(!4&j1 V]22r'rcDK||\}}}|i|d{VSrr&rrSr$r#rxr(rns r(r,z6_normalize_comparison_evaluator_func..awrapper[sH%4OD'$B$B!vq!T426222222222r'r!cl|||r|jnid|D|r |jpinid}i}g}i}jD]i\}}||vr`|j|j|jfvr|||n ||||<|dvrt||n||||<j|||fS)Nc g|] }|jpi Sr&r>r?s r(rzQ_normalize_comparison_evaluator_func.._prepare_inputs..qr@r'r6rArBrCs r(r(z=_normalize_comparison_evaluator_func.._prepare_inputsjrDr'c4||\}}}|i|Srr&rFs r(r/z5_normalize_comparison_evaluator_func..wrappers2%4OD'$B$B!vqtT,V,,,r')rrrSrTrWr)rrrSrTrWrr0r2s ` @@@@r(rrsSN  D ! !CXXcn&:&:&<&<XXXH,,.. o,      PX     o, DDDDHDDD E E J J G1? G G G ooLT`,   Tz  &t , ,X , 3 3 3 3 3 3< 3 3 3 3 3 3 34,,'j)))&   _, , 3 3 3 3 3 3< - - - - - - -4,,&j)))%   O+ +r'r;Union[EvaluationResults, dict, str, int, bool, float, list]Union[EvaluationResults, dict]ct|tttfrd|i}n|st d|t|t r1t d|Dst d|dd|i}nBt|trd|i}n(t|trnt d||S) Nr3zdExpected a non-empty dict, str, bool, int, float, list, EvaluationResult, or EvaluationResults. Got c3@K|]}t|tVdSr)rDr7)rxs r(rz+_format_evaluator_result..s,771:a&&777777r'z8Expected a list of dicts or EvaluationResults. Received .rOrzZExpected a dict, str, bool, int, float, list, EvaluationResult, or EvaluationResults. Got ) rDrrFrErrrrr7)rs r(rrs&4,-- 6"   D;A D D    FD ! ! 7777777 T6TTT V$ FC  6" FD ! !   /&, / /    Mr'SUMMARY_EVALUATOR_Tc0dtjdjD}djD|r;t fd|Ds?t fd|Ddkr dd}|r |d |dz }t |t fd |Dr|d d gkrSdfd }tdrtdn|j |_ |S)Nrexamplesrsrrcg|]\}}|Sr&r&rs r(rz0_normalize_summary_evaluator..s===(%===r'cHg|]\}}|jtjju| Sr&rrs r(rz0_normalize_summary_evaluator..r r'c3(K|] }|vp|vV dSrr&r s r(rz/_normalize_summary_evaluator..rr'cg|]}|v| Sr&r&rs r(rz0_normalize_summary_evaluator..rr'rrrOz Received arguments c3 K|]}|vV dSrr&)rrrs r(rz/_normalize_summary_evaluator..s(??U.(??????r'rrSSequence[schemas.Run]Sequence[schemas.Example]rWrXcr||d|Dd|Dd|Dd}i}g} jD]E\}}||vr<|j|j|jfvr|||:||||<F|i|}t |tr|St|S)Ncg|] }|j Sr&)rsrrSs r(rzA_normalize_summary_evaluator..wrapper..sBBBg7>BBBr'c g|] }|jpi Sr&r>r?s r(rzA_normalize_summary_evaluator..wrapper..s>>>#CK-2>>>r'c g|] }|jpi Sr&r>r]s r(rzA_normalize_summary_evaluator..wrapper..s%T%T%Tgo&;%T%T%Tr'rR) rrrrr rrDr1r) rrSr"r#r$r%r&rrnr's r(r/z-_normalize_summary_evaluator..wrappers $BBBBB>>>>>%T%T8%T%T%T GFD%(^%9%9%;%; A A! E((z3-& GJ$78888-4Z-@z*T4*6**F&"233  +F33 3r'r!)rrYrSrZrWrX) rr1rrrrrrrr!)rnr3rr/r r'rs` @@@r(_normalize_summary_evaluatorr`sSN  D ! !C==cn&:&:&<&<===H,,.. 5      PX     5 DDDDHDDD E E J J C1? C C C   6 5(555 5Coo????h??? ? ?&8PDD  4 4 4 4 4 4 4:*1z)B)B XGD* % % %HX r')rnro)rr)rnrrWr)rnrrWr)rnrrWr4)rrJrWrK)rnrrWrP)?r$ __future__rrcrrabcrcollections.abcrrtypingrrr r r r typing_extensionsr rrr^r pydantic.v1rrrr ImportErrorpydanticlogging functoolsrlangsmith.schemasrrrr getLoggerr!rGrr*r1rNrQr7rrjrrmrrrrrrrrrrPr`r&r'r(rmsEE"""""" ////////('''''''''''  BBBBBBBBBBBB  8 $ $'''''y''' 6 6 6 6 6Ye 6 6 6 6,,,,,y,,,^""""" """"YYYYYYYY8)+>>>>>s AA'&A'