U hB@sdZddlmZddlmZmZmZmZddlZ ddl m Z m Z m Z ddlmZddlmZddlmZdd lmZdd lmZmZdd lmZed d dZGdddeeZGdddeZGdddeeZGdddeeZ dS)z@A chain for comparing the output of two models using embeddings.)Enum)AnyDictListOptionalN)AsyncCallbackManagerForChainRunCallbackManagerForChainRun Callbacks) Embeddings)Field)pre_init)Chain)PairwiseStringEvaluatorStringEvaluatorRUN_KEYreturnc CsZzddlm}WnBtk rRzddlm}Wntk rLtdYnXYnX|S)zaCreate an Embeddings object. Returns: Embeddings: The created Embeddings object. rOpenAIEmbeddingstCould not import OpenAIEmbeddings. Please install the OpenAIEmbeddings package using `pip install langchain-openai`.)langchain_openair ImportError%langchain_community.embeddings.openairrP/tmp/pip-unpacked-wheel-bo69hh5q/langchain/evaluation/embedding_distance/base.py_embedding_factorysrc@s$eZdZdZdZdZdZdZdZdS)EmbeddingDistancea Embedding Distance Metric. Attributes: COSINE: Cosine distance metric. EUCLIDEAN: Euclidean distance metric. MANHATTAN: Manhattan distance metric. CHEBYSHEV: Chebyshev distance metric. HAMMING: Hamming distance metric. ZcosineZ euclideanZ manhattanZ chebyshevZhammingN) __name__ __module__ __qualname____doc__COSINE EUCLIDEAN MANHATTAN CHEBYSHEVHAMMINGrrrrr*s  rc@s>eZdZUdZeedZeed<ee j dZ e ed<e e eefe eefdddZGd d d Zeeed d d ZeedddZe edddZeejejejdddZeejejejdddZeejejejdddZeejejejdddZeejejejdddZ eje!dd d!Z"d"S)#_EmbeddingDistanceChainMixina0Shared functionality for embedding distance evaluators. Attributes: embeddings (Embeddings): The embedding objects to vectorize the outputs. distance_metric (EmbeddingDistance): The distance metric to use for comparing the embeddings. )default_factory embeddings)defaultdistance_metric)valuesrcCs|d}g}zddlm}||Wntk r<YnXzddlm}||Wntk rlYnX|sztdt|t|rz ddl}Wntk rtdYnX|S)zValidate that the TikTok library is installed. Args: values (Dict[str, Any]): The values to validate. Returns: Dict[str, Any]: The validated values. r)rrrNzThe tiktoken library is required to use the default OpenAI embeddings with embedding distance evaluators. Please either manually select a different Embeddings object or install tiktoken using `pip install tiktoken`.) getrrappendrr isinstancetupletiktoken)clsr,r)Ztypes_rr1rrr_validate_tiktoken_installedHs0     z9_EmbeddingDistanceChainMixin._validate_tiktoken_installedc@seZdZUdZeed<dS)z#_EmbeddingDistanceChainMixin.ConfigTarbitrary_types_allowedN)rrr r4bool__annotations__rrrrConfigts r7rcCsdgS)zgReturn the output keys of the chain. Returns: List[str]: The output keys. scorerselfrrr output_keyswsz(_EmbeddingDistanceChainMixin.output_keys)resultrcCs$d|di}t|kr |t|t<|S)Nr8r)r:r<parsedrrr_prepare_outputs  z,_EmbeddingDistanceChainMixin._prepare_output)metricrc CsNtj|jtj|jtj|jtj|jtj |j i}||kr<||St d|dS)zGet the metric function for the given metric name. Args: metric (EmbeddingDistance): The metric name. Returns: Any: The metric function. zInvalid metric: N) rr"_cosine_distancer#_euclidean_distancer$_manhattan_distancer%_chebyshev_distancer&_hamming_distance ValueError)r:r?Zmetricsrrr _get_metrics z(_EmbeddingDistanceChainMixin._get_metric)abrcCs<zddlm}Wntk r,tdYnXd|||S)zCompute the cosine distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.ndarray: The cosine distance. r)cosine_similarityzThe cosine_similarity function is required to compute cosine distance. Please install the langchain-community package using `pip install langchain-community`.g?)Zlangchain_community.utils.mathrIr)rGrHrIrrrr@s  z-_EmbeddingDistanceChainMixin._cosine_distancecCstj||S)zCompute the Euclidean distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Euclidean distance. )npZlinalgZnormrGrHrrrrAs z0_EmbeddingDistanceChainMixin._euclidean_distancecCstt||S)zCompute the Manhattan distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Manhattan distance. )rJsumabsrKrrrrBs z0_EmbeddingDistanceChainMixin._manhattan_distancecCstt||S)zCompute the Chebyshev distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Chebyshev distance. )rJmaxrMrKrrrrCs z0_EmbeddingDistanceChainMixin._chebyshev_distancecCst||kS)zCompute the Hamming distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Hamming distance. )rJZmeanrKrrrrDs z._EmbeddingDistanceChainMixin._hamming_distance)vectorsrcCs6||j}||ddd|ddd}|S)zCompute the score based on the distance metric. Args: vectors (np.ndarray): The input vectors. Returns: float: The computed score. r)rFr+Zreshapeitem)r:rOr?r8rrr_compute_scores &z+_EmbeddingDistanceChainMixin._compute_scoreN)#rrr r!r rr)r r6rr"r+r rstrrr3r7propertyrr;dictr>rF staticmethodrJZndarrayr@ZfloatingrArBrCrDfloatrSrrrrr'<s* "+    r'c @seZdZdZeedddZeedddZee edddZ de ee fe ee ee fd d d Zde ee fe ee ee fd d dZd d d d ddee eee e ee e ee fee edddZd d d d ddee eee e ee e ee fee edddZd S)EmbeddingDistanceEvalChaina"Use embedding distances to score semantic difference between a prediction and reference. Examples: >>> chain = EmbeddingDistanceEvalChain() >>> result = chain.evaluate_strings(prediction="Hello", reference="Hi") >>> print(result) {'score': 0.5} rcCsdS)zReturn whether the chain requires a reference. Returns: bool: True if a reference is required, False otherwise. Trr9rrrrequires_referencesz-EmbeddingDistanceEvalChain.requires_referencecCsd|jjdS)NZ embedding_ _distancer+valuer9rrrevaluation_namesz*EmbeddingDistanceEvalChain.evaluation_namecCsddgS)eReturn the input keys of the chain. Returns: List[str]: The input keys. prediction referencerr9rrr input_keys sz%EmbeddingDistanceEvalChain.input_keysNinputs run_managerrcCs0t|j|d|dg}||}d|iS)a0Compute the score for a prediction and reference. Args: inputs (Dict[str, Any]): The input data. run_manager (Optional[CallbackManagerForChainRun], optional): The callback manager. Returns: Dict[str, Any]: The computed score. r`rar8rJarrayr)Zembed_documentsrSr:rdrerOr8rrr_calls  z EmbeddingDistanceEvalChain._callcs:|j|d|dgIdH}t|}||}d|iS)a:Asynchronously compute the score for a prediction and reference. Args: inputs (Dict[str, Any]): The input data. run_manager (AsyncCallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. r`raNr8r)Zaembed_documentsrJrgrSr:rdreZembeddedrOr8rrr_acall(s    z!EmbeddingDistanceEvalChain._acallF)ra callbackstagsmetadatainclude_run_info)r`rarmrnrorpkwargsrc Ks"|||d||||d}||S)aEvaluate the embedding distance between a prediction and reference. Args: prediction (str): The output string from the first model. reference (str): The reference string (required) callbacks (Callbacks, optional): The callbacks to use. **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. r`rardrmrnrorpr> r:r`rarmrnrorprqr<rrr_evaluate_strings>sz,EmbeddingDistanceEvalChain._evaluate_stringsc s*|j||d||||dIdH}||S)aAsynchronously evaluate the embedding distance between a prediction and reference. Args: prediction (str): The output string from the first model. reference (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rrrsNZacallr>rurrr_aevaluate_strings`s z-EmbeddingDistanceEvalChain._aevaluate_strings)N)N)rrr r!rUr5rZrTr^rrbrrrrrirrlr rVrvrxrrrrrYsb        & rYc @seZdZdZeeedddZeedddZde ee fe e e ee fdd d Z de ee fe ee ee fdd d Zdddd deeee eee e ee fee edddZdddd deeee eee e ee fee edddZdS)"PairwiseEmbeddingDistanceEvalChainaUse embedding distances to score semantic difference between two predictions. Examples: >>> chain = PairwiseEmbeddingDistanceEvalChain() >>> result = chain.evaluate_string_pairs(prediction="Hello", prediction_b="Hi") >>> print(result) {'score': 0.5} rcCsddgS)r_r` prediction_brr9rrrrbsz-PairwiseEmbeddingDistanceEvalChain.input_keyscCsd|jjdS)NZpairwise_embedding_r[r\r9rrrr^sz2PairwiseEmbeddingDistanceEvalChain.evaluation_nameNrccCs0t|j|d|dg}||}d|iS)aCompute the score for two predictions. Args: inputs (Dict[str, Any]): The input data. run_manager (CallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. r`rzr8rfrhrrrris z(PairwiseEmbeddingDistanceEvalChain._callcs:|j|d|dgIdH}t|}||}d|iS)a/Asynchronously compute the score for two predictions. Args: inputs (Dict[str, Any]): The input data. run_manager (AsyncCallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. r`rzNr8rjrkrrrrls    z)PairwiseEmbeddingDistanceEvalChain._acallF)rmrnrorp)r`rzrmrnrorprqrc Ks"|||d||||d}||S)aEvaluate the embedding distance between two predictions. Args: prediction (str): The output string from the first model. prediction_b (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. tags (List[str], optional): Tags to apply to traces metadata (Dict[str, Any], optional): metadata to apply to **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. r`rzrsrt r:r`rzrmrnrorprqr<rrr_evaluate_string_pairssz9PairwiseEmbeddingDistanceEvalChain._evaluate_string_pairsc s*|j||d||||dIdH}||S)aAsynchronously evaluate the embedding distance between two predictions. Args: prediction (str): The output string from the first model. prediction_b (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. tags (List[str], optional): Tags to apply to traces metadata (Dict[str, Any], optional): metadata to apply to traces **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. r{rsNrwr|rrr_aevaluate_string_pairss z:PairwiseEmbeddingDistanceEvalChain._aevaluate_string_pairs)N)N)rrr r!rUrrTrbr^rrrrrirrlr r5rVr}r~rrrrrysZ       ( ry)!r!enumrtypingrrrrZnumpyrJZ langchain_core.callbacks.managerrrr Zlangchain_core.embeddingsr Zlangchain_core.pydantic_v1r Zlangchain_core.utilsr Zlangchain.chains.baser Zlangchain.evaluation.schemarrZlangchain.schemarrrTrr'rYryrrrrs(      7