U ™ÀÂhwã@s^ddlmZmZmZddlmZddlmZeeeeeedœdd„Z Gdd„deeƒZ d S) é)ÚAnyÚCallableÚList)Ú Embeddings)ÚSelfHostedPipeline)ÚpipelineÚargsÚkwargsÚreturncOs |||ŽS)z©Inference function to send to the remote hardware. Accepts a sentence_transformer model_id and returns a list of embeddings for each document in the batch. ©)rrr r r úN/tmp/pip-unpacked-wheel-9gdii04g/langchain_community/embeddings/self_hosted.pyÚ_embed_documentssr c@sheZdZUdZeZeed<dZe ed<Gdd„dƒZ e e e e e dœdd „Ze e e d œd d „ZdS) ÚSelfHostedEmbeddingsa„Custom embedding models on self-hosted remote hardware. Supported hardware includes auto-launched instances on AWS, GCP, Azure, and Lambda, as well as servers specified by IP address and SSH credentials (such as on-prem, or another cloud like Paperspace, Coreweave, etc.). To use, you should have the ``runhouse`` python package installed. Example using a model load function: .. code-block:: python from langchain_community.embeddings import SelfHostedEmbeddings from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline import runhouse as rh gpu = rh.cluster(name="rh-a10x", instance_type="A100:1") def get_pipeline(): model_id = "facebook/bart-large" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained(model_id) return pipeline("feature-extraction", model=model, tokenizer=tokenizer) embeddings = SelfHostedEmbeddings( model_load_fn=get_pipeline, hardware=gpu model_reqs=["./", "torch", "transformers"], ) Example passing in a pipeline path: .. code-block:: python from langchain_community.embeddings import SelfHostedHFEmbeddings import runhouse as rh from transformers import pipeline gpu = rh.cluster(name="rh-a10x", instance_type="A100:1") pipeline = pipeline(model="bert-base-uncased", task="feature-extraction") rh.blob(pickle.dumps(pipeline), path="models/pipeline.pkl").save().to(gpu, path="models") embeddings = SelfHostedHFEmbeddings.from_pipeline( pipeline="models/pipeline.pkl", hardware=gpu, model_reqs=["./", "torch", "transformers"], ) Ú inference_fnNÚinference_kwargsc@seZdZdZdS)zSelfHostedEmbeddings.ConfigZforbidN)Ú__name__Ú __module__Ú __qualname__Úextrar r r r ÚConfigDsr)Útextsr cCs6ttdd„|ƒƒ}| |j|¡}t|tƒs2| ¡S|S)zÊCompute doc embeddings using a HuggingFace transformer model. Args: texts: The list of texts to embed.s Returns: List of embeddings, one for each text. cSs | dd¡S)NÚ ú )Úreplace)Úxr r r ÚPóz6SelfHostedEmbeddings.embed_documents..)ÚlistÚmapÚclientÚ pipeline_refÚ isinstanceÚtolist)ÚselfrÚ embeddingsr r r Úembed_documentsGs  z$SelfHostedEmbeddings.embed_documents)Útextr cCs0| dd¡}| |j|¡}t|tƒs,| ¡S|S)z³Compute query embeddings using a HuggingFace transformer model. Args: text: The text to embed. Returns: Embeddings for the text. rr)rrr r!rr")r#r&r$r r r Ú embed_queryVs  z SelfHostedEmbeddings.embed_query)rrrÚ__doc__r rrÚ__annotations__rrrrÚstrÚfloatr%r'r r r r rs -  rN) ÚtypingrrrZlangchain_core.embeddingsrZ$langchain_community.llms.self_hostedrr+r rr r r r Ús