U h@sddlZddlmZmZmZmZmZmZmZddl Z ddl m Z ddl m Z ddlmZddlmZmZGdddZGd d d e ZGd d d ZGd dde ZdS)N)AnyDict GeneratorIteratorListOptionalUnion)CallbackManagerForLLMRun)LLM)GenerationChunk)get_from_dict_or_envpre_initc@seZdZUdZdZeed<edddZee j e ddd Z ee j e e d d fdd d Zed ddZdeeeeeeefeeee dddZdeeeeeeefeeee dddZd S)SVEndpointHandlerzy SambaNova Systems Interface for Sambaverse endpoint. :param str host_url: Base URL of the DaaS API service z /api/predict API_BASE_PATH)host_urlcCs||_t|_dS)zr Initialize the SVEndpointHandler. :param str host_url: Base URL of the DaaS API service N)rrequestsSession http_session)selfrrF/tmp/pip-unpacked-wheel-9gdii04g/langchain_community/llms/sambanova.py__init__szSVEndpointHandler.__init__responsereturnc Csi}z|jd}|d}|jdkrt|drd}|ddD]"}|t|ddd d 7}qH|d }t|}||ddd d <n t|}Wn.tk r}zt||d <W5d}~XYnXd|kr|j|d<|S)L Processes the API response and returns the resulting dict. All resulting dicts, regardless of success or failure, will contain the `status_code` key with the API response status code. If the API returned an error, the resulting dict will contain the key `detail` with the error message. If the API call was successful, the resulting dict will contain the key `data` with the response data. :param requests.Response response: the response object to process :return: the response dict :type: dict  errorNresult responsesr stream_token completiondetail status_code) textstripsplitr'jsonloadsget Exceptionstr)rr!Z lines_resultZ text_resultr%lineerrr_process_responses&  z#SVEndpointHandler._process_responseNc cszf|D]X}t|}d|kr*|j|d<|ddkr\|dr\dddigi|d<|WS|Vq Wn0tk r}ztd|W5d }~XYnXd S) Process the streaming responser'rrr"r#r r!%Error processing streaming response: N) iter_linesr+r,r'r-r. RuntimeError)rr0chunkr1rrr_process_streaming_responseDs     z-SVEndpointHandler._process_streaming_responsercCs|j|jS)z Return the full API URL for a given path. :returns: the full API URL for the sub-path :type: str )rrrrrr _get_full_urlUszSVEndpointHandler._get_full_urlr F)keysambaverse_model_nameinputparamsstreamrcCsF|r|t|d}nd|i}|jj||d|d|d}t|S)Y NLP predict using inline input string. :param str project: Project ID in which the endpoint exists :param str endpoint: Endpoint ID :param str key: API Key :param str input_str: Input string :param str params: Input params string :returns: Prediction results :type: dict instancer?rCapplication/jsonr<z Content-TypeZ modelNameheadersr+)r+r,rpostr;rr2)rr<r=r>r?r@datarrrr nlp_predict]s zSVEndpointHandler.nlp_predict)r<r=r>r?rccsX|r|t|d}nd|i}|jj||d|d|dd}t|D] }|VqHdS)rArBrCrDrETrGr+r@N)r+r,rrHr;rr8)rr<r=r>r?rIrr7rrrnlp_predict_streams z$SVEndpointHandler.nlp_predict_stream)r F)r )__name__ __module__ __qualname____doc__rr/__annotations__r staticmethodrResponserr2rr8r;rrrboolrJrrLrrrrr s8   &   'rc@seZdZUdZdZeed<dZeed<dZe eed<dZ e e ed<dZ e e ed <Gd d d Zee d d dZeeedddZeeeefd ddZeed ddZe eeedddZeeeeefeedddZeeeefe eeedddZeeeeefeeeddd Z d(eeeefe eee e!eeed!d"d#Z"eeeefe eee e!eeefed!d$d%Z#d)eeeefe eee e!eed!d&d'Z$dS)* Sambaversea Sambaverse large language models. To use, you should have the environment variable ``SAMBAVERSE_API_KEY`` set with your API key. get one in https://sambaverse.sambanova.ai read extra documentation in https://docs.sambanova.ai/sambaverse/latest/index.html Example: .. code-block:: python from langchain_community.llms.sambanova import Sambaverse Sambaverse( sambaverse_url="https://sambaverse.sambanova.ai", sambaverse_api_key="your-sambaverse-api-key", sambaverse_model_name="Meta/llama-2-7b-chat-hf", streaming: = False model_kwargs={ "select_expert": "llama-2-7b-chat-hf", "do_sample": False, "max_tokens_to_generate": 100, "temperature": 0.7, "top_p": 1.0, "repetition_penalty": 1.0, "top_k": 50, "process_prompt": False }, ) r sambaverse_urlsambaverse_api_keyNr= model_kwargsF streamingc@seZdZdZdS)zSambaverse.ConfigforbidNrMrNrOextrarrrrConfigsr]r9cCsdSNTrclsrrris_lc_serializableszSambaverse.is_lc_serializablevaluesrcCs8t|dddd|d<t|dd|d<t|dd|d<|S) z,Validate that api key exists in environment.rVZSAMBAVERSE_URLzhttps://sambaverse.sambanova.aidefaultrWZSAMBAVERSE_API_KEYr=ZSAMBAVERSE_MODEL_NAMEr r`rcrrrvalidate_environments" zSambaverse.validate_environmentcCs d|jiSzGet the identifying parameters.rXrXr:rrr_identifying_paramsszSambaverse._identifying_paramscCsdS)Return type of llm.zSambaverse LLMrr:rrr _llm_typeszSambaverse._llm_typestoprcCsb|jpi}|dg}|p|}|s:ddd|D|d<dd|D}||d<t|}|S)1 Get the tuning parameters to use when calling the LLM. Args: stop: Stop words to use when generating. Model output is cut off at the first occurrence of any of the stop substrings. Returns: The tuning parameters as a JSON string. stop_sequences,css|]}d|dVqdS)"Nr).0xrrr sz0Sambaverse._get_tuning_params..cSs&i|]\}}|t|jt|dqS)typevaluerxrMr/rtkvrrr sz1Sambaverse._get_tuning_params..)rXr-joinitemsr+dumpsrroZ _model_kwargsZ_kwarg_stop_sequencesZ_stop_sequencesZtuning_params_dict tuning_paramsrrr_get_tuning_paramss     zSambaverse._get_tuning_paramssdkpromptrrc Cs||j|j||}|ddkr|d}|rt|d}|d}|d}td|dd|d |d |d ntd|dd |d |d dddS)a Perform an NLP prediction using the Sambaverse endpoint handler. Args: sdk: The SVEndpointHandler to use for the prediction. prompt: The prompt to use for the prediction. tuning_params: The tuning parameters to use for the prediction. Returns: The prediction result. Raises: ValueError: If the prediction fails. r'rrcodedetailsmessage1Sambanova /complete call failed with status code . Message: Details:  Code: r.r!r"rr%)rJrWr=r-r6) rrrrrr optional_codeoptional_detailsoptional_messagerrr_handle_nlp_predicts&      zSambaverse._handle_nlp_predictrrorcCs"t|j}||}||||S)a/ Perform a prediction using the Sambaverse endpoint handler. Args: prompt: The prompt to use for the prediction. stop: stop sequences. Returns: The prediction result. Raises: ValueError: If the prediction fails. )rrVrrrrro ss_endpointrrrr_handle_completion_requests<s  z&Sambaverse._handle_completion_requestsc cs||j|j||D]}|ddkr|d}|rx|d}|d}|d}td|dd|d |d |d ntd|dd |d |d ddd} t| d} | VqdS)O Perform a streaming request to the LLM. Args: sdk: The SVEndpointHandler to use for the prediction. prompt: The prompt to use for the prediction. tuning_params: The tuning parameters to use for the prediction. Returns: An iterator of GenerationChunks. r'rrrrrrrrrrrr!r"rr#r(N)rLrWr=r- ValueErrorr6r rrrrr7rrrrr(Zgenerated_chunkrrr_handle_nlp_predict_streamPs*       z%Sambaverse._handle_nlp_predict_streamrro run_managerkwargsrc kst|j}||}z<|jrH||||D]}|r>||j|Vq*nWdSWn2tk r}ztd||W5d}~XYnXdS)aStream the Sambaverse's LLM on the given prompt. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. run_manager: Callback manager for the run. kwargs: Additional keyword arguments. directly passed to the sambaverse model in API call. Returns: An iterator of GenerationChunks. N(Error raised by the inference endpoint: ) rrVrrYron_llm_new_tokenr(r.r rrrorrrrr7r1rrr_streamxs     zSambaverse._streamcCs0d}|jf|||d|D]}||j7}q|Sa Perform a streaming request to the LLM. Args: prompt: The prompt to generate from. stop: Stop words to use when generating. Model output is cut off at the first occurrence of any of the stop substrings. run_manager: Callback manager for the run. kwargs: Additional keyword arguments. directly passed to the sambaverse model in API call. Returns: The model output as a string. r )rrorrr(rrrorrr%r7rrr_handle_stream_requests  z!Sambaverse._handle_stream_requestc Ks^z&|jr|||||WS|||WStk rX}ztd||W5d}~XYnXdS)aRun the LLM on the given input. Args: prompt: The prompt to generate from. stop: Stop words to use when generating. Model output is cut off at the first occurrence of any of the stop substrings. run_manager: Callback manager for the run. kwargs: Additional keyword arguments. directly passed to the sambaverse model in API call. Returns: The model output as a string. rN)rYrrr.rrrrorrr1rrr_calls zSambaverse._call)NN)NN)%rMrNrOrPrVr/rQrWr=rrXdictrYrTr] classmethodrar rrhpropertyrrkrmrrrrrrrr rr rrrrrrrrUsl    -   +  %    rUc @seZdZdZeedddZejedddZ eje eddfdd d Z eed d d Z deeee eeefeeeedddZdeeee eeefeeeedddZdS)SSEndpointHandlerz SambaNova Systems Interface for SambaStudio model endpoints. :param str host_url: Base URL of the DaaS API service r api_base_uricCs||_||_t|_dS)z Initialize the SSEndpointHandler. :param str host_url: Base URL of the DaaS API service :param str api_base_uri: Base URI of the DaaS API service N)rrrrr)rrrrrrrszSSEndpointHandler.__init__rc CsVi}z |}Wn.tk r>}zt||d<W5d}~XYnXd|krR|j|d<|S)rr&Nr')r+r.r/r')rrr!r1rrrr2s  z#SSEndpointHandler._process_responseNc csd|jkrz ddl}Wntk r2tdYnX||}d}|D]*}|jdkr\d}|j|j|jd}|VqJ|r|nd |jksd |jkrz4| D]&}t |}d |kr|j|d <|VqWn0t k r}zt d |W5d}~XYnXntd |jddS)r3api/predict/nlprNzTcould not import sseclient libraryPlease install it with `pip install sseclient-py`.FZ error_eventT)eventrIr'api/v2/predict/genericapi/predict/genericr'r4handling of endpoint uri:  not implemented)r sseclient ImportErrorZ SSEClienteventsrrIr'closer5r+r,r.r6r) rrrclientZ close_connrr7r0r1rrrr8sD           " z-SSEndpointHandler._process_streaming_response)pathrcCs|jd|jd|S)z Return the full API URL for a given path. :param str path: the sub-path :returns: the full API URL for the sub-path :type: str /r)rrrrrr;+szSSEndpointHandler._get_full_urlr F)projectendpointr<r>r?r@rc Cst|tr|g}d|jkr:|r0|t|d}qd|i}nxd|jkrvddt|D}|rl|t|d}qd|i}nQsz1SSEndpointHandler.nlp_predict..rr?rr) instancesr?rrrrr<rF) isinstancer/rr+r, enumeraterrrHr;r2) rrrr<r>r?r@rIrrrrrrJ5s0        zSSEndpointHandler.nlp_predict)rrr<r>r?rc csd|jkr:t|tr|g}|r0|t|d}qd|i}nd|jkrt|trT|g}ddt|D}|r||t|d}qd|i}nNd |jkrt|tr|d }|r|t|d }qd |i}ntd |jd|jj | d|d|d|i|dd}| |D] } | VqdS)rArrrrcSs g|]\}}d||dqSrrrrrrrsz8SSEndpointHandler.nlp_predict_stream..rrrrrBrCrrzstream/rr<TrKN) rrr/r+r,rlistrrrHr;r8) rrrr<r>r?rIrrr7rrrrLfs<          z$SSEndpointHandler.nlp_predict_stream)r F)r )rMrNrOrPr/rrrSrr2rr8r;rrrrTrJrrLrrrrrs6  + 7rc@seZdZUdZdZeed<dZeed<dZeed<dZ eed<dZ eed<dZ e e ed <d Ze eed <Gd d d ZeedddZeeeefdddZeedddZeeedddZe eeedddZeeeeefeedddZeeeefe eeeddd Zeeeeefee e!dd!d"Z"d*eeeefe eee e#ee e!d#d$d%Z$eeeefe eee e#eeefed#d&d'Z%d+eeeefe eee e#eed#d(d)Z&dS), SambaStudioa SambaStudio large language models. To use, you should have the environment variables ``SAMBASTUDIO_BASE_URL`` set with your SambaStudio environment URL. ``SAMBASTUDIO_BASE_URI`` set with your SambaStudio api base URI. ``SAMBASTUDIO_PROJECT_ID`` set with your SambaStudio project ID. ``SAMBASTUDIO_ENDPOINT_ID`` set with your SambaStudio endpoint ID. ``SAMBASTUDIO_API_KEY`` set with your SambaStudio endpoint API key. https://sambanova.ai/products/enterprise-ai-platform-sambanova-suite read extra documentation in https://docs.sambanova.ai/sambastudio/latest/index.html Example: .. code-block:: python from langchain_community.llms.sambanova import SambaStudio SambaStudio( sambastudio_base_url="your-SambaStudio-environment-URL", sambastudio_base_uri="your-SambaStudio-base-URI", sambastudio_project_id="your-SambaStudio-project-ID", sambastudio_endpoint_id="your-SambaStudio-endpoint-ID", sambastudio_api_key="your-SambaStudio-endpoint-API-key, streaming=False model_kwargs={ "do_sample": False, "max_tokens_to_generate": 1000, "temperature": 0.7, "top_p": 1.0, "repetition_penalty": 1, "top_k": 50, #"process_prompt": False, #"select_expert": "Meta-Llama-3-8B-Instruct" }, ) r sambastudio_base_urlsambastudio_base_urisambastudio_project_idsambastudio_endpoint_idsambastudio_api_keyNrXFrYc@seZdZdZdS)zSambaStudio.ConfigrZNr[rrrrr]sr]r9cCsdSr^rr_rrrraszSambaStudio.is_lc_serializablecCs d|jiSrirjr:rrrrkszSambaStudio._identifying_paramscCsdS)rlzSambastudio LLMrr:rrrrmszSambaStudio._llm_typerbcCsXt|dd|d<t|dddd|d<t|dd|d<t|d d |d <t|d d |d <|S) z?Validate that api key and python package exists in environment.rZSAMBASTUDIO_BASE_URLrZSAMBASTUDIO_BASE_URIrrdrZSAMBASTUDIO_PROJECT_IDrZSAMBASTUDIO_ENDPOINT_IDrZSAMBASTUDIO_API_KEYrfrgrrrrhs6 z SambaStudio.validate_environmentrncCsN|jpi}|dg}|p|}d|jkr.|}ndd|D}t|}|S)rprqrcSs&i|]\}}|t|jt|dqSrwrzr{rrrr~sz2SambaStudio._get_tuning_params..)rXr-rrr+rrrrrrs    zSambaStudio._get_tuning_paramsrcCs||j|j|j||}|ddkrd|d}|rLtd|dd|ntd|dd|d|jkr~|dd d Sd |jkr|d d d d Sd|jkr|dd d Std|jddS)a Perform an NLP prediction using the SambaStudio endpoint handler. Args: sdk: The SSEndpointHandler to use for the prediction. prompt: The prompt to use for the prediction. tuning_params: The tuning parameters to use for the prediction. Returns: The prediction result. Raises: ValueError: If the prediction fails. r'rr&rz . Details: z . response rrIrr%rrryrZ predictionsrrN)rJrrrr-r6rr)rrrrrZoptional_detailrrrrs2      zSambaStudio._handle_nlp_predictrcCs&t|j|j}||}||||S)a0 Perform a prediction using the SambaStudio endpoint handler. Args: prompt: The prompt to use for the prediction. stop: stop sequences. Returns: The prediction result. Raises: ValueError: If the prediction fails. )rrrrrrrrrrNs  z'SambaStudio._handle_completion_requestsc cs@||j|j|j||D] }|ddkr|d}|r~|d}|d}|d}td|dd|d |d |d ntd|dd |d d |jkrt |dd} ntd|jkr|ddddd} nPd|jkrt |dddkr|dddd} nd} ntd|jdt | d} | VqdS)rr'rrrrrrrrrrrrrIr#rr!rrryrr"r rznot implementedrN) rLrrrr-rr6rr+r,lenr rrrrrds@            z&SambaStudio._handle_nlp_predict_streamrc kst|j|j}||}z<|jrL||||D]}|rB||j|Vq.nWdSWn2tk r}zt d||W5d}~XYnXdS)Call out to Sambanova's complete endpoint. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. Returns: The string generated by the model. Nr) rrrrrYrrr(r.rrrrrrs$    zSambaStudio._streamcCs0d}|jf|||d|D]}||j7}q|Srrrrrrrs  z"SambaStudio._handle_stream_requestc Ksn|dk rtdz&|jr*|||||WS|||WStk rh}ztd||W5d}~XYnXdS)rNzstop not implementedr)r.rYrrrrrrrrszSambaStudio._call)NN)NN)'rMrNrOrPrr/rQrrrrrXrrrYrTr]rrarrrrkrmr rhrrrrrrrr rr rrrrrrrrsp &      0   <  $    r)r+typingrrrrrrrrZ langchain_core.callbacks.managerr Z#langchain_core.language_models.llmsr Zlangchain_core.outputsr Zlangchain_core.utilsr r rrUrrrrrrs$   2K