iPl~dZddlZddlZddlmZddlmZmZmZm Z m Z  ddl m Z e e neZddlmZmZmZmZmZddlmZdd lmZdd lmZdd lmZdd lmZmZdd l m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(ddl)m*Z*ddl+m,Z,ddl-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3e*e4Z5d2de6de7de6fdZ8de6de6de6fdZ9GddZ:GddZ;GddZ<GddZ=Gd d!Z>d"e6de e e6e6e6ffd#Z?d$e6de@fd%ZAd&ZBd'eedeefd(ZCGd)d*ZDGd+d,ZEGd-d.ZFd/ede6fd0ZGd/ede6fd1ZHy#e $rdZ YwxYw)3z~ Deep Analysis Agent nodes and routers. Class-based nodes; executor logic split into QueryBatchExecutor for easier debugging. N)ThreadPoolExecutor)ListDictAnyTupleOptional)ContextThreadPoolExecutor) BaseMessage HumanMessage AIMessage ToolMessage SystemMessage)DeepAnalysisState)REASONER_QUERIES_MAX)ReasonerQueriesBatch)execute_sql_query)serialize_sql_resultnormalize_markdown_output)INVESTIGATION_PROMPTREPORTING_PROMPTREASONER_SYSTEM_PROMPTREASONER_PROMPTQUERY_FIXER_PROMPTFINALIZER_PROMPT FINDING_FROM_QUERY_RESULT_PROMPT"DECIDE_CONTINUE_OR_FINALIZE_PROMPT) get_logger)get_llm)cap_result_contentMAX_CHARS_PER_RESULT count_tokens TARGET_TOKENSinvoke_with_retryrewrite_query_reduced_windowprompt temperaturereturnct|jt|g}|r t|ddnd}|xsdj S)zGSingle HumanMessage prompt; returns content string. Never returns None.r'contentr,N)rinvoker getattrstrip)r&r'responser,s B/var/www/html/userprofiledev.eatanceapp.com/deep_analysis/nodes.py _invoke_llmr34sD;/66 V8T7UVH4t|dtr+|dj d r|dd }d|xsdd|d |St|tr#|j d rd|xsdd|d|d S|r t|nd }t|td }d|xsdd|d|S#t$r}d|xsdd|d|cYd}~Sd}~wwxYw)z Runs the given SQL query and returns full ToolMessage content string (hypothesis + query + answer). Used when retrying after context overflow with a reduced-date-window query. r6 hypothesis: r- query:  answer: SYSTEM_ERROR: N _sys_errorrerror answer: SQL_ERROR: No rows.zretry reduced window max_chars query_label answer: ) rr. Exception isinstancedictgetlistrr r!)r5r6resulteerr raw_answer answer_texts r2!_run_query_and_build_tool_contentrM>sg \"))7E*:;&$FJJ|$<j.B/y?WX^_kXlWmnn&$Fz&)T/JvVWy}}]dOeQi j.B/y?TUXTYZZ&$FJJw$7j.B/y?WX^_fXgWhii17%f-ZJ$Z;O]stK **+9UG:k] SS \j.B/y?WXYWZ[[\sC?? D!DD!D!ceZdZdZdedeeeeffdZdeeeefdee fdZ de defdZ d ed ede de ee ffd Z d e eeee fde eee eeeffd Zd ed ede dedede eeff dZdedeee ffdZy)QueryBatchExecutorz Runs a batch of SQL queries in parallel and builds tool messages + findings. Each step is a separate method for easier debugging and testing. stater(c|jdxsg}|jdW|jdF|d}t|}d|cxkrt|kr"n|S||jdd|dd||<|S)z]Resolve batch from state; apply fixed_query when set (reserved for a future QueryFixer loop).last_queries_batch fixed_queryfailed_query_indexrr5r-r5r6)rFrGlen)selfrPbatchidxs r2 resolve_batchz QueryBatchExecutor.resolve_batch^s ./52 99] # /EII>R4S4_,-CKEC$#e*$  #(*..r"B"=1c  r4rXc|Dcgc]_}|jdxsdjs'|jdxsdj|jdxsdda}}dtdtfd}t t t |t5}t|j||cd d d Scc}w#1swYy xYw) zRRun all queries in parallel; returns list of results in same order as valid items.r6r-r5)r6r5payloadr(c~ tjd|diS#t$r}dt|icYd}~Sd}~wwxYw)Nr6r;)rr.rCstr)r\rIs r2run_onez8QueryBatchExecutor.run_queries_parallel..run_oneosA .(//'':J0KLL .$c!f-- .s < 7<< max_workersN) rFr0rEr _ExecutorminrVrrGmap)rWrXitemvalidr_exs r2run_queries_parallelz'QueryBatchExecutor.run_queries_parallelksx}dx}ptCGCKCKLSCTCZXZBaBaBcDHHW-3:: !ufdVZZ-@-FBH H"&"C"CE:W]"^ K:{It<z No data rows.zquery r of r?r8r-r9rBr, tool_call_idzQuery N2P  [z] ...)rVrr r!r replacer0)rWr6r5rHr|r}ris_emptyrKrLr,msghypansfindings r2build_success_messagez(QueryBatchExecutor.build_success_messages; w' :1V!15;)&1 -3o9K *  4 {;: !!1r 2)E7*[MZ w<.4 4(z(QueryBatchExecutor.run..sQqTr4)keyrz-----------------zHYPOTHESIS : %szLLM query : %sz-----------------------rjz"Executor system failure for query z: zSYSTEM_ERROR: r8r9r:rz[Query z failed] dTrkzGSQL Error (query %s): %s | HYPOTHESIS: %s | QUERY (first 200 chars): %sz(none)rz SQL_ERROR: z | QUERY: r=rl)messagesrrRrTrSrzAll zU queries in this turn returned no data; will ask LLM whether to continue or finalize.rg?) rfindings last_queryrRrrTrS retry_countrcontext_near_limitbatch_all_empty)rZrFrprrrV enumerater0rhrangerbrcrrGrdr{sortr<appendr rallr"r#)rWrPrX turn_displayrrwre valid_items results_listjitems_for_processrg processed tool_messagesfindings_this_turn results_emptylast_error_outfailed_query_index_outryrzrx error_msgr5rrrr new_messages total_tokensrs r2runzQueryBatchExecutor.runs1""5) "HI I./!3 IIk1-  0U OL>Y]^g]hhijk%U+ +4!'R..0'"(b//1DHH\4J4Pb R+   "HI I007 3{+, ,^A  Aq 1;q>!3DlSTo V,  3s+<'=?S#T UY[RVVD$<$<>OPQIV>*+- (*$& (,04=F 9Q Ky8D((<06BJ#A3*.*.#'',->'?!'C  55J QE LC   %  % %g .  [!IC 4D4I J>GBk*Q.p3}3E[IY3Yp^abo^p  KK$s5zl*A B$ INS]I^|E*$5 $DEde )MC,?? %*"&"&$%67!;"4.  E  V Us(A&Q'>"Q-&Q22Q<N)__name__ __module__ __qualname____doc__rrrr^rZrrhrnrruintrr{r rrrr4r2rOrOXs- #4 d38n9M  0$tCH~*> 049 0cc'#'3'PS'X]^acf^fXg'8=#sC,-= sCc8C=0 1=       {C 6w *w tCH~w r4rOc6eZdZdZdefdZdedeeeffdZ y) ReasonerNodeu~Generates 3–5 investigative SQL queries (single turn) using structured LLM output. Does not run queries; Executor does that.analytical_schemac||_yNrrWrs r2__init__zReasonerNode.__init__H !2r4rPr(c |tjdtd|dtj|d|j |dt }tjdrdnd}td| jt}t| ttj|d  g}t||}|j Dcgc]}|j"|j$d }}tjd t'|dt)dt'|d gd|ddddddScc}w)NuReasoner generating 3–z queries for User user_id profile_json)rrrinvestigation_promptOPENAI_API_KEYopenaig?)r'providerr+)rrUzReasoner produced z queries.z Generated z investigative queries.rF)rrrRrrTrSrr)rprrrrformatrrosgetenvrwith_structured_outputrrr rr$queriesr5r6rVr ) rWrP system_promptreasoner_providermodelrrXq queries_lists r2__call__zReasonerNode.__call__Ks> ./C.DDVW\]fWgVhij.55)$"44~.!5  )+ 2B(CHC2CD[[\pq - 0 !7!7i@P!Q R '8x&HPUP]P]^P]1q||aggFP] ^ (\):(;9EF#ZL8I7JJa+bcd"."&"'  _s D9N rrrrr^rrrrrrr4r2rrEs0I3#3 / DcN r4rc0eZdZdZdZdedeeeffdZ y) ExecutorNodeufRuns the batch of SQL queries (from Reasoner, 3–5) in parallel and appends tool messages + findings.c"t|_yr)rO_runner)rWs r2rzExecutorNode.__init__ns )+ r4rPr(c8|jj|Sr)rr)rWrPs r2rzExecutorNode.__call__qs||&&r4N) rrrrrrrr^rrrr4r2rrks&p,'/'DcN'r4rc6eZdZdZdefdZdedeeeffdZ y)QueryFixerNodez'Repairs one failed SQL query using LLM.rc||_yrrrs r2rzQueryFixerNode.__init__xrr4rPr(c|d}|jdd}tjd|dtj||j }t |d}|j}tjd|tj }|r|jd j}||d d zdd S) NrrTrz2Query Fixer attempting SQL repair for query index r) error_logrr*z```(?:sql)?\s*([\s\S]*?)```flagsrr)rSrr) rFrprrrrrr3r0research IGNORECASEgroup)rWrPrrYr&r,rS fence_matchs r2rzQueryFixerNode.__call__{s,' ii,a0 HSQR#**YRVRhRhif!4mmo ii > SUS`S`a %++A.446K' /!3  r4Nrrr4r2rrus-13#3 / DcN r4rc*eZdZdZdedeeeffdZy) FinalizerNodez=Produces the final behavioral report from profile + findings.rPr(ctjdtjd|d|jdxsg}|jdxsg}|Dcgc]}t|tr d|vrd|vr|}}|r+dj |}|r)|d d j |zz }ndj |}|sd }t j|d |t }t|d}t|}t|tr|xsdjnt |xsd}|rd|jvsd|jvrtjdtjdd|tj tj"z}tjdd|tj tj"z}d|iScc}w)NzEStarting finalize node (synthesizing report from profile + findings).z*Finalizing Behavioral Analysis for User %sr llm_findingsrz[Queryzfailed]z zC --- Queries that failed before analysis (no LLM interpretation): rzThe investigation concludes that the initial profile is consistent with raw event logs, with no major supplemental anomalies found.r)rcombined_findingsreporting_prompt皙?r*r-z```sqlzselect zESQL leakage detected in Finalizer output. Stripping technical blocks.z ```sql.*?```rzSELECT\s+.*?\s+FROM\s+.*?;zechnical Detail Redacted]final_analysis)rprrrFrDr^joinrrrr3rr0lowerrqrsubDOTALLr) rWrPrexecutor_findingsf failure_notesrr&r,s r2rzFinalizerNode.__call__s [\ @% BRSyy06B !IIj17R' &!S!h!m Q &   & L 9 !]ii ./! !' ,= >  !f !((~./-  f#6+G4-7-E7=b'')3w}Z\K] GMMO3yGMMO7S NNb cff_b'R]]AZ[Gff:>#U1X^^%5&D DS\c&kC&7UR @FT!1Ha(+33M2FLLNHa(+113''4Ha(+113FF## f r4answerc|xsdj}|j}|jdxs|jdS)uWTrue when the executor/tool layer failed — do not call the findings LLM for this row.r-z SQL_ERROR:z SYSTEM_ERROR:)r0upper startswith)raus r2_answer_should_skip_finding_llmrs< 2A  A << % Fo)FFr4zinvestigative queries.rc>d}t|D]7\}}t|tst|ddxsd}t|vs1d|vs6|}9|dkr!|Dcgc]}t|t s|c}S||dzdDcgc]}t|t s|c}Scc}wcc}w)z When the graph loops (e.g. reasoner_abort -> reasoner -> executor), ``messages`` accumulates ToolMessages from earlier turns. Only analyze tools after the most recent Reasoner batch marker. r,r- GeneratedrrN)rrDr r/_REASONER_BATCH_AIMSG_MARKERr )rlast_reasoner_idxrwmcs r2)_tool_messages_from_latest_reasoner_batchrs (#1 a #9b)/RA+q0[A5E$%! $ 1#B8az!['A8BB 1A 5 78 W8!Jq+ one finding. Skips SQL/system errors (syntax, invalid object, etc.) — no LLM spend on those rows.rPr(c 6|jdxsg}t|}g}d}|D]G}tt|ddxsd}|s |\}} } t | r|dz }7|j |I|rt jd||st jddgiSt|t jd d ttttfd tfd d ttttttffd tffd } ttt5} t!| j#| t!t%|} dddt jdt d| iS#1swY-xYw)Nrrr,r-rzSFindingsNode: skipping %s tool message(s) with SQL/SYSTEM errors (no findings LLM).z^FindingsNode: no successful (hypothesis, query, result) triples; returning empty llm_findings.rzUFindingsNode: starting %s parallel LLM analyses (hypothesis+query+result -> finding).argsr(c|\}}}tj|||} t|dS#t$r4}tj d|dt |dddcYd}~Sd}~wwxYw)N)r5r6rHrr*zFindingsNode LLM call failed: z[Finding unavailable: r])rrr3rCrprqr^)rrrresr&rIs r2 one_findingz*FindingsNode.__call__..one_findingsxKCC5<<F  @"6s;; @!?sCD/At ~Q?? @s - A*)A%A*%A*recR|\}}tjd|dz|S)NzAnalyzing finding %s/%s...r)rprr)rerYtriplenrs r2 analyze_onez*FindingsNode.__call__..analyze_ones,KC KK4cAgq Av& &r4r`z=FindingsNode: all %s findings complete; produced %s findings.)rFrrr/rrrprrrqrVrr^rrbrcrrGrdr)rWrPrrtriplesskipped_failuresr parsed_hyp_qrrrg findings_listrrs @@r2rzFindingsNode.__call__s~99Z(.BA(K .0A4WQ 25N5TRTUF"MD"c.s3 A%  NN6 "  KKe   NN{ |"B' ' L kmno @eCcM2 @s @ 'eCsC})=$=> '3 ' 3q*>#? @B  T)G:L5M!NOMA SUVX[\iXjk ..A @s 5.FFNrrr4r2rrs$^/////DcN//r4rc*eZdZdZdedeeeffdZy) DecideNodez;After each batch, asks LLM whether to continue or finalize.rPr(c|d}|jdxsddd}|jdxsg}|rdj|nd}|ddt|dkDrd ndz}td |jd d |d z }|jddu}|rdnd}t j |||||} t | d } | xsdjr3| xsdjjjd nd} | dk(rdnd} tjd| d|dd| dS#t$r%} tjd| dd} Yd} ~ Jd} ~ wwxYw)Nrrr-i rrzNo findings yet.irrrrrrTz+ This batch returned no data for any query.)rprofile_snippet findings_textturns_remaining empty_noter*FINALIZEfinalizecontinuezDecide node LLM failed: z; defaulting to finalize.zAfter-batch decision: z (turns_remaining=rF)rempty_turn_decision)rFrrVmaxrrr3r0rrrCrprqrr)rWrPrr% raw_findingsr&r'rr(r&r, first_worddecisionrIs r2rzDecideNode.__call__ s " 99^4:DSAyy,2 3? ,/EW %et,]9Kd9RXZ[ a;!:UCT=U!UV))$56$>FUB[] 3::+'+!   "!&a8GHO SUG\G\G^'-R..0668>>@CdfJ%/:%=z:H  ,XJ6HHYYZ[\#(JJ  " NN5aS8QR S!H "s6AD66 E$?EE$Nrrr4r2r#r#s$EK/KDcNKr4r#c*eZdZdZdedeeeffdZy)ReasonerAbortNodezMCleans up state after investigation abort (e.g. system error or retry limit).rPr(c Z|dr|ddnd}d}|r*t|dr|jr|jdd}d|d}t|| }|jd xs+|jd xrd |jd xsd}|g|d|dgdddddd|ddzd S)Nrr abort tool_callsridzINVESTIGATION ABORTED: rrrrRzBatch query index rTQueryz | Error: rr) rfailed_investigationsrrRrrTrSrr)hasattrr5r rF)rWrP last_ai_msgr error_summarytool_msg failed_descs r2rzReasonerAbortNode.__call__@s/4Z/@eJ'+d  7; =+BXBX&11!4T:L1% 2E1FG }<Pii -Y%))rUs  133%9 *C)N %Tf dd$+($D   *% H  ##%##T#TcTcT4h h Z# # L''  2(+(+VSXeCcM>R5S&GCGDG 8 X[8I XdS^N_ X 3/3/lKK@  4/C*(9c% $%sD11D<;D<