Eval results missing details and agent output

[bai-uipath]

When running uipath eval, the results JSON only contains a score, no justification or agent output:

  {
      "evaluationName": "From Agent run - 3.2.AnalysisAgent",
      "evaluationRunResults": [
          {
              "evaluatorName": "Default Evaluator",
              "evaluatorId": "9824233e-ece6-4bdf-8ad8-2297644ec272",
              "result": {
                  "score": 0.45,
                  "details": {},
                  "evaluationTime": 2.595767021179199
              }
          }
      ],
      "agentExecutionOutput": null
  },

[cristipufu]

Investigation Summary

Both issues have been identified as pre-existing bugs, not regressions.

Bug 1: "details": {}

Root cause: Pydantic v2 serialization of a field declared as BaseModel loses subclass-specific fields.

EvaluationResultDto.details is typed as str | BaseModel | None. When an LLM-as-judge evaluator runs, it sets details to a LLMJudgeJustification instance (a BaseModel subclass with expected, actual, and justification fields). However, Pydantic v2 generates the serialization schema from the declared type (BaseModel), not the runtime type. Since BaseModel has no user-defined fields, it serializes to {}.

Reproduction:

from pydantic import BaseModel

class Justification(BaseModel):
    expected: str
    actual: str
    justification: str

class Result(BaseModel):
    score: float
    details: str | BaseModel | None = None

j = Justification(expected="foo", actual="bar", justification="because reasons")
r = Result(score=0.45, details=j)
print(r.model_dump())  # {'score': 0.45, 'details': {}}

Fix: Change the details field type in EvaluationResultDto to str | dict | None and convert BaseModel instances to dicts before assignment, or use SerializeAsAny[BaseModel] from Pydantic to preserve subclass fields during serialization.

Bug 2: "agentExecutionOutput": null

Root cause: agent_execution_output is gated behind self.context.verbose, which defaults to False and is never set to True from the CLI.

if self.context.verbose:
    evaluation_run_results.agent_execution_output = (
        convert_eval_execution_output_to_serializable(agent_execution_output)
    )

UiPathEvalContext.verbose defaults to False, and no CLI flag wires to it. The --verbose CLI flag that existed at one point only configured Python logging levels, not the eval context.

Fix: Either remove the verbose guard so agent_execution_output is always populated, or wire a CLI --verbose flag to eval_context.verbose.