data [required]
\nobject
Data object for creating or updating a custom LLM Observability evaluator configuration.
attributes [required]
\nobject
Attributes for creating or updating a custom LLM Observability evaluator configuration.
category
\nstring
Category of the evaluator.
eval_name
\nstring
Name of the custom evaluator. If provided, must match the eval_name path parameter.
llm_judge_config
\nobject
LLM judge configuration for a custom evaluator.
assessment_criteria
\nobject
Criteria used to assess the pass/fail result of a custom evaluator.
max_threshold
\ndouble
Maximum numeric threshold for a passing result.
min_threshold
\ndouble
Minimum numeric threshold for a passing result.
pass_values
\n[string]
Specific output values considered as a passing result.
pass_when
\nboolean
When true, a boolean output of true is treated as passing.
context_query
\nstring
Query used to extract additional context for the evaluation.
inference_params [required]
\nobject
LLM inference parameters for a custom evaluator.
frequency_penalty
\ndouble
Frequency penalty to reduce repetition.
max_tokens
\nint64
Maximum number of tokens to generate.
presence_penalty
\ndouble
Presence penalty to reduce repetition.
temperature
\ndouble
Sampling temperature for the LLM.
top_k
\nint64
Top-k sampling parameter.
top_p
\ndouble
Top-p (nucleus) sampling parameter.
last_used_library_prompt_template_name
\nstring
Name of the last library prompt template used.
modified_library_prompt_template
\nboolean
Whether the library prompt template was modified.
output_schema
\nobject
JSON schema describing the expected output format of the LLM judge.
parsing_type
\nenum
Output parsing type for a custom LLM judge evaluator. \nAllowed enum values: structured_output,json,keyword_search
prompt_template
\n[object]
List of messages forming the LLM judge prompt template.
content
\nstring
Text content of the message.
contents
\n[object]
Multi-part content blocks for the message.
type [required]
\nstring
Content block type.
value [required]
\nobject
Value of a prompt message content block.
text
\nstring
Text content of the message block.
tool_call
\nobject
A tool call within a prompt message.
arguments
\nstring
JSON-encoded arguments for the tool call.
id
\nstring
Unique identifier of the tool call.
name
\nstring
Name of the tool being called.
type
\nstring
Type of the tool call.
tool_call_result
\nobject
A tool call result within a prompt message.
name
\nstring
Name of the tool that produced this result.
result
\nstring
The result returned by the tool.
tool_id
\nstring
Identifier of the tool call this result corresponds to.
type
\nstring
Type of the tool result.
role [required]
\nstring
Role of the message author.
target_query
\nstring
Query used to extract the target value to evaluate.
user_specified_json_post_processing_function
\nstring
User-provided function applied to post-process the JSON output of the LLM judge.
llm_provider
\nobject
LLM provider configuration for a custom evaluator.
bedrock
\nobject
AWS Bedrock-specific options for LLM provider configuration.
inference_profile
\nstring
Bedrock inference profile identifier, such as an application inference profile ARN.
region
\nstring
AWS region for Bedrock.
integration_account_id
\nstring
Integration account identifier.
integration_provider
\nenum
Name of the LLM integration provider. \nAllowed enum values: openai,amazon-bedrock,anthropic,azure-openai,vertex-ai,llm-proxy
model_name
\nstring
Name of the LLM model.
vertex_ai
\nobject
Google Vertex AI-specific options for LLM provider configuration.
location
\nstring
Google Cloud region.
project
\nstring
Google Cloud project ID.
target [required]
\nobject
Target application configuration for a custom evaluator.
application_name [required]
\nstring
Name of the ML application this evaluator targets.
enabled [required]
\nboolean
Whether the evaluator is active for the target application.
eval_scope
\nenum
Scope at which to evaluate spans. \nAllowed enum values: span,trace,session
experiment_project_ids
\n[string]
Experiment project IDs this evaluator is scoped to.
filter
\nstring
Filter expression to select which spans to evaluate.
root_spans_only
\nboolean
When true, only root spans are evaluated.
sampling_percentage
\ndouble
Percentage of traces to evaluate. Must be greater than 0 and at most 100.
id
\nstring
Name of the evaluator. If provided, must match the eval_name path parameter.
type [required]
\nenum
Type of the custom LLM Observability evaluator configuration resource. \nAllowed enum values: evaluator_config
data [required]
\nobject
Data object for an LLM Observability data deletion response.
attributes [required]
\nobject
Attributes of a submitted LLM Observability data deletion request.
created_at [required]
\ndate-time
Timestamp when the deletion request was created.
created_by [required]
\nstring
UUID of the user who created the deletion request.
from_time [required]
\nint64
Start of the deletion time range in milliseconds since Unix epoch.
org_id [required]
\nint64
ID of the organization that submitted the deletion request.
product [required]
\nstring
Product name for the deletion request.
query [required]
\nstring
The query string used to select data for deletion.
to_time [required]
\nint64
End of the deletion time range in milliseconds since Unix epoch.
id [required]
\nstring
Unique identifier of the deletion request.
type [required]
\nenum
Resource type for an LLM Observability data deletion response. \nAllowed enum values: deletion_request
errors [required]
\n[object]
A list of errors.
detail
\nstring
A human-readable explanation specific to this occurrence of the error.
meta
\nobject
Non-standard meta-information about the error
source
\nobject
References to the source of the error.
header
\nstring
A string indicating the name of a single request header which caused the error.
parameter
\nstring
A string indicating which URI query parameter caused the error.
pointer
\nstring
A JSON pointer to the value in the request document that caused the error.
status
\nstring
Status code of the response.
title
\nstring
Short human-readable summary of the error.
errors [required]
\n[object]
A list of errors.
detail
\nstring
A human-readable explanation specific to this occurrence of the error.
meta
\nobject
Non-standard meta-information about the error
source
\nobject
References to the source of the error.
header
\nstring
A string indicating the name of a single request header which caused the error.
parameter
\nstring
A string indicating which URI query parameter caused the error.
pointer
\nstring
A JSON pointer to the value in the request document that caused the error.
status
\nstring
Status code of the response.
title
\nstring
Short human-readable summary of the error.
errors [required]
\n[object]
A list of errors.
detail
\nstring
A human-readable explanation specific to this occurrence of the error.
meta
\nobject
Non-standard meta-information about the error
source
\nobject
References to the source of the error.
header
\nstring
A string indicating the name of a single request header which caused the error.
parameter
\nstring
A string indicating which URI query parameter caused the error.
pointer
\nstring
A JSON pointer to the value in the request document that caused the error.
status
\nstring
Status code of the response.
title
\nstring
Short human-readable summary of the error.
errors [required]
\n[string]
A list of errors.
data [required]
\nobject
Data object for an LLM Observability data deletion request.
attributes [required]
\nobject
Attributes for an LLM Observability data deletion request.
delay
\nint64
Optional delay in seconds before the deletion is executed.
from [required]
\nint64
Start of the deletion time range in milliseconds since Unix epoch.
query [required]
\nobject
Query filters selecting the data to delete. Must include a query key with an @trace_id filter.
<any-key>
\nstring
to [required]
\nint64
End of the deletion time range in milliseconds since Unix epoch.
type [required]
\nenum
Resource type for an LLM Observability data deletion request. \nAllowed enum values: create_deletion_req