close()
Close httplib2 connections.
+list(userKey, applicationName, actorIpAddress=None, applicationInfoFilter=None, customerId=None, endTime=None, eventName=None, filters=None, groupIdFilter=None, includeSensitiveData=None, maxResults=None, networkInfoFilter=None, orgUnitID=None, pageToken=None, resourceDetailsFilter=None, startTime=None, statusFilter=None, x__xgafv=None)
Retrieves a list of activities for a specific customer's account and application such as the Admin console application or the Google Drive application. For more information, see the guides for administrator and Google Drive activity reports. For more information about the activity report's parameters, see the activity parameters reference guides.
@@ -93,7 +93,7 @@list(userKey, applicationName, actorIpAddress=None, applicationInfoFilter=None, customerId=None, endTime=None, eventName=None, filters=None, groupIdFilter=None, maxResults=None, networkInfoFilter=None, orgUnitID=None, pageToken=None, resourceDetailsFilter=None, startTime=None, statusFilter=None, x__xgafv=None)
+ list(userKey, applicationName, actorIpAddress=None, applicationInfoFilter=None, customerId=None, endTime=None, eventName=None, filters=None, groupIdFilter=None, includeSensitiveData=None, maxResults=None, networkInfoFilter=None, orgUnitID=None, pageToken=None, resourceDetailsFilter=None, startTime=None, statusFilter=None, x__xgafv=None)
Retrieves a list of activities for a specific customer's account and application such as the Admin console application or the Google Drive application. For more information, see the guides for administrator and Google Drive activity reports. For more information about the activity report's parameters, see the activity parameters reference guides. Args: @@ -145,6 +145,7 @@Method Details
eventName: string, The name of the event being queried by the API. Each `eventName` is related to a specific Google Workspace service or feature which the API organizes into types of events. An example is the Google Calendar events in the Admin console application's reports. The Calendar Settings `type` structure has all of the Calendar `eventName` activities reported by the API. When an administrator changes a Calendar setting, the API reports this activity in the Calendar Settings `type` and `eventName` parameters. For more information about `eventName` query strings and parameters, see the list of event names for various applications above in `applicationName`. filters: string, The `filters` query string is a comma-separated list composed of event parameters manipulated by relational operators. Event parameters are in the form `{parameter1 name}{relational operator}{parameter1 value},{parameter2 name}{relational operator}{parameter2 value},...` These event parameters are associated with a specific `eventName`. An empty report is returned if the request's parameter doesn't belong to the `eventName`. For more information about the available `eventName` fields for each application and their associated parameters, go to the [ApplicationName](#applicationname) table, then click through to the Activity Events page in the Appendix for the application you want. In the following Drive activity examples, the returned list consists of all `edit` events where the `doc_id` parameter value matches the conditions defined by the relational operator. In the first example, the request returns all edited documents with a `doc_id` value equal to `12345`. In the second example, the report returns any edited documents where the `doc_id` value is not equal to `98765`. The `<>` operator is URL-encoded in the request's query string (`%3C%3E`): ``` GET...&eventName=edit&filters=doc_id==12345 GET...&eventName=edit&filters=doc_id%3C%3E98765 ``` A `filters` query supports these relational operators: * `==`—'equal to'. * `<>`—'not equal to'. Must be URL-encoded (%3C%3E). * `<`—'less than'. Must be URL-encoded (%3C). * `<=`—'less than or equal to'. Must be URL-encoded (%3C=). * `>`—'greater than'. Must be URL-encoded (%3E). * `>=`—'greater than or equal to'. Must be URL-encoded (%3E=). **Note:** The API doesn't accept multiple values of the same parameter. If a parameter is supplied more than once in the API request, the API only accepts the last value of that parameter. In addition, if an invalid parameter is supplied in the API request, the API ignores that parameter and returns the response corresponding to the remaining valid parameters. If no parameters are requested, all parameters are returned. groupIdFilter: string, Comma separated group ids (obfuscated) on which user activities are filtered, i.e. the response will contain activities for only those users that are a part of at least one of the group ids mentioned here. Format: "id:abc123,id:xyz456" *Important:* To filter by groups, you must explicitly add the groups to your filtering groups allowlist. For more information about adding groups to filtering groups allowlist, see [Filter results by Google Group](https://support.google.com/a/answer/11482175) + includeSensitiveData: boolean, Optional. When set to `true`, this field allows sensitive user-generated content to be included in the returned audit logs. This parameter is supported only for Rules (DLP) and Chat applications; using it with any other application will result in a permission error. maxResults: integer, Determines how many activity records are shown on each response page. For example, if the request sets `maxResults=1` and the report has two activities, the report has two pages. The response's `nextPageToken` property has the token to the second page. The `maxResults` query string is optional in the request. The default value is 1000. networkInfoFilter: string, Optional. Used to filter on the `regionCode` field present in [`NetworkInfo`](#networkinfo) message. **Usage** ``` GET...&networkInfoFilter=regionCode="IN" GET...&networkInfoFilter=regionCode=%22IN%22 ``` orgUnitID: string, ID of the organizational unit to report on. Activity records will be shown only for users who belong to the specified organizational unit. Data before Dec 17, 2018 doesn't appear in the filtered results. @@ -236,6 +237,60 @@Method Details
"resourceIds": [ # Resource ids associated with the event. "A String", ], + "sensitiveParameters": [ # Includes sensitive parameter value pairs for various applications. + { + "boolValue": True or False, # Boolean value of the parameter. + "intValue": "A String", # Integer value of the parameter. + "messageValue": { # Nested parameter value pairs associated with this parameter. Complex value type for a parameter are returned as a list of parameter values. For example, the address parameter may have a value as `[{parameter: [{name: city, value: abc}]}]` + "parameter": [ # Parameter values + { # JSON template for a parameter used in various reports. + "boolValue": True or False, # Boolean value of the parameter. + "intValue": "A String", # Integer value of the parameter. + "multiBoolValue": [ # Multiple boolean values of the parameter. + True or False, + ], + "multiIntValue": [ # Multiple integer values of the parameter. + "A String", + ], + "multiValue": [ # Multiple string values of the parameter. + "A String", + ], + "name": "A String", # The name of the parameter. + "value": "A String", # String value of the parameter. + }, + ], + }, + "multiIntValue": [ # Integer values of the parameter. + "A String", + ], + "multiMessageValue": [ # List of `messageValue` objects. + { + "parameter": [ # Parameter values + { # JSON template for a parameter used in various reports. + "boolValue": True or False, # Boolean value of the parameter. + "intValue": "A String", # Integer value of the parameter. + "multiBoolValue": [ # Multiple boolean values of the parameter. + True or False, + ], + "multiIntValue": [ # Multiple integer values of the parameter. + "A String", + ], + "multiValue": [ # Multiple string values of the parameter. + "A String", + ], + "name": "A String", # The name of the parameter. + "value": "A String", # String value of the parameter. + }, + ], + }, + ], + "multiValue": [ # String values of the parameter. + "A String", + ], + "name": "A String", # The name of the parameter. + "value": "A String", # String value of the parameter. + }, + ], "status": { # Status of the event. Note: Not all events have status. # Status of the event. Note: Not all events have status. "errorCode": "A String", # Error code of the event. Note: Field can be empty. "errorMessage": "A String", # Error message of the event. Note: Field can be empty. diff --git a/docs/dyn/agentregistry_v1alpha.html b/docs/dyn/agentregistry_v1alpha.html new file mode 100644 index 0000000000..c0fc2128cc --- /dev/null +++ b/docs/dyn/agentregistry_v1alpha.html @@ -0,0 +1,111 @@ + + + +Agent Registry API
+Instance Methods
++
+projects()+Returns the projects Resource.
+ ++
+close()Close httplib2 connections.
+ +Create a BatchHttpRequest object based on the discovery document.
+Method Details
+++ +close()+Close httplib2 connections.+++ + \ No newline at end of file diff --git a/docs/dyn/agentregistry_v1alpha.projects.html b/docs/dyn/agentregistry_v1alpha.projects.html new file mode 100644 index 0000000000..044bd27b83 --- /dev/null +++ b/docs/dyn/agentregistry_v1alpha.projects.html @@ -0,0 +1,91 @@ + + + +new_batch_http_request()+Create a BatchHttpRequest object based on the discovery document. + + Args: + callback: callable, A callback to be called for each response, of the + form callback(id, response, exception). The first parameter is the + request id, and the second is the deserialized response object. The + third is an apiclient.errors.HttpError exception object if an HTTP + error occurred while processing the request, or None if no error + occurred. + + Returns: + A BatchHttpRequest object based on the discovery document. ++Agent Registry API . projects
+Instance Methods
++
+locations()+Returns the locations Resource.
+ ++
+close()Close httplib2 connections.
+Method Details
+++ + \ No newline at end of file diff --git a/docs/dyn/agentregistry_v1alpha.projects.locations.agents.html b/docs/dyn/agentregistry_v1alpha.projects.locations.agents.html new file mode 100644 index 0000000000..4687e4f496 --- /dev/null +++ b/docs/dyn/agentregistry_v1alpha.projects.locations.agents.html @@ -0,0 +1,244 @@ + + + +close()+Close httplib2 connections.+Agent Registry API . projects . locations . agents
+Instance Methods
++
+close()Close httplib2 connections.
+ +Gets details of a single Agent.
++
+list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)Lists Agents in a given project and location.
+ +Retrieves the next page of results.
+Method Details
+++ +close()+Close httplib2 connections.+++ +get(name, x__xgafv=None)+Gets details of a single Agent. + +Args: + name: string, Required. Name of the resource (required) + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # Represents an Agent. "A2A" below refers to the Agent-to-Agent protocol. + "agentId": "A String", # Output only. A stable, globally unique identifier for agents. + "attributes": { # Output only. Attributes of the Agent. Valid values: * `agentregistry.googleapis.com/system/Framework`: {"framework": "google-adk"} - the agent framework used to develop the Agent. Example values: "google-adk", "langchain", "custom". * `agentregistry.googleapis.com/system/RuntimeIdentity`: {"principal": "principal://..."} - the runtime identity associated with the Agent. * `agentregistry.googleapis.com/system/RuntimeReference`: {"uri": "//..."} - the URI of the underlying resource hosting the Agent, for example, the Reasoning Engine URI. + "a_key": { + "a_key": "", # Properties of the object. + }, + }, + "card": { # Full Agent Card payload, often obtained from the A2A Agent Card. # Output only. Full Agent Card payload, when available. + "content": { # Output only. The content of the agent card. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Output only. The type of agent card. + }, + "createTime": "A String", # Output only. Create time. + "description": "A String", # Output only. The description of the Agent, often obtained from the A2A Agent Card. Empty if Agent Card has no description. + "displayName": "A String", # Output only. The display name of the agent, often obtained from the A2A Agent Card. + "location": "A String", # Output only. The location where agent is hosted. The value is defined by the hosting environment (i.e. cloud provider). + "name": "A String", # Identifier. The resource name of an Agent. Format: `projects/{project}/locations/{location}/agents/{agent}`. + "protocols": [ # Output only. The connection details for the Agent. + { # Represents the protocol of an Agent. + "interfaces": [ # Output only. The connection details for the Agent. + { # Represents the connection details for an Agent or MCP Server. + "protocolBinding": "A String", # Required. The protocol binding of the interface. + "url": "A String", # Required. The destination URL. + }, + ], + "protocolVersion": "A String", # Output only. The version of the protocol, for example, the A2A Agent Card version. + "type": "A String", # Output only. The type of the protocol. + }, + ], + "skills": [ # Output only. Skills the agent possesses, often obtained from the A2A Agent Card. + { # Represents the skills of an Agent. + "description": "A String", # Output only. A more detailed description of the skill. + "examples": [ # Output only. Example prompts or scenarios this skill can handle. + "A String", + ], + "id": "A String", # Output only. A unique identifier for the agent's skill. + "name": "A String", # Output only. A human-readable name for the agent's skill. + "tags": [ # Output only. Keywords describing the skill. + "A String", + ], + }, + ], + "uid": "A String", # Output only. A universally unique identifier for the Agent. + "updateTime": "A String", # Output only. Update time. + "version": "A String", # Output only. The version of the Agent, often obtained from the A2A Agent Card. Empty if Agent Card has no version or agent is not an A2A Agent. +}+++ +list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)+Lists Agents in a given project and location. + +Args: + parent: string, Required. Parent value for ListAgentsRequest (required) + filter: string, Optional. Filtering results + orderBy: string, Optional. Hint for how to order the results + pageSize: integer, Optional. Requested page size. Server may return fewer items than requested. If unspecified, server will pick an appropriate default. + pageToken: string, Optional. A token identifying a page of results the server should return. + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # Message for response to listing Agents + "agents": [ # The list of Agents. + { # Represents an Agent. "A2A" below refers to the Agent-to-Agent protocol. + "agentId": "A String", # Output only. A stable, globally unique identifier for agents. + "attributes": { # Output only. Attributes of the Agent. Valid values: * `agentregistry.googleapis.com/system/Framework`: {"framework": "google-adk"} - the agent framework used to develop the Agent. Example values: "google-adk", "langchain", "custom". * `agentregistry.googleapis.com/system/RuntimeIdentity`: {"principal": "principal://..."} - the runtime identity associated with the Agent. * `agentregistry.googleapis.com/system/RuntimeReference`: {"uri": "//..."} - the URI of the underlying resource hosting the Agent, for example, the Reasoning Engine URI. + "a_key": { + "a_key": "", # Properties of the object. + }, + }, + "card": { # Full Agent Card payload, often obtained from the A2A Agent Card. # Output only. Full Agent Card payload, when available. + "content": { # Output only. The content of the agent card. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Output only. The type of agent card. + }, + "createTime": "A String", # Output only. Create time. + "description": "A String", # Output only. The description of the Agent, often obtained from the A2A Agent Card. Empty if Agent Card has no description. + "displayName": "A String", # Output only. The display name of the agent, often obtained from the A2A Agent Card. + "location": "A String", # Output only. The location where agent is hosted. The value is defined by the hosting environment (i.e. cloud provider). + "name": "A String", # Identifier. The resource name of an Agent. Format: `projects/{project}/locations/{location}/agents/{agent}`. + "protocols": [ # Output only. The connection details for the Agent. + { # Represents the protocol of an Agent. + "interfaces": [ # Output only. The connection details for the Agent. + { # Represents the connection details for an Agent or MCP Server. + "protocolBinding": "A String", # Required. The protocol binding of the interface. + "url": "A String", # Required. The destination URL. + }, + ], + "protocolVersion": "A String", # Output only. The version of the protocol, for example, the A2A Agent Card version. + "type": "A String", # Output only. The type of the protocol. + }, + ], + "skills": [ # Output only. Skills the agent possesses, often obtained from the A2A Agent Card. + { # Represents the skills of an Agent. + "description": "A String", # Output only. A more detailed description of the skill. + "examples": [ # Output only. Example prompts or scenarios this skill can handle. + "A String", + ], + "id": "A String", # Output only. A unique identifier for the agent's skill. + "name": "A String", # Output only. A human-readable name for the agent's skill. + "tags": [ # Output only. Keywords describing the skill. + "A String", + ], + }, + ], + "uid": "A String", # Output only. A universally unique identifier for the Agent. + "updateTime": "A String", # Output only. Update time. + "version": "A String", # Output only. The version of the Agent, often obtained from the A2A Agent Card. Empty if Agent Card has no version or agent is not an A2A Agent. + }, + ], + "nextPageToken": "A String", # A token identifying a page of results the server should return. +}+++ + \ No newline at end of file diff --git a/docs/dyn/agentregistry_v1alpha.projects.locations.endpoints.html b/docs/dyn/agentregistry_v1alpha.projects.locations.endpoints.html new file mode 100644 index 0000000000..37c968664b --- /dev/null +++ b/docs/dyn/agentregistry_v1alpha.projects.locations.endpoints.html @@ -0,0 +1,187 @@ + + + +list_next()+Retrieves the next page of results. + + Args: + previous_request: The request for the previous page. (required) + previous_response: The response from the request for the previous page. (required) + + Returns: + A request object that you can call 'execute()' on to request the next + page. Returns None if there are no more items in the collection. ++Agent Registry API . projects . locations . endpoints
+Instance Methods
++
+close()Close httplib2 connections.
+ +Gets details of a single Endpoint.
++
+list(parent, filter=None, pageSize=None, pageToken=None, x__xgafv=None)Lists Endpoints in a given project and location.
+ +Retrieves the next page of results.
+Method Details
+++ +close()+Close httplib2 connections.+++ +get(name, x__xgafv=None)+Gets details of a single Endpoint. + +Args: + name: string, Required. The name of the endpoint to retrieve. Format: `projects/{project}/locations/{location}/endpoints/{endpoint}` (required) + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # Represents an Endpoint. + "attributes": { # Output only. Attributes of the Endpoint. Valid values: * `agentregistry.googleapis.com/system/RuntimeReference`: {"uri": "//..."} - the URI of the underlying resource hosting the Endpoint, for example, the GKE Deployment. + "a_key": { + "a_key": "", # Properties of the object. + }, + }, + "createTime": "A String", # Output only. Create time. + "description": "A String", # Output only. Description of an Endpoint. + "displayName": "A String", # Output only. Display name for the Endpoint. + "endpointId": "A String", # Output only. A stable, globally unique identifier for Endpoint. + "interfaces": [ # Required. The connection details for the Endpoint. + { # Represents the connection details for an Agent or MCP Server. + "protocolBinding": "A String", # Required. The protocol binding of the interface. + "url": "A String", # Required. The destination URL. + }, + ], + "name": "A String", # Identifier. The resource name of the Endpoint. Format: `projects/{project}/locations/{location}/endpoints/{endpoint}`. + "updateTime": "A String", # Output only. Update time. +}+++ +list(parent, filter=None, pageSize=None, pageToken=None, x__xgafv=None)+Lists Endpoints in a given project and location. + +Args: + parent: string, Required. The project and location to list endpoints in. Expected format: `projects/{project}/locations/{location}`. (required) + filter: string, Optional. A query string used to filter the list of endpoints returned. The filter expression must follow AIP-160 syntax. Filtering is supported on the `name`, `display_name`, `description`, `version`, and `interfaces` fields. Some examples: * `name = "projects/p1/locations/l1/endpoints/e1"` * `display_name = "my-endpoint"` * `description = "my-endpoint-description"` * `version = "v1"` * `interfaces.transport = "HTTP_JSON"` + pageSize: integer, Optional. Requested page size. Server may return fewer items than requested. If unspecified, server will pick an appropriate default. + pageToken: string, Optional. A token identifying a page of results the server should return. + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # Message for response to listing Endpoints + "endpoints": [ # The list of Endpoint resources matching the parent and filter criteria in the request. Each Endpoint resource follows the format: `projects/{project}/locations/{location}/endpoints/{endpoint}`. + { # Represents an Endpoint. + "attributes": { # Output only. Attributes of the Endpoint. Valid values: * `agentregistry.googleapis.com/system/RuntimeReference`: {"uri": "//..."} - the URI of the underlying resource hosting the Endpoint, for example, the GKE Deployment. + "a_key": { + "a_key": "", # Properties of the object. + }, + }, + "createTime": "A String", # Output only. Create time. + "description": "A String", # Output only. Description of an Endpoint. + "displayName": "A String", # Output only. Display name for the Endpoint. + "endpointId": "A String", # Output only. A stable, globally unique identifier for Endpoint. + "interfaces": [ # Required. The connection details for the Endpoint. + { # Represents the connection details for an Agent or MCP Server. + "protocolBinding": "A String", # Required. The protocol binding of the interface. + "url": "A String", # Required. The destination URL. + }, + ], + "name": "A String", # Identifier. The resource name of the Endpoint. Format: `projects/{project}/locations/{location}/endpoints/{endpoint}`. + "updateTime": "A String", # Output only. Update time. + }, + ], + "nextPageToken": "A String", # A token identifying a page of results the server should return. Used in page_token. +}+++ + \ No newline at end of file diff --git a/docs/dyn/agentregistry_v1alpha.projects.locations.html b/docs/dyn/agentregistry_v1alpha.projects.locations.html new file mode 100644 index 0000000000..d946577bc8 --- /dev/null +++ b/docs/dyn/agentregistry_v1alpha.projects.locations.html @@ -0,0 +1,197 @@ + + + +list_next()+Retrieves the next page of results. + + Args: + previous_request: The request for the previous page. (required) + previous_response: The response from the request for the previous page. (required) + + Returns: + A request object that you can call 'execute()' on to request the next + page. Returns None if there are no more items in the collection. ++Agent Registry API . projects . locations
+Instance Methods
++
+agents()+Returns the agents Resource.
+ ++
+endpoints()+Returns the endpoints Resource.
+ ++
+mcpServers()+Returns the mcpServers Resource.
+ ++
+operations()+Returns the operations Resource.
+ ++
+services()+Returns the services Resource.
+ ++
+close()Close httplib2 connections.
+ +Gets information about a location.
++
+list(name, extraLocationTypes=None, filter=None, pageSize=None, pageToken=None, x__xgafv=None)Lists information about the supported locations for this service. This method lists locations based on the resource scope provided in the [ListLocationsRequest.name] field: * **Global locations**: If `name` is empty, the method lists the public locations available to all projects. * **Project-specific locations**: If `name` follows the format `projects/{project}`, the method lists locations visible to that specific project. This includes public, private, or other project-specific locations enabled for the project. For gRPC and client library implementations, the resource name is passed as the `name` field. For direct service calls, the resource name is incorporated into the request path based on the specific service implementation and version.
+ +Retrieves the next page of results.
+Method Details
+++ +close()+Close httplib2 connections.+++ +get(name, x__xgafv=None)+Gets information about a location. + +Args: + name: string, Resource name for the location. (required) + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # A resource that represents a Google Cloud location. + "displayName": "A String", # The friendly name for this location, typically a nearby city name. For example, "Tokyo". + "labels": { # Cross-service attributes for the location. For example {"cloud.googleapis.com/region": "us-east1"} + "a_key": "A String", + }, + "locationId": "A String", # The canonical id for this location. For example: `"us-east1"`. + "metadata": { # Service-specific metadata. For example the available capacity at the given location. + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + "name": "A String", # Resource name for the location, which may vary between implementations. For example: `"projects/example-project/locations/us-east1"` +}+++ +list(name, extraLocationTypes=None, filter=None, pageSize=None, pageToken=None, x__xgafv=None)+Lists information about the supported locations for this service. This method lists locations based on the resource scope provided in the [ListLocationsRequest.name] field: * **Global locations**: If `name` is empty, the method lists the public locations available to all projects. * **Project-specific locations**: If `name` follows the format `projects/{project}`, the method lists locations visible to that specific project. This includes public, private, or other project-specific locations enabled for the project. For gRPC and client library implementations, the resource name is passed as the `name` field. For direct service calls, the resource name is incorporated into the request path based on the specific service implementation and version. + +Args: + name: string, The resource that owns the locations collection, if applicable. (required) + extraLocationTypes: string, Optional. Do not use this field. It is unsupported and is ignored unless explicitly documented otherwise. This is primarily for internal usage. (repeated) + filter: string, A filter to narrow down results to a preferred subset. The filtering language accepts strings like `"displayName=tokyo"`, and is documented in more detail in [AIP-160](https://google.aip.dev/160). + pageSize: integer, The maximum number of results to return. If not set, the service selects a default. + pageToken: string, A page token received from the `next_page_token` field in the response. Send that page token to receive the subsequent page. + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # The response message for Locations.ListLocations. + "locations": [ # A list of locations that matches the specified filter in the request. + { # A resource that represents a Google Cloud location. + "displayName": "A String", # The friendly name for this location, typically a nearby city name. For example, "Tokyo". + "labels": { # Cross-service attributes for the location. For example {"cloud.googleapis.com/region": "us-east1"} + "a_key": "A String", + }, + "locationId": "A String", # The canonical id for this location. For example: `"us-east1"`. + "metadata": { # Service-specific metadata. For example the available capacity at the given location. + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + "name": "A String", # Resource name for the location, which may vary between implementations. For example: `"projects/example-project/locations/us-east1"` + }, + ], + "nextPageToken": "A String", # The standard List next-page token. +}+++ + \ No newline at end of file diff --git a/docs/dyn/agentregistry_v1alpha.projects.locations.mcpServers.html b/docs/dyn/agentregistry_v1alpha.projects.locations.mcpServers.html new file mode 100644 index 0000000000..1983a1cb88 --- /dev/null +++ b/docs/dyn/agentregistry_v1alpha.projects.locations.mcpServers.html @@ -0,0 +1,214 @@ + + + +list_next()+Retrieves the next page of results. + + Args: + previous_request: The request for the previous page. (required) + previous_response: The response from the request for the previous page. (required) + + Returns: + A request object that you can call 'execute()' on to request the next + page. Returns None if there are no more items in the collection. ++Agent Registry API . projects . locations . mcpServers
+Instance Methods
++
+close()Close httplib2 connections.
+ +Gets details of a single McpServer.
++
+list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)Lists McpServers in a given project and location.
+ +Retrieves the next page of results.
+Method Details
+++ +close()+Close httplib2 connections.+++ +get(name, x__xgafv=None)+Gets details of a single McpServer. + +Args: + name: string, Required. Name of the resource (required) + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # Represents an MCP (Model Context Protocol) Server. + "attributes": { # Output only. Attributes of the MCP Server. Valid values: * `agentregistry.googleapis.com/system/RuntimeIdentity`: {"principal": "principal://..."} - the runtime identity associated with the MCP Server. * `agentregistry.googleapis.com/system/RuntimeReference`: {"uri": "//..."} - the URI of the underlying resource hosting the MCP Server, for example, the GKE Deployment. + "a_key": { + "a_key": "", # Properties of the object. + }, + }, + "createTime": "A String", # Output only. Create time. + "description": "A String", # Output only. The description of the MCP Server. + "displayName": "A String", # Output only. The display name of the MCP Server. + "interfaces": [ # Output only. The connection details for the MCP Server. + { # Represents the connection details for an Agent or MCP Server. + "protocolBinding": "A String", # Required. The protocol binding of the interface. + "url": "A String", # Required. The destination URL. + }, + ], + "mcpServerId": "A String", # Output only. A stable, globally unique identifier for MCP Servers. + "name": "A String", # Identifier. The resource name of the MCP Server. Format: `projects/{project}/locations/{location}/mcpServers/{mcp_server}`. + "tools": [ # Output only. Tools provided by the MCP Server. + { # Represents a single tool provided by an MCP Server. + "annotations": { # Annotations describing the characteristics and behavior of a tool or operation. # Output only. Annotations associated with the tool. + "destructiveHint": True or False, # Output only. If true, the tool may perform destructive updates to its environment. If false, the tool performs only additive updates. NOTE: This property is meaningful only when `read_only_hint == false` Default: true + "idempotentHint": True or False, # Output only. If true, calling the tool repeatedly with the same arguments will have no additional effect on its environment. NOTE: This property is meaningful only when `read_only_hint == false. Default: false + "openWorldHint": True or False, # Output only. If true, this tool may interact with an "open world" of external entities. If false, the tool's domain of interaction is closed. For example, the world of a web search tool is open, whereas that of a memory tool is not. Default: true + "readOnlyHint": True or False, # Output only. If true, the tool does not modify its environment. Default: false + "title": "A String", # Output only. A human-readable title for the tool. + }, + "description": "A String", # Output only. Description of what the tool does. + "name": "A String", # Output only. Human-readable name of the tool. + }, + ], + "updateTime": "A String", # Output only. Update time. +}+++ +list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)+Lists McpServers in a given project and location. + +Args: + parent: string, Required. Parent value for ListMcpServersRequest. Format: `projects/{project}/locations/{location}`. (required) + filter: string, Optional. Filtering results + orderBy: string, Optional. Hint for how to order the results + pageSize: integer, Optional. Requested page size. Server may return fewer items than requested. If unspecified, server will pick an appropriate default. + pageToken: string, Optional. A token identifying a page of results the server should return. + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # Message for response to listing McpServers + "mcpServers": [ # The list of McpServers. + { # Represents an MCP (Model Context Protocol) Server. + "attributes": { # Output only. Attributes of the MCP Server. Valid values: * `agentregistry.googleapis.com/system/RuntimeIdentity`: {"principal": "principal://..."} - the runtime identity associated with the MCP Server. * `agentregistry.googleapis.com/system/RuntimeReference`: {"uri": "//..."} - the URI of the underlying resource hosting the MCP Server, for example, the GKE Deployment. + "a_key": { + "a_key": "", # Properties of the object. + }, + }, + "createTime": "A String", # Output only. Create time. + "description": "A String", # Output only. The description of the MCP Server. + "displayName": "A String", # Output only. The display name of the MCP Server. + "interfaces": [ # Output only. The connection details for the MCP Server. + { # Represents the connection details for an Agent or MCP Server. + "protocolBinding": "A String", # Required. The protocol binding of the interface. + "url": "A String", # Required. The destination URL. + }, + ], + "mcpServerId": "A String", # Output only. A stable, globally unique identifier for MCP Servers. + "name": "A String", # Identifier. The resource name of the MCP Server. Format: `projects/{project}/locations/{location}/mcpServers/{mcp_server}`. + "tools": [ # Output only. Tools provided by the MCP Server. + { # Represents a single tool provided by an MCP Server. + "annotations": { # Annotations describing the characteristics and behavior of a tool or operation. # Output only. Annotations associated with the tool. + "destructiveHint": True or False, # Output only. If true, the tool may perform destructive updates to its environment. If false, the tool performs only additive updates. NOTE: This property is meaningful only when `read_only_hint == false` Default: true + "idempotentHint": True or False, # Output only. If true, calling the tool repeatedly with the same arguments will have no additional effect on its environment. NOTE: This property is meaningful only when `read_only_hint == false. Default: false + "openWorldHint": True or False, # Output only. If true, this tool may interact with an "open world" of external entities. If false, the tool's domain of interaction is closed. For example, the world of a web search tool is open, whereas that of a memory tool is not. Default: true + "readOnlyHint": True or False, # Output only. If true, the tool does not modify its environment. Default: false + "title": "A String", # Output only. A human-readable title for the tool. + }, + "description": "A String", # Output only. Description of what the tool does. + "name": "A String", # Output only. Human-readable name of the tool. + }, + ], + "updateTime": "A String", # Output only. Update time. + }, + ], + "nextPageToken": "A String", # A token identifying a page of results the server should return. +}+++ + \ No newline at end of file diff --git a/docs/dyn/agentregistry_v1alpha.projects.locations.operations.html b/docs/dyn/agentregistry_v1alpha.projects.locations.operations.html new file mode 100644 index 0000000000..6697727e9b --- /dev/null +++ b/docs/dyn/agentregistry_v1alpha.projects.locations.operations.html @@ -0,0 +1,239 @@ + + + +list_next()+Retrieves the next page of results. + + Args: + previous_request: The request for the previous page. (required) + previous_response: The response from the request for the previous page. (required) + + Returns: + A request object that you can call 'execute()' on to request the next + page. Returns None if there are no more items in the collection. ++Agent Registry API . projects . locations . operations
+Instance Methods
++
+cancel(name, body=None, x__xgafv=None)Starts asynchronous cancellation on a long-running operation. The server makes a best effort to cancel the operation, but success is not guaranteed. If the server doesn't support this method, it returns `google.rpc.Code.UNIMPLEMENTED`. Clients can use Operations.GetOperation or other methods to check whether the cancellation succeeded or whether the operation completed despite cancellation. On successful cancellation, the operation is not deleted; instead, it becomes an operation with an Operation.error value with a google.rpc.Status.code of `1`, corresponding to `Code.CANCELLED`.
++
+close()Close httplib2 connections.
+ +Deletes a long-running operation. This method indicates that the client is no longer interested in the operation result. It does not cancel the operation. If the server doesn't support this method, it returns `google.rpc.Code.UNIMPLEMENTED`.
+ +Gets the latest state of a long-running operation. Clients can use this method to poll the operation result at intervals as recommended by the API service.
++
+list(name, filter=None, pageSize=None, pageToken=None, returnPartialSuccess=None, x__xgafv=None)Lists operations that match the specified filter in the request. If the server doesn't support this method, it returns `UNIMPLEMENTED`.
+ +Retrieves the next page of results.
+Method Details
+++ +cancel(name, body=None, x__xgafv=None)+Starts asynchronous cancellation on a long-running operation. The server makes a best effort to cancel the operation, but success is not guaranteed. If the server doesn't support this method, it returns `google.rpc.Code.UNIMPLEMENTED`. Clients can use Operations.GetOperation or other methods to check whether the cancellation succeeded or whether the operation completed despite cancellation. On successful cancellation, the operation is not deleted; instead, it becomes an operation with an Operation.error value with a google.rpc.Status.code of `1`, corresponding to `Code.CANCELLED`. + +Args: + name: string, The name of the operation resource to be cancelled. (required) + body: object, The request body. + The object takes the form of: + +{ # The request message for Operations.CancelOperation. +} + + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance: service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); } +}+++ +close()+Close httplib2 connections.+++ +delete(name, x__xgafv=None)+Deletes a long-running operation. This method indicates that the client is no longer interested in the operation result. It does not cancel the operation. If the server doesn't support this method, it returns `google.rpc.Code.UNIMPLEMENTED`. + +Args: + name: string, The name of the operation resource to be deleted. (required) + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance: service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); } +}+++ +get(name, x__xgafv=None)+Gets the latest state of a long-running operation. Clients can use this method to poll the operation result at intervals as recommended by the API service. + +Args: + name: string, The name of the operation resource. (required) + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # This resource represents a long-running operation that is the result of a network API call. + "done": True or False, # If the value is `false`, it means the operation is still in progress. If `true`, the operation is completed, and either `error` or `response` is available. + "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # The error result of the operation in case of failure or cancellation. + "code": 42, # The status code, which should be an enum value of google.rpc.Code. + "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use. + { + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + ], + "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client. + }, + "metadata": { # Service-specific metadata associated with the operation. It typically contains progress information and common metadata such as create time. Some services might not provide such metadata. Any method that returns a long-running operation should document the metadata type, if any. + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + "name": "A String", # The server-assigned name, which is only unique within the same service that originally returns it. If you use the default HTTP mapping, the `name` should be a resource name ending with `operations/{unique_id}`. + "response": { # The normal, successful response of the operation. If the original method returns no data on success, such as `Delete`, the response is `google.protobuf.Empty`. If the original method is standard `Get`/`Create`/`Update`, the response should be the resource. For other methods, the response should have the type `XxxResponse`, where `Xxx` is the original method name. For example, if the original method name is `TakeSnapshot()`, the inferred response type is `TakeSnapshotResponse`. + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, +}+++ +list(name, filter=None, pageSize=None, pageToken=None, returnPartialSuccess=None, x__xgafv=None)+Lists operations that match the specified filter in the request. If the server doesn't support this method, it returns `UNIMPLEMENTED`. + +Args: + name: string, The name of the operation's parent resource. (required) + filter: string, The standard list filter. + pageSize: integer, The standard list page size. + pageToken: string, The standard list page token. + returnPartialSuccess: boolean, When set to `true`, operations that are reachable are returned as normal, and those that are unreachable are returned in the ListOperationsResponse.unreachable field. This can only be `true` when reading across collections. For example, when `parent` is set to `"projects/example/locations/-"`. This field is not supported by default and will result in an `UNIMPLEMENTED` error if set unless explicitly documented otherwise in service or product specific documentation. + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # The response message for Operations.ListOperations. + "nextPageToken": "A String", # The standard List next-page token. + "operations": [ # A list of operations that matches the specified filter in the request. + { # This resource represents a long-running operation that is the result of a network API call. + "done": True or False, # If the value is `false`, it means the operation is still in progress. If `true`, the operation is completed, and either `error` or `response` is available. + "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # The error result of the operation in case of failure or cancellation. + "code": 42, # The status code, which should be an enum value of google.rpc.Code. + "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use. + { + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + ], + "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client. + }, + "metadata": { # Service-specific metadata associated with the operation. It typically contains progress information and common metadata such as create time. Some services might not provide such metadata. Any method that returns a long-running operation should document the metadata type, if any. + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + "name": "A String", # The server-assigned name, which is only unique within the same service that originally returns it. If you use the default HTTP mapping, the `name` should be a resource name ending with `operations/{unique_id}`. + "response": { # The normal, successful response of the operation. If the original method returns no data on success, such as `Delete`, the response is `google.protobuf.Empty`. If the original method is standard `Get`/`Create`/`Update`, the response should be the resource. For other methods, the response should have the type `XxxResponse`, where `Xxx` is the original method name. For example, if the original method name is `TakeSnapshot()`, the inferred response type is `TakeSnapshotResponse`. + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + }, + ], + "unreachable": [ # Unordered list. Unreachable resources. Populated when the request sets `ListOperationsRequest.return_partial_success` and reads across collections. For example, when attempting to list all resources across all supported locations. + "A String", + ], +}+++ + \ No newline at end of file diff --git a/docs/dyn/agentregistry_v1alpha.projects.locations.services.html b/docs/dyn/agentregistry_v1alpha.projects.locations.services.html new file mode 100644 index 0000000000..353150de18 --- /dev/null +++ b/docs/dyn/agentregistry_v1alpha.projects.locations.services.html @@ -0,0 +1,400 @@ + + + +list_next()+Retrieves the next page of results. + + Args: + previous_request: The request for the previous page. (required) + previous_response: The response from the request for the previous page. (required) + + Returns: + A request object that you can call 'execute()' on to request the next + page. Returns None if there are no more items in the collection. ++Agent Registry API . projects . locations . services
+Instance Methods
++
+close()Close httplib2 connections.
++
+create(parent, body=None, requestId=None, serviceId=None, x__xgafv=None)Creates a new Service in a given project and location.
++
+delete(name, requestId=None, x__xgafv=None)Deletes a single Service.
+ +Gets details of a single Service.
++
+list(parent, filter=None, pageSize=None, pageToken=None, x__xgafv=None)Lists Services in a given project and location.
+ +Retrieves the next page of results.
++
+patch(name, body=None, requestId=None, updateMask=None, x__xgafv=None)Updates the parameters of a single Service.
+Method Details
+++ +close()+Close httplib2 connections.+++ +create(parent, body=None, requestId=None, serviceId=None, x__xgafv=None)+Creates a new Service in a given project and location. + +Args: + parent: string, Required. The project and location to create the Service in. Expected format: `projects/{project}/locations/{location}`. (required) + body: object, The request body. + The object takes the form of: + +{ # Represents a user-defined Service. + "agentSpec": { # The spec of the agent. # Optional. The spec of the Agent. When `agent_spec` is set, the type of the service is Agent. + "content": { # Optional. The content of the Agent spec in the JSON format. This payload is validated against the schema for the specified type. The content size is limited to `10KB`. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the agent spec content. + }, + "createTime": "A String", # Output only. Create time. + "description": "A String", # Optional. User-defined description of an Service. Can have a maximum length of `2048` characters. + "displayName": "A String", # Optional. User-defined display name for the Service. Can have a maximum length of `63` characters. + "endpointSpec": { # The spec of the endpoint. # Optional. The spec of the Endpoint. When `endpoint_spec` is set, the type of the service is Endpoint. + "content": { # Optional. The content of the endpoint spec. Reserved for future use. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the endpoint spec content. + }, + "interfaces": [ # Optional. The connection details for the Service. + { # Represents the connection details for an Agent or MCP Server. + "protocolBinding": "A String", # Required. The protocol binding of the interface. + "url": "A String", # Required. The destination URL. + }, + ], + "mcpServerSpec": { # The spec of the MCP Server. # Optional. The spec of the MCP Server. When `mcp_server_spec` is set, the type of the service is MCP Server. + "content": { # Optional. The content of the MCP Server spec. This payload is validated against the schema for the specified type. The content size is limited to `10KB`. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the MCP Server spec content. + }, + "name": "A String", # Identifier. The resource name of the Service. Format: `projects/{project}/locations/{location}/services/{service}`. + "updateTime": "A String", # Output only. Update time. +} + + requestId: string, Optional. An optional request ID to identify requests. Specify a unique request ID so that if you must retry your request, the server will know to ignore the request if it has already been completed. The server will guarantee that for at least 60 minutes since the first request. For example, consider a situation where you make an initial request and the request times out. If you make the request again with the same request ID, the server can check if original operation with the same request ID was received, and if so, will ignore the second request. This prevents clients from accidentally creating duplicate commitments. The request ID must be a valid UUID with the exception that zero UUID is not supported (00000000-0000-0000-0000-000000000000). + serviceId: string, Required. The ID to use for the service, which will become the final component of the service's resource name. This value should be 4-63 characters, and valid characters are `/a-z-/`. + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # This resource represents a long-running operation that is the result of a network API call. + "done": True or False, # If the value is `false`, it means the operation is still in progress. If `true`, the operation is completed, and either `error` or `response` is available. + "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # The error result of the operation in case of failure or cancellation. + "code": 42, # The status code, which should be an enum value of google.rpc.Code. + "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use. + { + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + ], + "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client. + }, + "metadata": { # Service-specific metadata associated with the operation. It typically contains progress information and common metadata such as create time. Some services might not provide such metadata. Any method that returns a long-running operation should document the metadata type, if any. + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + "name": "A String", # The server-assigned name, which is only unique within the same service that originally returns it. If you use the default HTTP mapping, the `name` should be a resource name ending with `operations/{unique_id}`. + "response": { # The normal, successful response of the operation. If the original method returns no data on success, such as `Delete`, the response is `google.protobuf.Empty`. If the original method is standard `Get`/`Create`/`Update`, the response should be the resource. For other methods, the response should have the type `XxxResponse`, where `Xxx` is the original method name. For example, if the original method name is `TakeSnapshot()`, the inferred response type is `TakeSnapshotResponse`. + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, +}+++ +delete(name, requestId=None, x__xgafv=None)+Deletes a single Service. + +Args: + name: string, Required. The name of the Service. Format: `projects/{project}/locations/{location}/services/{service}`. (required) + requestId: string, Optional. An optional request ID to identify requests. Specify a unique request ID so that if you must retry your request, the server will know to ignore the request if it has already been completed. The server will guarantee that for at least 60 minutes after the first request. For example, consider a situation where you make an initial request and the request times out. If you make the request again with the same request ID, the server can check if original operation with the same request ID was received, and if so, will ignore the second request. This prevents clients from accidentally creating duplicate commitments. The request ID must be a valid UUID with the exception that zero UUID is not supported (00000000-0000-0000-0000-000000000000). + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # This resource represents a long-running operation that is the result of a network API call. + "done": True or False, # If the value is `false`, it means the operation is still in progress. If `true`, the operation is completed, and either `error` or `response` is available. + "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # The error result of the operation in case of failure or cancellation. + "code": 42, # The status code, which should be an enum value of google.rpc.Code. + "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use. + { + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + ], + "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client. + }, + "metadata": { # Service-specific metadata associated with the operation. It typically contains progress information and common metadata such as create time. Some services might not provide such metadata. Any method that returns a long-running operation should document the metadata type, if any. + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + "name": "A String", # The server-assigned name, which is only unique within the same service that originally returns it. If you use the default HTTP mapping, the `name` should be a resource name ending with `operations/{unique_id}`. + "response": { # The normal, successful response of the operation. If the original method returns no data on success, such as `Delete`, the response is `google.protobuf.Empty`. If the original method is standard `Get`/`Create`/`Update`, the response should be the resource. For other methods, the response should have the type `XxxResponse`, where `Xxx` is the original method name. For example, if the original method name is `TakeSnapshot()`, the inferred response type is `TakeSnapshotResponse`. + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, +}+++ +get(name, x__xgafv=None)+Gets details of a single Service. + +Args: + name: string, Required. The name of the Service. Format: `projects/{project}/locations/{location}/services/{service}`. (required) + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # Represents a user-defined Service. + "agentSpec": { # The spec of the agent. # Optional. The spec of the Agent. When `agent_spec` is set, the type of the service is Agent. + "content": { # Optional. The content of the Agent spec in the JSON format. This payload is validated against the schema for the specified type. The content size is limited to `10KB`. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the agent spec content. + }, + "createTime": "A String", # Output only. Create time. + "description": "A String", # Optional. User-defined description of an Service. Can have a maximum length of `2048` characters. + "displayName": "A String", # Optional. User-defined display name for the Service. Can have a maximum length of `63` characters. + "endpointSpec": { # The spec of the endpoint. # Optional. The spec of the Endpoint. When `endpoint_spec` is set, the type of the service is Endpoint. + "content": { # Optional. The content of the endpoint spec. Reserved for future use. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the endpoint spec content. + }, + "interfaces": [ # Optional. The connection details for the Service. + { # Represents the connection details for an Agent or MCP Server. + "protocolBinding": "A String", # Required. The protocol binding of the interface. + "url": "A String", # Required. The destination URL. + }, + ], + "mcpServerSpec": { # The spec of the MCP Server. # Optional. The spec of the MCP Server. When `mcp_server_spec` is set, the type of the service is MCP Server. + "content": { # Optional. The content of the MCP Server spec. This payload is validated against the schema for the specified type. The content size is limited to `10KB`. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the MCP Server spec content. + }, + "name": "A String", # Identifier. The resource name of the Service. Format: `projects/{project}/locations/{location}/services/{service}`. + "updateTime": "A String", # Output only. Update time. +}+++ +list(parent, filter=None, pageSize=None, pageToken=None, x__xgafv=None)+Lists Services in a given project and location. + +Args: + parent: string, Required. The project and location to list services in. Expected format: `projects/{project}/locations/{location}`. (required) + filter: string, Optional. A query string used to filter the list of services returned. The filter expression must follow AIP-160 syntax. Filtering is supported on the `name`, `display_name`, `description`, and `labels` fields. Some examples: * `name = "projects/p1/locations/l1/services/s1"` * `display_name = "my-service"` * `description : "myservice description"` * `labels.env = "prod"` + pageSize: integer, Optional. Requested page size. Server may return fewer items than requested. If unspecified, server will pick an appropriate default. + pageToken: string, Optional. A token identifying a page of results the server should return. + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # Message for response to listing Services + "nextPageToken": "A String", # A token identifying a page of results the server should return. Used in page_token. + "services": [ # The list of Service resources matching the parent and filter criteria in the request. Each Service resource follows the format: `projects/{project}/locations/{location}/services/{service}`. + { # Represents a user-defined Service. + "agentSpec": { # The spec of the agent. # Optional. The spec of the Agent. When `agent_spec` is set, the type of the service is Agent. + "content": { # Optional. The content of the Agent spec in the JSON format. This payload is validated against the schema for the specified type. The content size is limited to `10KB`. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the agent spec content. + }, + "createTime": "A String", # Output only. Create time. + "description": "A String", # Optional. User-defined description of an Service. Can have a maximum length of `2048` characters. + "displayName": "A String", # Optional. User-defined display name for the Service. Can have a maximum length of `63` characters. + "endpointSpec": { # The spec of the endpoint. # Optional. The spec of the Endpoint. When `endpoint_spec` is set, the type of the service is Endpoint. + "content": { # Optional. The content of the endpoint spec. Reserved for future use. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the endpoint spec content. + }, + "interfaces": [ # Optional. The connection details for the Service. + { # Represents the connection details for an Agent or MCP Server. + "protocolBinding": "A String", # Required. The protocol binding of the interface. + "url": "A String", # Required. The destination URL. + }, + ], + "mcpServerSpec": { # The spec of the MCP Server. # Optional. The spec of the MCP Server. When `mcp_server_spec` is set, the type of the service is MCP Server. + "content": { # Optional. The content of the MCP Server spec. This payload is validated against the schema for the specified type. The content size is limited to `10KB`. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the MCP Server spec content. + }, + "name": "A String", # Identifier. The resource name of the Service. Format: `projects/{project}/locations/{location}/services/{service}`. + "updateTime": "A String", # Output only. Update time. + }, + ], +}+++ +list_next()+Retrieves the next page of results. + + Args: + previous_request: The request for the previous page. (required) + previous_response: The response from the request for the previous page. (required) + + Returns: + A request object that you can call 'execute()' on to request the next + page. Returns None if there are no more items in the collection. ++++ + \ No newline at end of file diff --git a/docs/dyn/aiplatform_v1.endpoints.html b/docs/dyn/aiplatform_v1.endpoints.html index 26a6612b79..b8b2df3f39 100644 --- a/docs/dyn/aiplatform_v1.endpoints.html +++ b/docs/dyn/aiplatform_v1.endpoints.html @@ -336,7 +336,7 @@patch(name, body=None, requestId=None, updateMask=None, x__xgafv=None)+Updates the parameters of a single Service. + +Args: + name: string, Identifier. The resource name of the Service. Format: `projects/{project}/locations/{location}/services/{service}`. (required) + body: object, The request body. + The object takes the form of: + +{ # Represents a user-defined Service. + "agentSpec": { # The spec of the agent. # Optional. The spec of the Agent. When `agent_spec` is set, the type of the service is Agent. + "content": { # Optional. The content of the Agent spec in the JSON format. This payload is validated against the schema for the specified type. The content size is limited to `10KB`. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the agent spec content. + }, + "createTime": "A String", # Output only. Create time. + "description": "A String", # Optional. User-defined description of an Service. Can have a maximum length of `2048` characters. + "displayName": "A String", # Optional. User-defined display name for the Service. Can have a maximum length of `63` characters. + "endpointSpec": { # The spec of the endpoint. # Optional. The spec of the Endpoint. When `endpoint_spec` is set, the type of the service is Endpoint. + "content": { # Optional. The content of the endpoint spec. Reserved for future use. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the endpoint spec content. + }, + "interfaces": [ # Optional. The connection details for the Service. + { # Represents the connection details for an Agent or MCP Server. + "protocolBinding": "A String", # Required. The protocol binding of the interface. + "url": "A String", # Required. The destination URL. + }, + ], + "mcpServerSpec": { # The spec of the MCP Server. # Optional. The spec of the MCP Server. When `mcp_server_spec` is set, the type of the service is MCP Server. + "content": { # Optional. The content of the MCP Server spec. This payload is validated against the schema for the specified type. The content size is limited to `10KB`. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the MCP Server spec content. + }, + "name": "A String", # Identifier. The resource name of the Service. Format: `projects/{project}/locations/{location}/services/{service}`. + "updateTime": "A String", # Output only. Update time. +} + + requestId: string, Optional. An optional request ID to identify requests. Specify a unique request ID so that if you must retry your request, the server will know to ignore the request if it has already been completed. The server will guarantee that for at least 60 minutes since the first request. For example, consider a situation where you make an initial request and the request times out. If you make the request again with the same request ID, the server can check if original operation with the same request ID was received, and if so, will ignore the second request. This prevents clients from accidentally creating duplicate commitments. The request ID must be a valid UUID with the exception that zero UUID is not supported (00000000-0000-0000-0000-000000000000). + updateMask: string, Optional. Field mask is used to specify the fields to be overwritten in the Service resource by the update. The fields specified in the update_mask are relative to the resource, not the full request. A field will be overwritten if it is in the mask. If the user does not provide a mask then all fields present in the request will be overwritten. + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # This resource represents a long-running operation that is the result of a network API call. + "done": True or False, # If the value is `false`, it means the operation is still in progress. If `true`, the operation is completed, and either `error` or `response` is available. + "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # The error result of the operation in case of failure or cancellation. + "code": 42, # The status code, which should be an enum value of google.rpc.Code. + "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use. + { + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + ], + "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client. + }, + "metadata": { # Service-specific metadata associated with the operation. It typically contains progress information and common metadata such as create time. Some services might not provide such metadata. Any method that returns a long-running operation should document the metadata type, if any. + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + "name": "A String", # The server-assigned name, which is only unique within the same service that originally returns it. If you use the default HTTP mapping, the `name` should be a resource name ending with `operations/{unique_id}`. + "response": { # The normal, successful response of the operation. If the original method returns no data on success, such as `Delete`, the response is `google.protobuf.Empty`. If the original method is standard `Get`/`Create`/`Update`, the response should be the resource. For other methods, the response should have the type `XxxResponse`, where `Xxx` is the original method name. For example, if the original method name is `TakeSnapshot()`, the inferred response type is `TakeSnapshotResponse`. + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, +}+Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -894,7 +894,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1649,6 +1649,9 @@Method Details
"instances": [ # Required. The instances that are the input to the prediction call. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint's DeployedModels' Model's PredictSchemata's instance_schema_uri. "", ], + "labels": { # Optional. The labels with user-defined metadata for the request. It is used for billing and reporting only. Label keys and values can be no longer than 63 characters (Unicode codepoints) and can only contain lowercase letters, numeric characters, underscores, and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter. + "a_key": "A String", + }, "parameters": "", # Optional. The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint's DeployedModels' Model's PredictSchemata's parameters_schema_uri. } @@ -1789,7 +1792,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], diff --git a/docs/dyn/aiplatform_v1.projects.locations.endpoints.html b/docs/dyn/aiplatform_v1.projects.locations.endpoints.html index e3bb6f47ee..e05b5489b6 100644 --- a/docs/dyn/aiplatform_v1.projects.locations.endpoints.html +++ b/docs/dyn/aiplatform_v1.projects.locations.endpoints.html @@ -404,7 +404,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1858,7 +1858,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -3867,6 +3867,9 @@Method Details
"instances": [ # Required. The instances that are the input to the prediction call. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint's DeployedModels' Model's PredictSchemata's instance_schema_uri. "", ], + "labels": { # Optional. The labels with user-defined metadata for the request. It is used for billing and reporting only. Label keys and values can be no longer than 63 characters (Unicode codepoints) and can only contain lowercase letters, numeric characters, underscores, and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter. + "a_key": "A String", + }, "parameters": "", # Optional. The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint's DeployedModels' Model's PredictSchemata's parameters_schema_uri. } @@ -4235,7 +4238,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], diff --git a/docs/dyn/aiplatform_v1.projects.locations.evaluationRuns.html b/docs/dyn/aiplatform_v1.projects.locations.evaluationRuns.html index 93be21e9d1..4e175834e4 100644 --- a/docs/dyn/aiplatform_v1.projects.locations.evaluationRuns.html +++ b/docs/dyn/aiplatform_v1.projects.locations.evaluationRuns.html @@ -187,7 +187,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -283,6 +283,7 @@Method Details
}, "datasetCustomMetrics": [ # Optional. Specifications for custom dataset-level aggregations. { # Defines a custom dataset-level aggregation. + "aggregationFunction": "A String", # Required. The Python code string containing the aggregation function. Expected function signature: `def aggregate(instances: list[dict[str, Any]]) -> dict[str, float]:` The `instances` argument is a list of dictionaries, where each dictionary represents a single evaluation result item. The structure of each dictionary corresponds to the fields in the `EvaluationResult` message. This includes: - `"request"`: Contains the original input data and model inputs (from `EvaluationResult.EvaluationRequest`). - `"candidate_results"`: Contains the results of any instance-level metrics (from `EvaluationResult.CandidateResults`). Example of a single item in the `instances` list: { "request": { "prompt": {"text": "What is the capital of France?"}, "golden_response": {"text": "Paris"}, "candidate_responses": [{"candidate": "model-v1", "text": "Paris"}] }, "candidate_results": [ {"metric": "exact_match", "score": 1.0}, {"metric": "bleu", "score": 0.9} ] } "displayName": "A String", # Optional. A display name for this custom summary metric. Used to prefix keys in the output summaryMetrics map. If not provided, a default name like "dataset_custom_metric_1", "dataset_custom_metric_2", etc., will be generated based on the order in the repeated field. }, ], @@ -321,7 +322,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -423,6 +424,7 @@Method Details
}, }, "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification. + "metricResourceName": "A String", # Optional. Resource name of the metric definition. "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. @@ -446,7 +448,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -596,7 +598,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -722,7 +724,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -825,6 +827,18 @@Method Details
"rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. "systemInstruction": "A String", # Optional. System instructions for the judge model. }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. @@ -853,6 +867,7 @@Method Details
"useStemmer": True or False, # Optional. Whether to use stemmer to compute rouge score. }, }, + "metricResourceName": "A String", # Optional. The resource name of the metric definition. "predefinedMetricSpec": { # Specification for a pre-defined metric. # Spec for a pre-defined metric. "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". "parameters": { # Optional. The parameters needed to run the pre-defined metric. @@ -897,7 +912,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -993,6 +1008,7 @@Method Details
}, "metricPromptTemplate": "A String", # Optional. Template for the prompt used by the judge model to evaluate against rubrics. "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics for evaluation using this specification. + "metricResourceName": "A String", # Optional. Resource name of the metric definition. "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. @@ -1016,7 +1032,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1141,6 +1157,7 @@Method Details
}, }, "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification. + "metricResourceName": "A String", # Optional. Resource name of the metric definition. "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. @@ -1164,7 +1181,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1280,7 +1297,136 @@Method Details
}, "evaluationSetSnapshot": "A String", # Output only. The specific evaluation set of the evaluation run. For runs with an evaluation set input, this will be that same set. For runs with BigQuery input, it's the sampled BigQuery dataset. "inferenceConfigs": { # Optional. The candidate to inference config map for the evaluation run. The candidate can be up to 128 characters long and can consist of any UTF-8 characters. - "a_key": { # An inference config used for model inference during the evaluation run. + "a_key": { # Defines the configuration for a candidate model or agent being evaluated. `InferenceConfig` encapsulates all the necessary information to invoke or scrape the candidate during the evaluation run. This includes direct model inference parameters, agent execution settings, and multi-turn scraping configurations (such as user simulators). It serves as the primary representation of the candidate across different stages of the evaluation process. + "agentRunConfig": { # Configuration for Agent Run. # Optional. Agent run config. + "agentEngine": "A String", # Optional. The resource name of the Agent Engine. Format: projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine} For example: projects/123/locations/us-central1/reasoningEngines/456 + "sessionInput": { # Session input to run an Agent. # Optional. The session input to get agent running results. + "parameters": { # Optional. Additional parameters for the session, like app_name, etc. For example, {"app_name": "my-app"}. + "a_key": "A String", + }, + "sessionState": { # Optional. Session specific memory which stores key conversation points. + "a_key": "", # Properties of the object. + }, + "userId": "A String", # Optional. The user id for the agent session. The ID can be up to 128 characters long. + }, + "userSimulatorConfig": { # Used for multi-turn agent scraping. Contains configuration for a user simulator that uses an LLM to generate messages on behalf of the user. # The configuration for a user simulator that uses an LLM to generate messages on behalf of the user. + "maxTurn": 42, # Maximum number of invocations allowed by the multi-turn agent scraping. This property allows us to stop a run-off conversation, where the agent and the user simulator get into a never ending loop. The initial fixed prompt is also counted as an invocation. + "modelConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # The configuration for the model. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "modelName": "A String", # The model name to use for multi-turn agent scraping to get next user message, e.g. "gemini-3-flash-preview". + }, + }, "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Generation config. "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. @@ -1302,7 +1448,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1394,7 +1540,7 @@Method Details
"topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. }, - "model": "A String", # Optional. The fully qualified name of the publisher model or endpoint to use. Anthropic and Llama third-party models are also supported through Model Garden. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Third-party model format: `projects/{project}/locations/{location}/publishers/anthropic/models/{model}` `projects/{project}/locations/{location}/publishers/llama/models/{model}` Endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "model": "A String", # Optional. The fully qualified name of the publisher model or endpoint to use. Anthropic and Llama third-party models are also supported through Model Garden. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Third-party model formats: `projects/{project}/locations/{location}/publishers/anthropic/models/{model}` or `projects/{project}/locations/{location}/publishers/llama/models/{model}` Endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` }, }, "labels": { # Optional. Labels for the evaluation run. @@ -1466,7 +1612,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1562,6 +1708,7 @@Method Details
}, "datasetCustomMetrics": [ # Optional. Specifications for custom dataset-level aggregations. { # Defines a custom dataset-level aggregation. + "aggregationFunction": "A String", # Required. The Python code string containing the aggregation function. Expected function signature: `def aggregate(instances: list[dict[str, Any]]) -> dict[str, float]:` The `instances` argument is a list of dictionaries, where each dictionary represents a single evaluation result item. The structure of each dictionary corresponds to the fields in the `EvaluationResult` message. This includes: - `"request"`: Contains the original input data and model inputs (from `EvaluationResult.EvaluationRequest`). - `"candidate_results"`: Contains the results of any instance-level metrics (from `EvaluationResult.CandidateResults`). Example of a single item in the `instances` list: { "request": { "prompt": {"text": "What is the capital of France?"}, "golden_response": {"text": "Paris"}, "candidate_responses": [{"candidate": "model-v1", "text": "Paris"}] }, "candidate_results": [ {"metric": "exact_match", "score": 1.0}, {"metric": "bleu", "score": 0.9} ] } "displayName": "A String", # Optional. A display name for this custom summary metric. Used to prefix keys in the output summaryMetrics map. If not provided, a default name like "dataset_custom_metric_1", "dataset_custom_metric_2", etc., will be generated based on the order in the repeated field. }, ], @@ -1600,7 +1747,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1702,6 +1849,7 @@Method Details
}, }, "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification. + "metricResourceName": "A String", # Optional. Resource name of the metric definition. "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. @@ -1725,7 +1873,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1875,7 +2023,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2001,7 +2149,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2104,6 +2252,18 @@Method Details
"rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. "systemInstruction": "A String", # Optional. System instructions for the judge model. }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. @@ -2132,6 +2292,7 @@Method Details
"useStemmer": True or False, # Optional. Whether to use stemmer to compute rouge score. }, }, + "metricResourceName": "A String", # Optional. The resource name of the metric definition. "predefinedMetricSpec": { # Specification for a pre-defined metric. # Spec for a pre-defined metric. "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". "parameters": { # Optional. The parameters needed to run the pre-defined metric. @@ -2176,7 +2337,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2272,6 +2433,7 @@Method Details
}, "metricPromptTemplate": "A String", # Optional. Template for the prompt used by the judge model to evaluate against rubrics. "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics for evaluation using this specification. + "metricResourceName": "A String", # Optional. Resource name of the metric definition. "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. @@ -2295,7 +2457,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2420,6 +2582,7 @@Method Details
}, }, "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification. + "metricResourceName": "A String", # Optional. Resource name of the metric definition. "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. @@ -2443,7 +2606,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2559,7 +2722,136 @@Method Details
}, "evaluationSetSnapshot": "A String", # Output only. The specific evaluation set of the evaluation run. For runs with an evaluation set input, this will be that same set. For runs with BigQuery input, it's the sampled BigQuery dataset. "inferenceConfigs": { # Optional. The candidate to inference config map for the evaluation run. The candidate can be up to 128 characters long and can consist of any UTF-8 characters. - "a_key": { # An inference config used for model inference during the evaluation run. + "a_key": { # Defines the configuration for a candidate model or agent being evaluated. `InferenceConfig` encapsulates all the necessary information to invoke or scrape the candidate during the evaluation run. This includes direct model inference parameters, agent execution settings, and multi-turn scraping configurations (such as user simulators). It serves as the primary representation of the candidate across different stages of the evaluation process. + "agentRunConfig": { # Configuration for Agent Run. # Optional. Agent run config. + "agentEngine": "A String", # Optional. The resource name of the Agent Engine. Format: projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine} For example: projects/123/locations/us-central1/reasoningEngines/456 + "sessionInput": { # Session input to run an Agent. # Optional. The session input to get agent running results. + "parameters": { # Optional. Additional parameters for the session, like app_name, etc. For example, {"app_name": "my-app"}. + "a_key": "A String", + }, + "sessionState": { # Optional. Session specific memory which stores key conversation points. + "a_key": "", # Properties of the object. + }, + "userId": "A String", # Optional. The user id for the agent session. The ID can be up to 128 characters long. + }, + "userSimulatorConfig": { # Used for multi-turn agent scraping. Contains configuration for a user simulator that uses an LLM to generate messages on behalf of the user. # The configuration for a user simulator that uses an LLM to generate messages on behalf of the user. + "maxTurn": 42, # Maximum number of invocations allowed by the multi-turn agent scraping. This property allows us to stop a run-off conversation, where the agent and the user simulator get into a never ending loop. The initial fixed prompt is also counted as an invocation. + "modelConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # The configuration for the model. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "modelName": "A String", # The model name to use for multi-turn agent scraping to get next user message, e.g. "gemini-3-flash-preview". + }, + }, "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Generation config. "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. @@ -2581,7 +2873,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2673,7 +2965,7 @@Method Details
"topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. }, - "model": "A String", # Optional. The fully qualified name of the publisher model or endpoint to use. Anthropic and Llama third-party models are also supported through Model Garden. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Third-party model format: `projects/{project}/locations/{location}/publishers/anthropic/models/{model}` `projects/{project}/locations/{location}/publishers/llama/models/{model}` Endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "model": "A String", # Optional. The fully qualified name of the publisher model or endpoint to use. Anthropic and Llama third-party models are also supported through Model Garden. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Third-party model formats: `projects/{project}/locations/{location}/publishers/anthropic/models/{model}` or `projects/{project}/locations/{location}/publishers/llama/models/{model}` Endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` }, }, "labels": { # Optional. Labels for the evaluation run. @@ -2787,7 +3079,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2883,6 +3175,7 @@Method Details
}, "datasetCustomMetrics": [ # Optional. Specifications for custom dataset-level aggregations. { # Defines a custom dataset-level aggregation. + "aggregationFunction": "A String", # Required. The Python code string containing the aggregation function. Expected function signature: `def aggregate(instances: list[dict[str, Any]]) -> dict[str, float]:` The `instances` argument is a list of dictionaries, where each dictionary represents a single evaluation result item. The structure of each dictionary corresponds to the fields in the `EvaluationResult` message. This includes: - `"request"`: Contains the original input data and model inputs (from `EvaluationResult.EvaluationRequest`). - `"candidate_results"`: Contains the results of any instance-level metrics (from `EvaluationResult.CandidateResults`). Example of a single item in the `instances` list: { "request": { "prompt": {"text": "What is the capital of France?"}, "golden_response": {"text": "Paris"}, "candidate_responses": [{"candidate": "model-v1", "text": "Paris"}] }, "candidate_results": [ {"metric": "exact_match", "score": 1.0}, {"metric": "bleu", "score": 0.9} ] } "displayName": "A String", # Optional. A display name for this custom summary metric. Used to prefix keys in the output summaryMetrics map. If not provided, a default name like "dataset_custom_metric_1", "dataset_custom_metric_2", etc., will be generated based on the order in the repeated field. }, ], @@ -2921,7 +3214,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -3023,6 +3316,7 @@Method Details
}, }, "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification. + "metricResourceName": "A String", # Optional. Resource name of the metric definition. "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. @@ -3046,7 +3340,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -3196,7 +3490,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -3322,7 +3616,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -3425,6 +3719,18 @@Method Details
"rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. "systemInstruction": "A String", # Optional. System instructions for the judge model. }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. @@ -3453,6 +3759,7 @@Method Details
"useStemmer": True or False, # Optional. Whether to use stemmer to compute rouge score. }, }, + "metricResourceName": "A String", # Optional. The resource name of the metric definition. "predefinedMetricSpec": { # Specification for a pre-defined metric. # Spec for a pre-defined metric. "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". "parameters": { # Optional. The parameters needed to run the pre-defined metric. @@ -3497,7 +3804,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -3593,6 +3900,7 @@Method Details
}, "metricPromptTemplate": "A String", # Optional. Template for the prompt used by the judge model to evaluate against rubrics. "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics for evaluation using this specification. + "metricResourceName": "A String", # Optional. Resource name of the metric definition. "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. @@ -3616,7 +3924,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -3741,6 +4049,7 @@Method Details
}, }, "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification. + "metricResourceName": "A String", # Optional. Resource name of the metric definition. "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. @@ -3764,7 +4073,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -3880,7 +4189,136 @@Method Details
}, "evaluationSetSnapshot": "A String", # Output only. The specific evaluation set of the evaluation run. For runs with an evaluation set input, this will be that same set. For runs with BigQuery input, it's the sampled BigQuery dataset. "inferenceConfigs": { # Optional. The candidate to inference config map for the evaluation run. The candidate can be up to 128 characters long and can consist of any UTF-8 characters. - "a_key": { # An inference config used for model inference during the evaluation run. + "a_key": { # Defines the configuration for a candidate model or agent being evaluated. `InferenceConfig` encapsulates all the necessary information to invoke or scrape the candidate during the evaluation run. This includes direct model inference parameters, agent execution settings, and multi-turn scraping configurations (such as user simulators). It serves as the primary representation of the candidate across different stages of the evaluation process. + "agentRunConfig": { # Configuration for Agent Run. # Optional. Agent run config. + "agentEngine": "A String", # Optional. The resource name of the Agent Engine. Format: projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine} For example: projects/123/locations/us-central1/reasoningEngines/456 + "sessionInput": { # Session input to run an Agent. # Optional. The session input to get agent running results. + "parameters": { # Optional. Additional parameters for the session, like app_name, etc. For example, {"app_name": "my-app"}. + "a_key": "A String", + }, + "sessionState": { # Optional. Session specific memory which stores key conversation points. + "a_key": "", # Properties of the object. + }, + "userId": "A String", # Optional. The user id for the agent session. The ID can be up to 128 characters long. + }, + "userSimulatorConfig": { # Used for multi-turn agent scraping. Contains configuration for a user simulator that uses an LLM to generate messages on behalf of the user. # The configuration for a user simulator that uses an LLM to generate messages on behalf of the user. + "maxTurn": 42, # Maximum number of invocations allowed by the multi-turn agent scraping. This property allows us to stop a run-off conversation, where the agent and the user simulator get into a never ending loop. The initial fixed prompt is also counted as an invocation. + "modelConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # The configuration for the model. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "modelName": "A String", # The model name to use for multi-turn agent scraping to get next user message, e.g. "gemini-3-flash-preview". + }, + }, "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Generation config. "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. @@ -3902,7 +4340,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -3994,7 +4432,7 @@Method Details
"topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. }, - "model": "A String", # Optional. The fully qualified name of the publisher model or endpoint to use. Anthropic and Llama third-party models are also supported through Model Garden. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Third-party model format: `projects/{project}/locations/{location}/publishers/anthropic/models/{model}` `projects/{project}/locations/{location}/publishers/llama/models/{model}` Endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "model": "A String", # Optional. The fully qualified name of the publisher model or endpoint to use. Anthropic and Llama third-party models are also supported through Model Garden. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Third-party model formats: `projects/{project}/locations/{location}/publishers/anthropic/models/{model}` or `projects/{project}/locations/{location}/publishers/llama/models/{model}` Endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` }, }, "labels": { # Optional. Labels for the evaluation run. @@ -4079,7 +4517,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -4175,6 +4613,7 @@Method Details
}, "datasetCustomMetrics": [ # Optional. Specifications for custom dataset-level aggregations. { # Defines a custom dataset-level aggregation. + "aggregationFunction": "A String", # Required. The Python code string containing the aggregation function. Expected function signature: `def aggregate(instances: list[dict[str, Any]]) -> dict[str, float]:` The `instances` argument is a list of dictionaries, where each dictionary represents a single evaluation result item. The structure of each dictionary corresponds to the fields in the `EvaluationResult` message. This includes: - `"request"`: Contains the original input data and model inputs (from `EvaluationResult.EvaluationRequest`). - `"candidate_results"`: Contains the results of any instance-level metrics (from `EvaluationResult.CandidateResults`). Example of a single item in the `instances` list: { "request": { "prompt": {"text": "What is the capital of France?"}, "golden_response": {"text": "Paris"}, "candidate_responses": [{"candidate": "model-v1", "text": "Paris"}] }, "candidate_results": [ {"metric": "exact_match", "score": 1.0}, {"metric": "bleu", "score": 0.9} ] } "displayName": "A String", # Optional. A display name for this custom summary metric. Used to prefix keys in the output summaryMetrics map. If not provided, a default name like "dataset_custom_metric_1", "dataset_custom_metric_2", etc., will be generated based on the order in the repeated field. }, ], @@ -4213,7 +4652,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -4315,6 +4754,7 @@Method Details
}, }, "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification. + "metricResourceName": "A String", # Optional. Resource name of the metric definition. "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. @@ -4338,7 +4778,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -4488,7 +4928,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -4614,7 +5054,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -4717,6 +5157,18 @@Method Details
"rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. "systemInstruction": "A String", # Optional. System instructions for the judge model. }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. @@ -4745,6 +5197,7 @@Method Details
"useStemmer": True or False, # Optional. Whether to use stemmer to compute rouge score. }, }, + "metricResourceName": "A String", # Optional. The resource name of the metric definition. "predefinedMetricSpec": { # Specification for a pre-defined metric. # Spec for a pre-defined metric. "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". "parameters": { # Optional. The parameters needed to run the pre-defined metric. @@ -4789,7 +5242,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -4885,6 +5338,7 @@Method Details
}, "metricPromptTemplate": "A String", # Optional. Template for the prompt used by the judge model to evaluate against rubrics. "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics for evaluation using this specification. + "metricResourceName": "A String", # Optional. Resource name of the metric definition. "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. @@ -4908,7 +5362,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -5033,6 +5487,7 @@Method Details
}, }, "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification. + "metricResourceName": "A String", # Optional. Resource name of the metric definition. "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. @@ -5056,7 +5511,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -5172,7 +5627,136 @@Method Details
}, "evaluationSetSnapshot": "A String", # Output only. The specific evaluation set of the evaluation run. For runs with an evaluation set input, this will be that same set. For runs with BigQuery input, it's the sampled BigQuery dataset. "inferenceConfigs": { # Optional. The candidate to inference config map for the evaluation run. The candidate can be up to 128 characters long and can consist of any UTF-8 characters. - "a_key": { # An inference config used for model inference during the evaluation run. + "a_key": { # Defines the configuration for a candidate model or agent being evaluated. `InferenceConfig` encapsulates all the necessary information to invoke or scrape the candidate during the evaluation run. This includes direct model inference parameters, agent execution settings, and multi-turn scraping configurations (such as user simulators). It serves as the primary representation of the candidate across different stages of the evaluation process. + "agentRunConfig": { # Configuration for Agent Run. # Optional. Agent run config. + "agentEngine": "A String", # Optional. The resource name of the Agent Engine. Format: projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine} For example: projects/123/locations/us-central1/reasoningEngines/456 + "sessionInput": { # Session input to run an Agent. # Optional. The session input to get agent running results. + "parameters": { # Optional. Additional parameters for the session, like app_name, etc. For example, {"app_name": "my-app"}. + "a_key": "A String", + }, + "sessionState": { # Optional. Session specific memory which stores key conversation points. + "a_key": "", # Properties of the object. + }, + "userId": "A String", # Optional. The user id for the agent session. The ID can be up to 128 characters long. + }, + "userSimulatorConfig": { # Used for multi-turn agent scraping. Contains configuration for a user simulator that uses an LLM to generate messages on behalf of the user. # The configuration for a user simulator that uses an LLM to generate messages on behalf of the user. + "maxTurn": 42, # Maximum number of invocations allowed by the multi-turn agent scraping. This property allows us to stop a run-off conversation, where the agent and the user simulator get into a never ending loop. The initial fixed prompt is also counted as an invocation. + "modelConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # The configuration for the model. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "modelName": "A String", # The model name to use for multi-turn agent scraping to get next user message, e.g. "gemini-3-flash-preview". + }, + }, "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Generation config. "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. @@ -5194,7 +5778,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -5286,7 +5870,7 @@Method Details
"topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. }, - "model": "A String", # Optional. The fully qualified name of the publisher model or endpoint to use. Anthropic and Llama third-party models are also supported through Model Garden. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Third-party model format: `projects/{project}/locations/{location}/publishers/anthropic/models/{model}` `projects/{project}/locations/{location}/publishers/llama/models/{model}` Endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "model": "A String", # Optional. The fully qualified name of the publisher model or endpoint to use. Anthropic and Llama third-party models are also supported through Model Garden. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Third-party model formats: `projects/{project}/locations/{location}/publishers/anthropic/models/{model}` or `projects/{project}/locations/{location}/publishers/llama/models/{model}` Endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` }, }, "labels": { # Optional. Labels for the evaluation run. diff --git a/docs/dyn/aiplatform_v1.projects.locations.html b/docs/dyn/aiplatform_v1.projects.locations.html index 5320f511e8..7d9e2642c3 100644 --- a/docs/dyn/aiplatform_v1.projects.locations.html +++ b/docs/dyn/aiplatform_v1.projects.locations.html @@ -289,6 +289,9 @@Instance Methods
generateSyntheticData(location, body=None, x__xgafv=None)Generates synthetic (artificial) data based on a description
++
+generateUserScenarios(location, body=None, x__xgafv=None)Generates user scenarios for agent evaluation.
Gets information about a location.
@@ -1488,7 +1491,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1640,7 +1643,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1766,7 +1769,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1869,6 +1872,18 @@Method Details
"rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. "systemInstruction": "A String", # Optional. System instructions for the judge model. }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. @@ -1968,7 +1983,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -3781,6 +3796,326 @@Method Details
}, }, "location": "A String", # Required. The resource name of the Location to evaluate the instances. Format: `projects/{project}/locations/{location}` + "metricSources": [ # Optional. The metrics (either inline or registered) used for evaluation. Currently, we only support evaluating a single metric. If multiple metrics are provided, only the first one will be evaluated. + { # The metric source used for evaluation. + "metric": { # The metric used for running evaluations. # Inline metric config. + "aggregationMetrics": [ # Optional. The aggregation metrics to use. + "A String", + ], + "bleuSpec": { # Spec for bleu score metric - calculates the precision of n-grams in the prediction as compared to reference - returns a score ranging between 0 to 1. # Spec for bleu metric. + "useEffectiveOrder": True or False, # Optional. Whether to use_effective_order to compute bleu score. + }, + "computationBasedMetricSpec": { # Specification for a computation based metric. # Spec for a computation based metric. + "parameters": { # Optional. A map of parameters for the metric, e.g. {"rouge_type": "rougeL"}. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the computation based metric. + }, + "customCodeExecutionSpec": { # Specificies a metric that is populated by evaluating user-defined Python code. # Spec for Custom Code Execution metric. + "evaluationFunction": "A String", # Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -> float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name]. Example: Example input: ``` instance= EvaluationInstance( response=EvaluationInstance.InstanceData(text="The answer is 4."), reference=EvaluationInstance.InstanceData(text="4") ) ``` Example converted input: ``` { 'response': {'text': 'The answer is 4.'}, 'reference': {'text': '4'} } ``` Example python function: ``` def evaluate(instance: dict[str, Any]) -> float: if instance'response' == instance'reference': return 1.0 return 0.0 ``` CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function. + }, + "exactMatchSpec": { # Spec for exact match metric - returns 1 if prediction and reference exactly matches, otherwise 0. # Spec for exact match metric. + }, + "llmBasedMetricSpec": { # Specification for an LLM based metric. # Spec for an LLM based metric. + "additionalConfig": { # Optional. Optional additional configuration for the metric. + "a_key": "", # Properties of the object. + }, + "judgeAutoraterConfig": { # The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. # Optional. Optional configuration for the judge LLM (Autorater). + "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "flipEnabled": True or False, # Optional. Default is true. Whether to flip the candidate and baseline responses. This is only applicable to the pairwise metric. If enabled, also provide PairwiseMetricSpec.candidate_response_field_name and PairwiseMetricSpec.baseline_response_field_name. When rendering PairwiseMetricSpec.metric_prompt_template, the candidate and baseline fields will be flipped for half of the samples to reduce bias. + "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. + }, + "metricPromptTemplate": "A String", # Required. Template for the prompt sent to the judge model. + "predefinedRubricGenerationSpec": { # The spec for a pre-defined metric. # Dynamically generate rubrics using a predefined spec. + "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". + "metricSpecParameters": { # Optional. The parameters needed to run the pre-defined metric. + "a_key": "", # Properties of the object. + }, + }, + "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification. + "modelConfig": { # The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. # Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. + "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "flipEnabled": True or False, # Optional. Default is true. Whether to flip the candidate and baseline responses. This is only applicable to the pairwise metric. If enabled, also provide PairwiseMetricSpec.candidate_response_field_name and PairwiseMetricSpec.baseline_response_field_name. When rendering PairwiseMetricSpec.metric_prompt_template, the candidate and baseline fields will be flipped for half of the samples to reduce bias. + "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. + }, + "promptTemplate": "A String", # Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements. + "rubricContentType": "A String", # The type of rubric content to be generated. + "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology. + "A String", + ], + }, + "rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. + "systemInstruction": "A String", # Optional. System instructions for the judge model. + }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, + "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. + "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. + "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. + "customOutputFormatConfig": { # Spec for custom output format configuration. # Optional. CustomOutputFormatConfig allows customization of metric output. When this config is set, the default output is replaced with the raw output string. If a custom format is chosen, the `pairwise_choice` and `explanation` fields in the corresponding metric result will be empty. + "returnRawOutput": True or False, # Optional. Whether to return raw output. + }, + "metricPromptTemplate": "A String", # Required. Metric prompt template for pairwise metric. + "systemInstruction": "A String", # Optional. System instructions for pairwise metric. + }, + "pointwiseMetricSpec": { # Spec for pointwise metric. # Spec for pointwise metric. + "customOutputFormatConfig": { # Spec for custom output format configuration. # Optional. CustomOutputFormatConfig allows customization of metric output. By default, metrics return a score and explanation. When this config is set, the default output is replaced with either: - The raw output string. - A parsed output based on a user-defined schema. If a custom format is chosen, the `score` and `explanation` fields in the corresponding metric result will be empty. + "returnRawOutput": True or False, # Optional. Whether to return raw output. + }, + "metricPromptTemplate": "A String", # Required. Metric prompt template for pointwise metric. + "systemInstruction": "A String", # Optional. System instructions for pointwise metric. + }, + "predefinedMetricSpec": { # The spec for a pre-defined metric. # The spec for a pre-defined metric. + "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". + "metricSpecParameters": { # Optional. The parameters needed to run the pre-defined metric. + "a_key": "", # Properties of the object. + }, + }, + "rougeSpec": { # Spec for rouge score metric - calculates the recall of n-grams in prediction as compared to reference - returns a score ranging between 0 and 1. # Spec for rouge metric. + "rougeType": "A String", # Optional. Supported rouge types are rougen[1-9], rougeL, and rougeLsum. + "splitSummaries": True or False, # Optional. Whether to split summaries while using rougeLsum. + "useStemmer": True or False, # Optional. Whether to use stemmer to compute rouge score. + }, + }, + "metricResourceName": "A String", # Resource name for registered metric. + }, + ], "metrics": [ # The metrics used for evaluation. Currently, we only support evaluating a single metric. If multiple metrics are provided, only the first one will be evaluated. { # The metric used for running evaluations. "aggregationMetrics": [ # Optional. The aggregation metrics to use. @@ -3828,7 +4163,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -3954,7 +4289,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -4057,6 +4392,18 @@Method Details
"rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. "systemInstruction": "A String", # Optional. System instructions for the judge model. }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. @@ -5242,6 +5589,7 @@Method Details
}, ], "location": "A String", # Required. The resource name of the Location to generate rubrics from. Format: `projects/{project}/locations/{location}` + "metricResourceName": "A String", # Optional. The resource name of a registered metric. Rubric generation using predefined metric spec or LLMBasedMetricSpec is supported. If this field is set, the configuration provided in this field is used for rubric generation. The `predefined_rubric_generation_spec` and `rubric_generation_spec` fields will be ignored. "predefinedRubricGenerationSpec": { # The spec for a pre-defined metric. # Optional. Specification for using the rubric generation configs of a pre-defined metric, e.g. "generic_quality_v1" and "instruction_following_v1". Some of the configs may be only used in rubric generation and not supporting evaluation, e.g. "fully_customized_generic_quality_v1". If this field is set, the `rubric_generation_spec` field will be ignored. "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". "metricSpecParameters": { # Optional. The parameters needed to run the pre-defined metric. @@ -5273,7 +5621,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -5600,6 +5948,270 @@Method Details
}
generateUserScenarios(location, body=None, x__xgafv=None)
+ Generates user scenarios for agent evaluation.
+
+Args:
+ location: string, Required. The resource name of the Location to run the job. Format: `projects/{project}/locations/{location}` (required)
+ body: object, The request body.
+ The object takes the form of:
+
+{ # Request message for DataFoundryService.GenerateUserScenarios.
+ "agents": { # Required. A map containing the static configurations for each agent in the system. Key: agent_id (matches the `author` field in events). Value: The static configuration of the agent.
+ "a_key": { # Represents configuration for an Agent.
+ "agentId": "A String", # Required. Unique identifier of the agent. This ID is used to refer to this agent, e.g., in AgentEvent.author, or in the `sub_agents` field. It must be unique within the `agents` map.
+ "agentType": "A String", # Optional. The type or class of the agent (e.g., "LlmAgent", "RouterAgent", "ToolUseAgent"). Useful for the autorater to understand the expected behavior of the agent.
+ "description": "A String", # Optional. A high-level description of the agent's role and responsibilities. Critical for evaluating if the agent is routing tasks correctly.
+ "instruction": "A String", # Optional. Provides instructions for the LLM model, guiding the agent's behavior. Can be static or dynamic. Dynamic instructions can contain placeholders like {variable_name} that will be resolved at runtime using the `AgentEvent.state_delta` field.
+ "subAgents": [ # Optional. The list of valid agent IDs that this agent can delegate to. This defines the directed edges in the multi-agent system graph topology.
+ "A String",
+ ],
+ "tools": [ # Optional. The list of tools available to this agent.
+ { # Tool details that the model may use to generate response. A `Tool` is a piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model. A Tool object should contain exactly one type of Tool (e.g FunctionDeclaration, Retrieval or GoogleSearchRetrieval).
+ "codeExecution": { # Tool that executes code generated by the model, and automatically returns the result to the model. See also ExecutableCode and CodeExecutionResult, which are input and output to this tool. # Optional. CodeExecution tool type. Enables the model to execute code as part of generation.
+ },
+ "computerUse": { # Tool to support computer use. # Optional. Tool to support the model interacting directly with the computer. If enabled, it automatically populates computer-use specific Function Declarations.
+ "environment": "A String", # Required. The environment being operated.
+ "excludedPredefinedFunctions": [ # Optional. By default, [predefined functions](https://cloud.google.com/vertex-ai/generative-ai/docs/computer-use#supported-actions) are included in the final model call. Some of them can be explicitly excluded from being automatically included. This can serve two purposes: 1. Using a more restricted / different action space. 2. Improving the definitions / instructions of predefined functions.
+ "A String",
+ ],
+ },
+ "enterpriseWebSearch": { # Tool to search public web data, powered by Vertex AI Search and Sec4 compliance. # Optional. Tool to support searching public web data, powered by Vertex AI Search and Sec4 compliance.
+ "blockingConfidence": "A String", # Optional. Sites with confidence level chosen & above this value will be blocked from the search results.
+ "excludeDomains": [ # Optional. List of domains to be excluded from the search results. The default limit is 2000 domains.
+ "A String",
+ ],
+ },
+ "functionDeclarations": [ # Optional. Function tool type. One or more function declarations to be passed to the model along with the current user query. Model may decide to call a subset of these functions by populating FunctionCall in the response. User should provide a FunctionResponse for each function call in the next turn. Based on the function responses, Model will generate the final response back to the user. Maximum 512 function declarations can be provided.
+ { # Structured representation of a function declaration as defined by the [OpenAPI 3.0 specification](https://spec.openapis.org/oas/v3.0.3). Included in this declaration are the function name, description, parameters and response type. This FunctionDeclaration is a representation of a block of code that can be used as a `Tool` by the model and executed by the client.
+ "description": "A String", # Optional. Description and purpose of the function. Model uses it to decide how and whether to call the function.
+ "name": "A String", # Required. The name of the function to call. Must start with a letter or an underscore. Must be a-z, A-Z, 0-9, or contain underscores, dots, colons and dashes, with a maximum length of 64.
+ "parameters": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Describes the parameters to this function in JSON Schema Object format. Reflects the Open API 3.03 Parameter Object. string Key: the name of the parameter. Parameter names are case sensitive. Schema Value: the Schema defining the type used for the parameter. For function with no parameters, this can be left unset. Parameter names must start with a letter or an underscore and must only contain chars a-z, A-Z, 0-9, or underscores with a maximum length of 64. Example with 1 required and 1 optional parameter: type: OBJECT properties: param1: type: STRING param2: type: INTEGER required: - param1
+ "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema.
+ "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value to use if the field is not specified.
+ "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt.
+ "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}`
+ "A String",
+ ],
+ "example": "", # Optional. Example of an instance of this schema.
+ "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type.
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array.
+ "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array.
+ "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string.
+ "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided.
+ "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value.
+ "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array.
+ "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string.
+ "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided.
+ "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value.
+ "nullable": True or False, # Optional. Indicates if the value of this field can be null.
+ "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match.
+ "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present.
+ "A String",
+ ],
+ "title": "A String", # Optional. Title for the schema.
+ "type": "A String", # Optional. Data type of the schema field.
+ },
+ "parametersJsonSchema": "", # Optional. Describes the parameters to the function in JSON Schema format. The schema must describe an object where the properties are the parameters to the function. For example: ``` { "type": "object", "properties": { "name": { "type": "string" }, "age": { "type": "integer" } }, "additionalProperties": false, "required": ["name", "age"], "propertyOrdering": ["name", "age"] } ``` This field is mutually exclusive with `parameters`.
+ "response": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Describes the output from this function in JSON Schema format. Reflects the Open API 3.03 Response Object. The Schema defines the type used for the response value of the function.
+ "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema.
+ "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value to use if the field is not specified.
+ "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt.
+ "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}`
+ "A String",
+ ],
+ "example": "", # Optional. Example of an instance of this schema.
+ "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type.
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array.
+ "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array.
+ "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string.
+ "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided.
+ "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value.
+ "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array.
+ "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string.
+ "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided.
+ "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value.
+ "nullable": True or False, # Optional. Indicates if the value of this field can be null.
+ "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match.
+ "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present.
+ "A String",
+ ],
+ "title": "A String", # Optional. Title for the schema.
+ "type": "A String", # Optional. Data type of the schema field.
+ },
+ "responseJsonSchema": "", # Optional. Describes the output from this function in JSON Schema format. The value specified by the schema is the response value of the function. This field is mutually exclusive with `response`.
+ },
+ ],
+ "googleMaps": { # Tool to retrieve public maps data for grounding, powered by Google. # Optional. GoogleMaps tool type. Tool to support Google Maps in Model.
+ "enableWidget": True or False, # Optional. If true, include the widget context token in the response.
+ },
+ "googleSearch": { # GoogleSearch tool type. Tool to support Google Search in Model. Powered by Google. # Optional. GoogleSearch tool type. Tool to support Google Search in Model. Powered by Google.
+ "blockingConfidence": "A String", # Optional. Sites with confidence level chosen & above this value will be blocked from the search results.
+ "excludeDomains": [ # Optional. List of domains to be excluded from the search results. The default limit is 2000 domains. Example: ["amazon.com", "facebook.com"].
+ "A String",
+ ],
+ "searchTypes": { # Different types of search that can be enabled on the GoogleSearch tool. # Optional. The set of search types to enable. If not set, web search is enabled by default.
+ "imageSearch": { # Image search for grounding and related configurations. # Optional. Setting this field enables image search. Image bytes are returned.
+ },
+ "webSearch": { # Standard web search for grounding and related configurations. Only text results are returned. # Optional. Setting this field enables web search. Only text results are returned.
+ },
+ },
+ },
+ "googleSearchRetrieval": { # Tool to retrieve public web data for grounding, powered by Google. # Optional. Specialized retrieval tool that is powered by Google Search.
+ "dynamicRetrievalConfig": { # Describes the options to customize dynamic retrieval. # Specifies the dynamic retrieval configuration for the given source.
+ "dynamicThreshold": 3.14, # Optional. The threshold to be used in dynamic retrieval. If not set, a system default value is used.
+ "mode": "A String", # The mode of the predictor to be used in dynamic retrieval.
+ },
+ },
+ "parallelAiSearch": { # ParallelAiSearch tool type. A tool that uses the Parallel.ai search engine for grounding. # Optional. If specified, Vertex AI will use Parallel.ai to search for information to answer user queries. The search results will be grounded on Parallel.ai and presented to the model for response generation
+ "apiKey": "A String", # Optional. The API key for ParallelAiSearch. If an API key is not provided, the system will attempt to verify access by checking for an active Parallel.ai subscription through the Google Cloud Marketplace. See https://docs.parallel.ai/search/search-quickstart for more details.
+ "customConfigs": { # Optional. Custom configs for ParallelAiSearch. This field can be used to pass any parameter from the Parallel.ai Search API. See the Parallel.ai documentation for the full list of available parameters and their usage: https://docs.parallel.ai/api-reference/search-beta/search Currently only `source_policy`, `excerpts`, `max_results`, `mode`, `fetch_policy` can be set via this field. For example: { "source_policy": { "include_domains": ["google.com", "wikipedia.org"], "exclude_domains": ["example.com"] }, "fetch_policy": { "max_age_seconds": 3600 } }
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "retrieval": { # Defines a retrieval tool that model can call to access external knowledge. # Optional. Retrieval tool type. System will always execute the provided retrieval tool(s) to get external knowledge to answer the prompt. Retrieval results are presented to the model for generation.
+ "disableAttribution": True or False, # Optional. Deprecated. This option is no longer supported.
+ "externalApi": { # Retrieve from data source powered by external API for grounding. The external API is not owned by Google, but need to follow the pre-defined API spec. # Use data source powered by external API for grounding.
+ "apiAuth": { # The generic reusable api auth config. Deprecated. Please use AuthConfig (google/cloud/aiplatform/master/auth.proto) instead. # The authentication config to access the API. Deprecated. Please use auth_config instead.
+ "apiKeyConfig": { # The API secret. # The API secret.
+ "apiKeySecretVersion": "A String", # Required. The SecretManager secret version resource name storing API key. e.g. projects/{project}/secrets/{secret}/versions/{version}
+ "apiKeyString": "A String", # The API key string. Either this or `api_key_secret_version` must be set.
+ },
+ },
+ "apiSpec": "A String", # The API spec that the external API implements.
+ "authConfig": { # Auth configuration to run the extension. # The authentication config to access the API.
+ "apiKeyConfig": { # Config for authentication with API key. # Config for API key auth.
+ "apiKeySecret": "A String", # Optional. The name of the SecretManager secret version resource storing the API key. Format: `projects/{project}/secrets/{secrete}/versions/{version}` - If both `api_key_secret` and `api_key_string` are specified, this field takes precedence over `api_key_string`. - If specified, the `secretmanager.versions.access` permission should be granted to Vertex AI Extension Service Agent (https://cloud.google.com/vertex-ai/docs/general/access-control#service-agents) on the specified resource.
+ "apiKeyString": "A String", # Optional. The API key to be used in the request directly.
+ "httpElementLocation": "A String", # Optional. The location of the API key.
+ "name": "A String", # Optional. The parameter name of the API key. E.g. If the API request is "https://example.com/act?api_key=", "api_key" would be the parameter name.
+ },
+ "authType": "A String", # Type of auth scheme.
+ "googleServiceAccountConfig": { # Config for Google Service Account Authentication. # Config for Google Service Account auth.
+ "serviceAccount": "A String", # Optional. The service account that the extension execution service runs as. - If the service account is specified, the `iam.serviceAccounts.getAccessToken` permission should be granted to Vertex AI Extension Service Agent (https://cloud.google.com/vertex-ai/docs/general/access-control#service-agents) on the specified service account. - If not specified, the Vertex AI Extension Service Agent will be used to execute the Extension.
+ },
+ "httpBasicAuthConfig": { # Config for HTTP Basic Authentication. # Config for HTTP Basic auth.
+ "credentialSecret": "A String", # Required. The name of the SecretManager secret version resource storing the base64 encoded credentials. Format: `projects/{project}/secrets/{secrete}/versions/{version}` - If specified, the `secretmanager.versions.access` permission should be granted to Vertex AI Extension Service Agent (https://cloud.google.com/vertex-ai/docs/general/access-control#service-agents) on the specified resource.
+ },
+ "oauthConfig": { # Config for user oauth. # Config for user oauth.
+ "accessToken": "A String", # Access token for extension endpoint. Only used to propagate token from [[ExecuteExtensionRequest.runtime_auth_config]] at request time.
+ "serviceAccount": "A String", # The service account used to generate access tokens for executing the Extension. - If the service account is specified, the `iam.serviceAccounts.getAccessToken` permission should be granted to Vertex AI Extension Service Agent (https://cloud.google.com/vertex-ai/docs/general/access-control#service-agents) on the provided service account.
+ },
+ "oidcConfig": { # Config for user OIDC auth. # Config for user OIDC auth.
+ "idToken": "A String", # OpenID Connect formatted ID token for extension endpoint. Only used to propagate token from [[ExecuteExtensionRequest.runtime_auth_config]] at request time.
+ "serviceAccount": "A String", # The service account used to generate an OpenID Connect (OIDC)-compatible JWT token signed by the Google OIDC Provider (accounts.google.com) for extension endpoint (https://cloud.google.com/iam/docs/create-short-lived-credentials-direct#sa-credentials-oidc). - The audience for the token will be set to the URL in the server url defined in the OpenApi spec. - If the service account is provided, the service account should grant `iam.serviceAccounts.getOpenIdToken` permission to Vertex AI Extension Service Agent (https://cloud.google.com/vertex-ai/docs/general/access-control#service-agents).
+ },
+ },
+ "elasticSearchParams": { # The search parameters to use for the ELASTIC_SEARCH spec. # Parameters for the elastic search API.
+ "index": "A String", # The ElasticSearch index to use.
+ "numHits": 42, # Optional. Number of hits (chunks) to request. When specified, it is passed to Elasticsearch as the `num_hits` param.
+ "searchTemplate": "A String", # The ElasticSearch search template to use.
+ },
+ "endpoint": "A String", # The endpoint of the external API. The system will call the API at this endpoint to retrieve the data for grounding. Example: https://acme.com:443/search
+ "simpleSearchParams": { # The search parameters to use for SIMPLE_SEARCH spec. # Parameters for the simple search API.
+ },
+ },
+ "vertexAiSearch": { # Retrieve from Vertex AI Search datastore or engine for grounding. datastore and engine are mutually exclusive. See https://cloud.google.com/products/agent-builder # Set to use data source powered by Vertex AI Search.
+ "dataStoreSpecs": [ # Specifications that define the specific DataStores to be searched, along with configurations for those data stores. This is only considered for Engines with multiple data stores. It should only be set if engine is used.
+ { # Define data stores within engine to filter on in a search call and configurations for those data stores. For more information, see https://cloud.google.com/generative-ai-app-builder/docs/reference/rpc/google.cloud.discoveryengine.v1#datastorespec
+ "dataStore": "A String", # Full resource name of DataStore, such as Format: `projects/{project}/locations/{location}/collections/{collection}/dataStores/{dataStore}`
+ "filter": "A String", # Optional. Filter specification to filter documents in the data store specified by data_store field. For more information on filtering, see [Filtering](https://cloud.google.com/generative-ai-app-builder/docs/filter-search-metadata)
+ },
+ ],
+ "datastore": "A String", # Optional. Fully-qualified Vertex AI Search data store resource ID. Format: `projects/{project}/locations/{location}/collections/{collection}/dataStores/{dataStore}`
+ "engine": "A String", # Optional. Fully-qualified Vertex AI Search engine resource ID. Format: `projects/{project}/locations/{location}/collections/{collection}/engines/{engine}`
+ "filter": "A String", # Optional. Filter strings to be passed to the search API.
+ "maxResults": 42, # Optional. Number of search results to return per query. The default value is 10. The maximumm allowed value is 10.
+ },
+ "vertexRagStore": { # Retrieve from Vertex RAG Store for grounding. # Set to use data source powered by Vertex RAG store. User data is uploaded via the VertexRagDataService.
+ "ragResources": [ # Optional. The representation of the rag source. It can be used to specify corpus only or ragfiles. Currently only support one corpus or multiple files from one corpus. In the future we may open up multiple corpora support.
+ { # The definition of the Rag resource.
+ "ragCorpus": "A String", # Optional. RagCorpora resource name. Format: `projects/{project}/locations/{location}/ragCorpora/{rag_corpus}`
+ "ragFileIds": [ # Optional. rag_file_id. The files should be in the same rag_corpus set in rag_corpus field.
+ "A String",
+ ],
+ },
+ ],
+ "ragRetrievalConfig": { # Specifies the context retrieval config. # Optional. The retrieval config for the Rag query.
+ "filter": { # Config for filters. # Optional. Config for filters.
+ "metadataFilter": "A String", # Optional. String for metadata filtering.
+ "vectorDistanceThreshold": 3.14, # Optional. Only returns contexts with vector distance smaller than the threshold.
+ "vectorSimilarityThreshold": 3.14, # Optional. Only returns contexts with vector similarity larger than the threshold.
+ },
+ "ranking": { # Config for ranking and reranking. # Optional. Config for ranking and reranking.
+ "llmRanker": { # Config for LlmRanker. # Optional. Config for LlmRanker.
+ "modelName": "A String", # Optional. The model name used for ranking. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ "rankService": { # Config for Rank Service. # Optional. Config for Rank Service.
+ "modelName": "A String", # Optional. The model name of the rank service. Format: `semantic-ranker-512@latest`
+ },
+ },
+ "topK": 42, # Optional. The number of contexts to retrieve.
+ },
+ "similarityTopK": 42, # Optional. Number of top k results to return from the selected corpora.
+ "vectorDistanceThreshold": 3.14, # Optional. Only return results with vector distance smaller than the threshold.
+ },
+ },
+ "urlContext": { # Tool to support URL context. # Optional. Tool to support URL context retrieval.
+ },
+ },
+ ],
+ },
+ },
+ "rootAgentId": "A String", # Required. The agent id to identify the root agent.
+ "userScenarioGenerationConfig": { # User scenario generation configuration. # Required. Configuration for generating user scenarios.
+ "environmentData": "A String", # Optional. Environment data in string type.
+ "modelName": "A String", # Optional. The model name to use for generation. It can be model name, e.g. "gemini-3-pro-preview". or the fully qualified name of the publisher model or endpoint. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "simulationInstruction": "A String", # Optional. Simulation instruction to guide the user scenario generation.
+ "userScenarioCount": "A String", # Required. The number of user scenarios to generate. The maximum number of scenarios that can be generated is 100.
+ },
+}
+
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # Response message for DataFoundryService.GenerateUserScenarios.
+ "userScenarios": [ # The generated user scenarios used to simulate multi-turn agent running results and agent evaluation.
+ { # Output of user scenario generation.
+ "conversationPlan": "A String", # Conversation plan to drive multi-turn agent run and get simulated agent eval dataset.
+ "startingPrompt": "A String", # Starting prompt for the conversation between simulated user and agent under the test.
+ },
+ ],
+}
+get(name, x__xgafv=None)
Gets information about a location. diff --git a/docs/dyn/aiplatform_v1.projects.locations.models.html b/docs/dyn/aiplatform_v1.projects.locations.models.html index 1ca273ae95..94ce57098f 100644 --- a/docs/dyn/aiplatform_v1.projects.locations.models.html +++ b/docs/dyn/aiplatform_v1.projects.locations.models.html @@ -614,7 +614,7 @@Method Details
"copy": True or False, # If this Model is copy of another Model. If true then source_type pertains to the original. "sourceType": "A String", # Type of the model source. }, - "name": "A String", # The resource name of the Model. + "name": "A String", # Identifier. The resource name of the Model. "originalModelInfo": { # Contains information about the original Model if this Model is a copy. # Output only. If this Model is a copy of another Model, this contains info about the original. "model": "A String", # Output only. The resource name of the Model this Model is a copy of, including the revision. Format: `projects/{project}/locations/{location}/models/{model_id}@{version_id}` }, @@ -997,7 +997,7 @@Method Details
"copy": True or False, # If this Model is copy of another Model. If true then source_type pertains to the original. "sourceType": "A String", # Type of the model source. }, - "name": "A String", # The resource name of the Model. + "name": "A String", # Identifier. The resource name of the Model. "originalModelInfo": { # Contains information about the original Model if this Model is a copy. # Output only. If this Model is a copy of another Model, this contains info about the original. "model": "A String", # Output only. The resource name of the Model this Model is a copy of, including the revision. Format: `projects/{project}/locations/{location}/models/{model_id}@{version_id}` }, @@ -1390,7 +1390,7 @@Method Details
"copy": True or False, # If this Model is copy of another Model. If true then source_type pertains to the original. "sourceType": "A String", # Type of the model source. }, - "name": "A String", # The resource name of the Model. + "name": "A String", # Identifier. The resource name of the Model. "originalModelInfo": { # Contains information about the original Model if this Model is a copy. # Output only. If this Model is a copy of another Model, this contains info about the original. "model": "A String", # Output only. The resource name of the Model this Model is a copy of, including the revision. Format: `projects/{project}/locations/{location}/models/{model_id}@{version_id}` }, @@ -1771,7 +1771,7 @@Method Details
"copy": True or False, # If this Model is copy of another Model. If true then source_type pertains to the original. "sourceType": "A String", # Type of the model source. }, - "name": "A String", # The resource name of the Model. + "name": "A String", # Identifier. The resource name of the Model. "originalModelInfo": { # Contains information about the original Model if this Model is a copy. # Output only. If this Model is a copy of another Model, this contains info about the original. "model": "A String", # Output only. The resource name of the Model this Model is a copy of, including the revision. Format: `projects/{project}/locations/{location}/models/{model_id}@{version_id}` }, @@ -1817,7 +1817,7 @@Method Details
Updates a Model. Args: - name: string, The resource name of the Model. (required) + name: string, Identifier. The resource name of the Model. (required) body: object, The request body. The object takes the form of: @@ -2107,7 +2107,7 @@Method Details
"copy": True or False, # If this Model is copy of another Model. If true then source_type pertains to the original. "sourceType": "A String", # Type of the model source. }, - "name": "A String", # The resource name of the Model. + "name": "A String", # Identifier. The resource name of the Model. "originalModelInfo": { # Contains information about the original Model if this Model is a copy. # Output only. If this Model is a copy of another Model, this contains info about the original. "model": "A String", # Output only. The resource name of the Model this Model is a copy of, including the revision. Format: `projects/{project}/locations/{location}/models/{model_id}@{version_id}` }, @@ -2442,7 +2442,7 @@Method Details
"copy": True or False, # If this Model is copy of another Model. If true then source_type pertains to the original. "sourceType": "A String", # Type of the model source. }, - "name": "A String", # The resource name of the Model. + "name": "A String", # Identifier. The resource name of the Model. "originalModelInfo": { # Contains information about the original Model if this Model is a copy. # Output only. If this Model is a copy of another Model, this contains info about the original. "model": "A String", # Output only. The resource name of the Model this Model is a copy of, including the revision. Format: `projects/{project}/locations/{location}/models/{model_id}@{version_id}` }, @@ -2916,7 +2916,7 @@Method Details
"copy": True or False, # If this Model is copy of another Model. If true then source_type pertains to the original. "sourceType": "A String", # Type of the model source. }, - "name": "A String", # The resource name of the Model. + "name": "A String", # Identifier. The resource name of the Model. "originalModelInfo": { # Contains information about the original Model if this Model is a copy. # Output only. If this Model is a copy of another Model, this contains info about the original. "model": "A String", # Output only. The resource name of the Model this Model is a copy of, including the revision. Format: `projects/{project}/locations/{location}/models/{model_id}@{version_id}` }, diff --git a/docs/dyn/aiplatform_v1.projects.locations.publishers.models.html b/docs/dyn/aiplatform_v1.projects.locations.publishers.models.html index 1c8de8653d..86be02d01c 100644 --- a/docs/dyn/aiplatform_v1.projects.locations.publishers.models.html +++ b/docs/dyn/aiplatform_v1.projects.locations.publishers.models.html @@ -343,7 +343,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1050,7 +1050,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1805,6 +1805,9 @@Method Details
"instances": [ # Required. The instances that are the input to the prediction call. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint's DeployedModels' Model's PredictSchemata's instance_schema_uri. "", ], + "labels": { # Optional. The labels with user-defined metadata for the request. It is used for billing and reporting only. Label keys and values can be no longer than 63 characters (Unicode codepoints) and can only contain lowercase letters, numeric characters, underscores, and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter. + "a_key": "A String", + }, "parameters": "", # Optional. The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint's DeployedModels' Model's PredictSchemata's parameters_schema_uri. } @@ -2173,7 +2176,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], diff --git a/docs/dyn/aiplatform_v1.projects.locations.reasoningEngines.html b/docs/dyn/aiplatform_v1.projects.locations.reasoningEngines.html index a4fbf25e7f..670f68df00 100644 --- a/docs/dyn/aiplatform_v1.projects.locations.reasoningEngines.html +++ b/docs/dyn/aiplatform_v1.projects.locations.reasoningEngines.html @@ -304,6 +304,9 @@Method Details
"a_key": "", # Properties of the object. }, ], + "containerSpec": { # Specification for deploying from a container image. # Deploy from a container image with a defined entrypoint and commands. + "imageUri": "A String", # Required. The Artifact Registry Docker image URI (e.g., us-central1-docker.pkg.dev/my-project/my-repo/my-image:tag) of the container image that is to be run on each worker replica. + }, "deploymentSpec": { # The specification of a Reasoning Engine deployment. # Optional. The specification of a Reasoning Engine deployment. "containerConcurrency": 42, # Optional. Concurrency for each container and agent server. Recommended value: 2 * cpu + 1. Defaults to 9. "env": [ # Optional. Environment variables to be set with the Reasoning Engine deployment. The environment variables can be updated through the UpdateReasoningEngine API. @@ -654,6 +657,9 @@Method Details
"a_key": "", # Properties of the object. }, ], + "containerSpec": { # Specification for deploying from a container image. # Deploy from a container image with a defined entrypoint and commands. + "imageUri": "A String", # Required. The Artifact Registry Docker image URI (e.g., us-central1-docker.pkg.dev/my-project/my-repo/my-image:tag) of the container image that is to be run on each worker replica. + }, "deploymentSpec": { # The specification of a Reasoning Engine deployment. # Optional. The specification of a Reasoning Engine deployment. "containerConcurrency": 42, # Optional. Concurrency for each container and agent server. Recommended value: 2 * cpu + 1. Defaults to 9. "env": [ # Optional. Environment variables to be set with the Reasoning Engine deployment. The environment variables can be updated through the UpdateReasoningEngine API. @@ -935,6 +941,9 @@Method Details
"a_key": "", # Properties of the object. }, ], + "containerSpec": { # Specification for deploying from a container image. # Deploy from a container image with a defined entrypoint and commands. + "imageUri": "A String", # Required. The Artifact Registry Docker image URI (e.g., us-central1-docker.pkg.dev/my-project/my-repo/my-image:tag) of the container image that is to be run on each worker replica. + }, "deploymentSpec": { # The specification of a Reasoning Engine deployment. # Optional. The specification of a Reasoning Engine deployment. "containerConcurrency": 42, # Optional. Concurrency for each container and agent server. Recommended value: 2 * cpu + 1. Defaults to 9. "env": [ # Optional. Environment variables to be set with the Reasoning Engine deployment. The environment variables can be updated through the UpdateReasoningEngine API. @@ -1186,6 +1195,9 @@Method Details
"a_key": "", # Properties of the object. }, ], + "containerSpec": { # Specification for deploying from a container image. # Deploy from a container image with a defined entrypoint and commands. + "imageUri": "A String", # Required. The Artifact Registry Docker image URI (e.g., us-central1-docker.pkg.dev/my-project/my-repo/my-image:tag) of the container image that is to be run on each worker replica. + }, "deploymentSpec": { # The specification of a Reasoning Engine deployment. # Optional. The specification of a Reasoning Engine deployment. "containerConcurrency": 42, # Optional. Concurrency for each container and agent server. Recommended value: 2 * cpu + 1. Defaults to 9. "env": [ # Optional. Environment variables to be set with the Reasoning Engine deployment. The environment variables can be updated through the UpdateReasoningEngine API. diff --git a/docs/dyn/aiplatform_v1.projects.locations.reasoningEngines.sessions.html b/docs/dyn/aiplatform_v1.projects.locations.reasoningEngines.sessions.html index eacfd09d51..cacd4a7a34 100644 --- a/docs/dyn/aiplatform_v1.projects.locations.reasoningEngines.sessions.html +++ b/docs/dyn/aiplatform_v1.projects.locations.reasoningEngines.sessions.html @@ -369,7 +369,7 @@Method Details
"userId": "A String", # Required. Immutable. String id provided by the user } - sessionId: string, Optional. The user defined ID to use for session, which will become the final component of the session resource name. If not provided, Vertex AI will generate a value for this ID. This value may be up to 63 characters, and valid characters are `[a-z0-9-]`. The first character must be a letter, and the last character must be a letter or number. + sessionId: string, Optional. The user defined ID to use for session, which will become the final component of the session resource name. If not provided, Vertex AI will generate a value for this ID. This value may be up to 63 characters, and valid characters are `[a-z0-9-]`. The first and last characters must be a letter or number. x__xgafv: string, V1 error format. Allowed values 1 - v1 error format diff --git a/docs/dyn/aiplatform_v1.projects.locations.trainingPipelines.html b/docs/dyn/aiplatform_v1.projects.locations.trainingPipelines.html index daf238d6da..ecaf90b7d9 100644 --- a/docs/dyn/aiplatform_v1.projects.locations.trainingPipelines.html +++ b/docs/dyn/aiplatform_v1.projects.locations.trainingPipelines.html @@ -483,7 +483,7 @@Method Details
"copy": True or False, # If this Model is copy of another Model. If true then source_type pertains to the original. "sourceType": "A String", # Type of the model source. }, - "name": "A String", # The resource name of the Model. + "name": "A String", # Identifier. The resource name of the Model. "originalModelInfo": { # Contains information about the original Model if this Model is a copy. # Output only. If this Model is a copy of another Model, this contains info about the original. "model": "A String", # Output only. The resource name of the Model this Model is a copy of, including the revision. Format: `projects/{project}/locations/{location}/models/{model_id}@{version_id}` }, @@ -884,7 +884,7 @@Method Details
"copy": True or False, # If this Model is copy of another Model. If true then source_type pertains to the original. "sourceType": "A String", # Type of the model source. }, - "name": "A String", # The resource name of the Model. + "name": "A String", # Identifier. The resource name of the Model. "originalModelInfo": { # Contains information about the original Model if this Model is a copy. # Output only. If this Model is a copy of another Model, this contains info about the original. "model": "A String", # Output only. The resource name of the Model this Model is a copy of, including the revision. Format: `projects/{project}/locations/{location}/models/{model_id}@{version_id}` }, @@ -1327,7 +1327,7 @@Method Details
"copy": True or False, # If this Model is copy of another Model. If true then source_type pertains to the original. "sourceType": "A String", # Type of the model source. }, - "name": "A String", # The resource name of the Model. + "name": "A String", # Identifier. The resource name of the Model. "originalModelInfo": { # Contains information about the original Model if this Model is a copy. # Output only. If this Model is a copy of another Model, this contains info about the original. "model": "A String", # Output only. The resource name of the Model this Model is a copy of, including the revision. Format: `projects/{project}/locations/{location}/models/{model_id}@{version_id}` }, @@ -1742,7 +1742,7 @@Method Details
"copy": True or False, # If this Model is copy of another Model. If true then source_type pertains to the original. "sourceType": "A String", # Type of the model source. }, - "name": "A String", # The resource name of the Model. + "name": "A String", # Identifier. The resource name of the Model. "originalModelInfo": { # Contains information about the original Model if this Model is a copy. # Output only. If this Model is a copy of another Model, this contains info about the original. "model": "A String", # Output only. The resource name of the Model this Model is a copy of, including the revision. Format: `projects/{project}/locations/{location}/models/{model_id}@{version_id}` }, diff --git a/docs/dyn/aiplatform_v1.projects.locations.tuningJobs.html b/docs/dyn/aiplatform_v1.projects.locations.tuningJobs.html index adcff101e5..cf9e872b09 100644 --- a/docs/dyn/aiplatform_v1.projects.locations.tuningJobs.html +++ b/docs/dyn/aiplatform_v1.projects.locations.tuningJobs.html @@ -278,7 +278,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -372,6 +372,12 @@Method Details
}, "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. }, + "datasetCustomMetrics": [ # Optional. Specifications for custom dataset-level aggregations. + { # Defines a custom dataset-level aggregation. + "aggregationFunction": "A String", # Required. The Python code string containing the aggregation function. Expected function signature: `def aggregate(instances: list[dict[str, Any]]) -> dict[str, float]:` The `instances` argument is a list of dictionaries, where each dictionary represents a single evaluation result item. The structure of each dictionary corresponds to the fields in the `EvaluationResult` message. This includes: - `"request"`: Contains the original input data and model inputs (from `EvaluationResult.EvaluationRequest`). - `"candidate_results"`: Contains the results of any instance-level metrics (from `EvaluationResult.CandidateResults`). Example of a single item in the `instances` list: { "request": { "prompt": {"text": "What is the capital of France?"}, "golden_response": {"text": "Paris"}, "candidate_responses": [{"candidate": "model-v1", "text": "Paris"}] }, "candidate_results": [ {"metric": "exact_match", "score": 1.0}, {"metric": "bleu", "score": 0.9} ] } + "displayName": "A String", # Optional. A display name for this custom summary metric. Used to prefix keys in the output summaryMetrics map. If not provided, a default name like "dataset_custom_metric_1", "dataset_custom_metric_2", etc., will be generated based on the order in the repeated field. + }, + ], "inferenceGenerationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for inference generation and outputs. If not set, default generation parameters are used. "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. @@ -393,7 +399,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -532,7 +538,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -658,7 +664,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -761,6 +767,18 @@Method Details
"rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. "systemInstruction": "A String", # Optional. System instructions for the judge model. }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. @@ -1345,7 +1363,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1439,6 +1457,12 @@Method Details
}, "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. }, + "datasetCustomMetrics": [ # Optional. Specifications for custom dataset-level aggregations. + { # Defines a custom dataset-level aggregation. + "aggregationFunction": "A String", # Required. The Python code string containing the aggregation function. Expected function signature: `def aggregate(instances: list[dict[str, Any]]) -> dict[str, float]:` The `instances` argument is a list of dictionaries, where each dictionary represents a single evaluation result item. The structure of each dictionary corresponds to the fields in the `EvaluationResult` message. This includes: - `"request"`: Contains the original input data and model inputs (from `EvaluationResult.EvaluationRequest`). - `"candidate_results"`: Contains the results of any instance-level metrics (from `EvaluationResult.CandidateResults`). Example of a single item in the `instances` list: { "request": { "prompt": {"text": "What is the capital of France?"}, "golden_response": {"text": "Paris"}, "candidate_responses": [{"candidate": "model-v1", "text": "Paris"}] }, "candidate_results": [ {"metric": "exact_match", "score": 1.0}, {"metric": "bleu", "score": 0.9} ] } + "displayName": "A String", # Optional. A display name for this custom summary metric. Used to prefix keys in the output summaryMetrics map. If not provided, a default name like "dataset_custom_metric_1", "dataset_custom_metric_2", etc., will be generated based on the order in the repeated field. + }, + ], "inferenceGenerationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for inference generation and outputs. If not set, default generation parameters are used. "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. @@ -1460,7 +1484,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1599,7 +1623,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1725,7 +1749,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1828,6 +1852,18 @@Method Details
"rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. "systemInstruction": "A String", # Optional. System instructions for the judge model. }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. @@ -2419,7 +2455,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2513,6 +2549,12 @@Method Details
}, "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. }, + "datasetCustomMetrics": [ # Optional. Specifications for custom dataset-level aggregations. + { # Defines a custom dataset-level aggregation. + "aggregationFunction": "A String", # Required. The Python code string containing the aggregation function. Expected function signature: `def aggregate(instances: list[dict[str, Any]]) -> dict[str, float]:` The `instances` argument is a list of dictionaries, where each dictionary represents a single evaluation result item. The structure of each dictionary corresponds to the fields in the `EvaluationResult` message. This includes: - `"request"`: Contains the original input data and model inputs (from `EvaluationResult.EvaluationRequest`). - `"candidate_results"`: Contains the results of any instance-level metrics (from `EvaluationResult.CandidateResults`). Example of a single item in the `instances` list: { "request": { "prompt": {"text": "What is the capital of France?"}, "golden_response": {"text": "Paris"}, "candidate_responses": [{"candidate": "model-v1", "text": "Paris"}] }, "candidate_results": [ {"metric": "exact_match", "score": 1.0}, {"metric": "bleu", "score": 0.9} ] } + "displayName": "A String", # Optional. A display name for this custom summary metric. Used to prefix keys in the output summaryMetrics map. If not provided, a default name like "dataset_custom_metric_1", "dataset_custom_metric_2", etc., will be generated based on the order in the repeated field. + }, + ], "inferenceGenerationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for inference generation and outputs. If not set, default generation parameters are used. "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. @@ -2534,7 +2576,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2673,7 +2715,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2799,7 +2841,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2902,6 +2944,18 @@Method Details
"rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. "systemInstruction": "A String", # Optional. System instructions for the judge model. }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. @@ -3499,7 +3553,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -3593,6 +3647,12 @@Method Details
}, "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. }, + "datasetCustomMetrics": [ # Optional. Specifications for custom dataset-level aggregations. + { # Defines a custom dataset-level aggregation. + "aggregationFunction": "A String", # Required. The Python code string containing the aggregation function. Expected function signature: `def aggregate(instances: list[dict[str, Any]]) -> dict[str, float]:` The `instances` argument is a list of dictionaries, where each dictionary represents a single evaluation result item. The structure of each dictionary corresponds to the fields in the `EvaluationResult` message. This includes: - `"request"`: Contains the original input data and model inputs (from `EvaluationResult.EvaluationRequest`). - `"candidate_results"`: Contains the results of any instance-level metrics (from `EvaluationResult.CandidateResults`). Example of a single item in the `instances` list: { "request": { "prompt": {"text": "What is the capital of France?"}, "golden_response": {"text": "Paris"}, "candidate_responses": [{"candidate": "model-v1", "text": "Paris"}] }, "candidate_results": [ {"metric": "exact_match", "score": 1.0}, {"metric": "bleu", "score": 0.9} ] } + "displayName": "A String", # Optional. A display name for this custom summary metric. Used to prefix keys in the output summaryMetrics map. If not provided, a default name like "dataset_custom_metric_1", "dataset_custom_metric_2", etc., will be generated based on the order in the repeated field. + }, + ], "inferenceGenerationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for inference generation and outputs. If not set, default generation parameters are used. "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. @@ -3614,7 +3674,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -3753,7 +3813,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -3879,7 +3939,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -3982,6 +4042,18 @@Method Details
"rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. "systemInstruction": "A String", # Optional. System instructions for the judge model. }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. @@ -4594,7 +4666,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -4688,6 +4760,12 @@Method Details
}, "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. }, + "datasetCustomMetrics": [ # Optional. Specifications for custom dataset-level aggregations. + { # Defines a custom dataset-level aggregation. + "aggregationFunction": "A String", # Required. The Python code string containing the aggregation function. Expected function signature: `def aggregate(instances: list[dict[str, Any]]) -> dict[str, float]:` The `instances` argument is a list of dictionaries, where each dictionary represents a single evaluation result item. The structure of each dictionary corresponds to the fields in the `EvaluationResult` message. This includes: - `"request"`: Contains the original input data and model inputs (from `EvaluationResult.EvaluationRequest`). - `"candidate_results"`: Contains the results of any instance-level metrics (from `EvaluationResult.CandidateResults`). Example of a single item in the `instances` list: { "request": { "prompt": {"text": "What is the capital of France?"}, "golden_response": {"text": "Paris"}, "candidate_responses": [{"candidate": "model-v1", "text": "Paris"}] }, "candidate_results": [ {"metric": "exact_match", "score": 1.0}, {"metric": "bleu", "score": 0.9} ] } + "displayName": "A String", # Optional. A display name for this custom summary metric. Used to prefix keys in the output summaryMetrics map. If not provided, a default name like "dataset_custom_metric_1", "dataset_custom_metric_2", etc., will be generated based on the order in the repeated field. + }, + ], "inferenceGenerationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for inference generation and outputs. If not set, default generation parameters are used. "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. @@ -4709,7 +4787,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -4848,7 +4926,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -4974,7 +5052,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -5077,6 +5155,18 @@Method Details
"rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. "systemInstruction": "A String", # Optional. System instructions for the judge model. }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. diff --git a/docs/dyn/aiplatform_v1.publishers.models.html b/docs/dyn/aiplatform_v1.publishers.models.html index 9ffabc74e8..4c289f7d3d 100644 --- a/docs/dyn/aiplatform_v1.publishers.models.html +++ b/docs/dyn/aiplatform_v1.publishers.models.html @@ -329,7 +329,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -887,7 +887,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2217,6 +2217,9 @@Method Details
"instances": [ # Required. The instances that are the input to the prediction call. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint's DeployedModels' Model's PredictSchemata's instance_schema_uri. "", ], + "labels": { # Optional. The labels with user-defined metadata for the request. It is used for billing and reporting only. Label keys and values can be no longer than 63 characters (Unicode codepoints) and can only contain lowercase letters, numeric characters, underscores, and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter. + "a_key": "A String", + }, "parameters": "", # Optional. The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint's DeployedModels' Model's PredictSchemata's parameters_schema_uri. } @@ -2357,7 +2360,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], diff --git a/docs/dyn/aiplatform_v1.reasoningEngines.html b/docs/dyn/aiplatform_v1.reasoningEngines.html index af6526b26d..06d8994179 100644 --- a/docs/dyn/aiplatform_v1.reasoningEngines.html +++ b/docs/dyn/aiplatform_v1.reasoningEngines.html @@ -294,6 +294,9 @@Method Details
"a_key": "", # Properties of the object. }, ], + "containerSpec": { # Specification for deploying from a container image. # Deploy from a container image with a defined entrypoint and commands. + "imageUri": "A String", # Required. The Artifact Registry Docker image URI (e.g., us-central1-docker.pkg.dev/my-project/my-repo/my-image:tag) of the container image that is to be run on each worker replica. + }, "deploymentSpec": { # The specification of a Reasoning Engine deployment. # Optional. The specification of a Reasoning Engine deployment. "containerConcurrency": 42, # Optional. Concurrency for each container and agent server. Recommended value: 2 * cpu + 1. Defaults to 9. "env": [ # Optional. Environment variables to be set with the Reasoning Engine deployment. The environment variables can be updated through the UpdateReasoningEngine API. @@ -645,6 +648,9 @@Method Details
"a_key": "", # Properties of the object. }, ], + "containerSpec": { # Specification for deploying from a container image. # Deploy from a container image with a defined entrypoint and commands. + "imageUri": "A String", # Required. The Artifact Registry Docker image URI (e.g., us-central1-docker.pkg.dev/my-project/my-repo/my-image:tag) of the container image that is to be run on each worker replica. + }, "deploymentSpec": { # The specification of a Reasoning Engine deployment. # Optional. The specification of a Reasoning Engine deployment. "containerConcurrency": 42, # Optional. Concurrency for each container and agent server. Recommended value: 2 * cpu + 1. Defaults to 9. "env": [ # Optional. Environment variables to be set with the Reasoning Engine deployment. The environment variables can be updated through the UpdateReasoningEngine API. @@ -891,6 +897,9 @@Method Details
"a_key": "", # Properties of the object. }, ], + "containerSpec": { # Specification for deploying from a container image. # Deploy from a container image with a defined entrypoint and commands. + "imageUri": "A String", # Required. The Artifact Registry Docker image URI (e.g., us-central1-docker.pkg.dev/my-project/my-repo/my-image:tag) of the container image that is to be run on each worker replica. + }, "deploymentSpec": { # The specification of a Reasoning Engine deployment. # Optional. The specification of a Reasoning Engine deployment. "containerConcurrency": 42, # Optional. Concurrency for each container and agent server. Recommended value: 2 * cpu + 1. Defaults to 9. "env": [ # Optional. Environment variables to be set with the Reasoning Engine deployment. The environment variables can be updated through the UpdateReasoningEngine API. @@ -1142,6 +1151,9 @@Method Details
"a_key": "", # Properties of the object. }, ], + "containerSpec": { # Specification for deploying from a container image. # Deploy from a container image with a defined entrypoint and commands. + "imageUri": "A String", # Required. The Artifact Registry Docker image URI (e.g., us-central1-docker.pkg.dev/my-project/my-repo/my-image:tag) of the container image that is to be run on each worker replica. + }, "deploymentSpec": { # The specification of a Reasoning Engine deployment. # Optional. The specification of a Reasoning Engine deployment. "containerConcurrency": 42, # Optional. Concurrency for each container and agent server. Recommended value: 2 * cpu + 1. Defaults to 9. "env": [ # Optional. Environment variables to be set with the Reasoning Engine deployment. The environment variables can be updated through the UpdateReasoningEngine API. diff --git a/docs/dyn/aiplatform_v1.v1.html b/docs/dyn/aiplatform_v1.v1.html index 74c3636d1c..c096c92bdc 100644 --- a/docs/dyn/aiplatform_v1.v1.html +++ b/docs/dyn/aiplatform_v1.v1.html @@ -125,7 +125,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -277,7 +277,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -403,7 +403,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -506,6 +506,18 @@Method Details
"rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. "systemInstruction": "A String", # Optional. System instructions for the judge model. }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. @@ -604,7 +616,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2417,6 +2429,326 @@Method Details
}, }, "location": "A String", # Required. The resource name of the Location to evaluate the instances. Format: `projects/{project}/locations/{location}` + "metricSources": [ # Optional. The metrics (either inline or registered) used for evaluation. Currently, we only support evaluating a single metric. If multiple metrics are provided, only the first one will be evaluated. + { # The metric source used for evaluation. + "metric": { # The metric used for running evaluations. # Inline metric config. + "aggregationMetrics": [ # Optional. The aggregation metrics to use. + "A String", + ], + "bleuSpec": { # Spec for bleu score metric - calculates the precision of n-grams in the prediction as compared to reference - returns a score ranging between 0 to 1. # Spec for bleu metric. + "useEffectiveOrder": True or False, # Optional. Whether to use_effective_order to compute bleu score. + }, + "computationBasedMetricSpec": { # Specification for a computation based metric. # Spec for a computation based metric. + "parameters": { # Optional. A map of parameters for the metric, e.g. {"rouge_type": "rougeL"}. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the computation based metric. + }, + "customCodeExecutionSpec": { # Specificies a metric that is populated by evaluating user-defined Python code. # Spec for Custom Code Execution metric. + "evaluationFunction": "A String", # Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -> float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name]. Example: Example input: ``` instance= EvaluationInstance( response=EvaluationInstance.InstanceData(text="The answer is 4."), reference=EvaluationInstance.InstanceData(text="4") ) ``` Example converted input: ``` { 'response': {'text': 'The answer is 4.'}, 'reference': {'text': '4'} } ``` Example python function: ``` def evaluate(instance: dict[str, Any]) -> float: if instance'response' == instance'reference': return 1.0 return 0.0 ``` CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function. + }, + "exactMatchSpec": { # Spec for exact match metric - returns 1 if prediction and reference exactly matches, otherwise 0. # Spec for exact match metric. + }, + "llmBasedMetricSpec": { # Specification for an LLM based metric. # Spec for an LLM based metric. + "additionalConfig": { # Optional. Optional additional configuration for the metric. + "a_key": "", # Properties of the object. + }, + "judgeAutoraterConfig": { # The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. # Optional. Optional configuration for the judge LLM (Autorater). + "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "flipEnabled": True or False, # Optional. Default is true. Whether to flip the candidate and baseline responses. This is only applicable to the pairwise metric. If enabled, also provide PairwiseMetricSpec.candidate_response_field_name and PairwiseMetricSpec.baseline_response_field_name. When rendering PairwiseMetricSpec.metric_prompt_template, the candidate and baseline fields will be flipped for half of the samples to reduce bias. + "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. + }, + "metricPromptTemplate": "A String", # Required. Template for the prompt sent to the judge model. + "predefinedRubricGenerationSpec": { # The spec for a pre-defined metric. # Dynamically generate rubrics using a predefined spec. + "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". + "metricSpecParameters": { # Optional. The parameters needed to run the pre-defined metric. + "a_key": "", # Properties of the object. + }, + }, + "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification. + "modelConfig": { # The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. # Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. + "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "flipEnabled": True or False, # Optional. Default is true. Whether to flip the candidate and baseline responses. This is only applicable to the pairwise metric. If enabled, also provide PairwiseMetricSpec.candidate_response_field_name and PairwiseMetricSpec.baseline_response_field_name. When rendering PairwiseMetricSpec.metric_prompt_template, the candidate and baseline fields will be flipped for half of the samples to reduce bias. + "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. + }, + "promptTemplate": "A String", # Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements. + "rubricContentType": "A String", # The type of rubric content to be generated. + "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology. + "A String", + ], + }, + "rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. + "systemInstruction": "A String", # Optional. System instructions for the judge model. + }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, + "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. + "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. + "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. + "customOutputFormatConfig": { # Spec for custom output format configuration. # Optional. CustomOutputFormatConfig allows customization of metric output. When this config is set, the default output is replaced with the raw output string. If a custom format is chosen, the `pairwise_choice` and `explanation` fields in the corresponding metric result will be empty. + "returnRawOutput": True or False, # Optional. Whether to return raw output. + }, + "metricPromptTemplate": "A String", # Required. Metric prompt template for pairwise metric. + "systemInstruction": "A String", # Optional. System instructions for pairwise metric. + }, + "pointwiseMetricSpec": { # Spec for pointwise metric. # Spec for pointwise metric. + "customOutputFormatConfig": { # Spec for custom output format configuration. # Optional. CustomOutputFormatConfig allows customization of metric output. By default, metrics return a score and explanation. When this config is set, the default output is replaced with either: - The raw output string. - A parsed output based on a user-defined schema. If a custom format is chosen, the `score` and `explanation` fields in the corresponding metric result will be empty. + "returnRawOutput": True or False, # Optional. Whether to return raw output. + }, + "metricPromptTemplate": "A String", # Required. Metric prompt template for pointwise metric. + "systemInstruction": "A String", # Optional. System instructions for pointwise metric. + }, + "predefinedMetricSpec": { # The spec for a pre-defined metric. # The spec for a pre-defined metric. + "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". + "metricSpecParameters": { # Optional. The parameters needed to run the pre-defined metric. + "a_key": "", # Properties of the object. + }, + }, + "rougeSpec": { # Spec for rouge score metric - calculates the recall of n-grams in prediction as compared to reference - returns a score ranging between 0 and 1. # Spec for rouge metric. + "rougeType": "A String", # Optional. Supported rouge types are rougen[1-9], rougeL, and rougeLsum. + "splitSummaries": True or False, # Optional. Whether to split summaries while using rougeLsum. + "useStemmer": True or False, # Optional. Whether to use stemmer to compute rouge score. + }, + }, + "metricResourceName": "A String", # Resource name for registered metric. + }, + ], "metrics": [ # The metrics used for evaluation. Currently, we only support evaluating a single metric. If multiple metrics are provided, only the first one will be evaluated. { # The metric used for running evaluations. "aggregationMetrics": [ # Optional. The aggregation metrics to use. @@ -2464,7 +2796,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2590,7 +2922,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2693,6 +3025,18 @@Method Details
"rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. "systemInstruction": "A String", # Optional. System instructions for the judge model. }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. @@ -3877,6 +4221,7 @@Method Details
}, ], "location": "A String", # Required. The resource name of the Location to generate rubrics from. Format: `projects/{project}/locations/{location}` + "metricResourceName": "A String", # Optional. The resource name of a registered metric. Rubric generation using predefined metric spec or LLMBasedMetricSpec is supported. If this field is set, the configuration provided in this field is used for rubric generation. The `predefined_rubric_generation_spec` and `rubric_generation_spec` fields will be ignored. "predefinedRubricGenerationSpec": { # The spec for a pre-defined metric. # Optional. Specification for using the rubric generation configs of a pre-defined metric, e.g. "generic_quality_v1" and "instruction_following_v1". Some of the configs may be only used in rubric generation and not supporting evaluation, e.g. "fully_customized_generic_quality_v1". If this field is set, the `rubric_generation_spec` field will be ignored. "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". "metricSpecParameters": { # Optional. The parameters needed to run the pre-defined metric. @@ -3908,7 +4253,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], diff --git a/docs/dyn/aiplatform_v1beta1.endpoints.html b/docs/dyn/aiplatform_v1beta1.endpoints.html index 0186be11f6..9f71a1354a 100644 --- a/docs/dyn/aiplatform_v1beta1.endpoints.html +++ b/docs/dyn/aiplatform_v1beta1.endpoints.html @@ -343,7 +343,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -915,7 +915,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -1684,6 +1684,9 @@Method Details
"instances": [ # Required. The instances that are the input to the prediction call. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint's DeployedModels' Model's PredictSchemata's instance_schema_uri. "", ], + "labels": { # Optional. The labels with user-defined metadata for the request. It is used for billing and reporting only. Label keys and values can be no longer than 63 characters (Unicode codepoints) and can only contain lowercase letters, numeric characters, underscores, and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter. + "a_key": "A String", + }, "parameters": "", # Optional. The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint's DeployedModels' Model's PredictSchemata's parameters_schema_uri. } @@ -1829,7 +1832,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], diff --git a/docs/dyn/aiplatform_v1beta1.projects.locations.datasets.html b/docs/dyn/aiplatform_v1beta1.projects.locations.datasets.html index 22c755770a..befb789231 100644 --- a/docs/dyn/aiplatform_v1beta1.projects.locations.datasets.html +++ b/docs/dyn/aiplatform_v1beta1.projects.locations.datasets.html @@ -268,7 +268,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -850,7 +850,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], diff --git a/docs/dyn/aiplatform_v1beta1.projects.locations.endpoints.html b/docs/dyn/aiplatform_v1beta1.projects.locations.endpoints.html index b94ca052b4..14be01375a 100644 --- a/docs/dyn/aiplatform_v1beta1.projects.locations.endpoints.html +++ b/docs/dyn/aiplatform_v1beta1.projects.locations.endpoints.html @@ -420,7 +420,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -2082,7 +2082,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], @@ -4310,6 +4310,9 @@Method Details
"instances": [ # Required. The instances that are the input to the prediction call. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint's DeployedModels' Model's PredictSchemata's instance_schema_uri. "", ], + "labels": { # Optional. The labels with user-defined metadata for the request. It is used for billing and reporting only. Label keys and values can be no longer than 63 characters (Unicode codepoints) and can only contain lowercase letters, numeric characters, underscores, and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter. + "a_key": "A String", + }, "parameters": "", # Optional. The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint's DeployedModels' Model's PredictSchemata's parameters_schema_uri. } @@ -4741,7 +4744,7 @@Method Details
"presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. - "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. "A String", ], diff --git a/docs/dyn/aiplatform_v1beta1.projects.locations.evaluationItems.html b/docs/dyn/aiplatform_v1beta1.projects.locations.evaluationItems.html index dd7657d794..dc77d7a7a4 100644 --- a/docs/dyn/aiplatform_v1beta1.projects.locations.evaluationItems.html +++ b/docs/dyn/aiplatform_v1beta1.projects.locations.evaluationItems.html @@ -2044,6 +2044,10 @@Method Details
}, }, "text": "A String", # Text prompt. + "userScenario": { # User scenario to help simulate multi-turn agent running results. # Optional. The generated user scenario used to drive multi-turn agent running results. + "conversationPlan": "A String", # Required. The plan for the conversation, used to drive the multi-turn agent run and generate the simulated agent evaluation dataset. + "startingPrompt": "A String", # Required. The prompt that starts the conversation between the simulated user and the agent under test. + }, "value": "", # Fields and values that can be used to populate the prompt template. }, "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group. @@ -4014,6 +4018,10 @@Method Details
}, }, "text": "A String", # Text prompt. + "userScenario": { # User scenario to help simulate multi-turn agent running results. # Optional. The generated user scenario used to drive multi-turn agent running results. + "conversationPlan": "A String", # Required. The plan for the conversation, used to drive the multi-turn agent run and generate the simulated agent evaluation dataset. + "startingPrompt": "A String", # Required. The prompt that starts the conversation between the simulated user and the agent under test. + }, "value": "", # Fields and values that can be used to populate the prompt template. }, "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group. @@ -5984,6 +5992,10 @@Method Details
}, }, "text": "A String", # Text prompt. + "userScenario": { # User scenario to help simulate multi-turn agent running results. # Optional. The generated user scenario used to drive multi-turn agent running results. + "conversationPlan": "A String", # Required. The plan for the conversation, used to drive the multi-turn agent run and generate the simulated agent evaluation dataset. + "startingPrompt": "A String", # Required. The prompt that starts the conversation between the simulated user and the agent under test. + }, "value": "", # Fields and values that can be used to populate the prompt template. }, "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group. @@ -7954,6 +7966,10 @@Method Details
}, }, "text": "A String", # Text prompt. + "userScenario": { # User scenario to help simulate multi-turn agent running results. # Optional. The generated user scenario used to drive multi-turn agent running results. + "conversationPlan": "A String", # Required. The plan for the conversation, used to drive the multi-turn agent run and generate the simulated agent evaluation dataset. + "startingPrompt": "A String", # Required. The prompt that starts the conversation between the simulated user and the agent under test. + }, "value": "", # Fields and values that can be used to populate the prompt template. }, "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group. @@ -9966,6 +9982,10 @@Method Details
}, }, "text": "A String", # Text prompt. + "userScenario": { # User scenario to help simulate multi-turn agent running results. # Optional. The generated user scenario used to drive multi-turn agent running results. + "conversationPlan": "A String", # Required. The plan for the conversation, used to drive the multi-turn agent run and generate the simulated agent evaluation dataset. + "startingPrompt": "A String", # Required. The prompt that starts the conversation between the simulated user and the agent under test. + }, "value": "", # Fields and values that can be used to populate the prompt template. }, "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group. @@ -11936,6 +11956,10 @@Method Details
}, }, "text": "A String", # Text prompt. + "userScenario": { # User scenario to help simulate multi-turn agent running results. # Optional. The generated user scenario used to drive multi-turn agent running results. + "conversationPlan": "A String", # Required. The plan for the conversation, used to drive the multi-turn agent run and generate the simulated agent evaluation dataset. + "startingPrompt": "A String", # Required. The prompt that starts the conversation between the simulated user and the agent under test. + }, "value": "", # Fields and values that can be used to populate the prompt template. }, "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group. @@ -13919,6 +13943,10 @@Method Details
}, }, "text": "A String", # Text prompt. + "userScenario": { # User scenario to help simulate multi-turn agent running results. # Optional. The generated user scenario used to drive multi-turn agent running results. + "conversationPlan": "A String", # Required. The plan for the conversation, used to drive the multi-turn agent run and generate the simulated agent evaluation dataset. + "startingPrompt": "A String", # Required. The prompt that starts the conversation between the simulated user and the agent under test. + }, "value": "", # Fields and values that can be used to populate the prompt template. }, "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group. @@ -15889,6 +15917,10 @@Method Details
}, }, "text": "A String", # Text prompt. + "userScenario": { # User scenario to help simulate multi-turn agent running results. # Optional. The generated user scenario used to drive multi-turn agent running results. + "conversationPlan": "A String", # Required. The plan for the conversation, used to drive the multi-turn agent run and generate the simulated agent evaluation dataset. + "startingPrompt": "A String", # Required. The prompt that starts the conversation between the simulated user and the agent under test. + }, "value": "", # Fields and values that can be used to populate the prompt template. }, "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group. diff --git a/docs/dyn/aiplatform_v1beta1.projects.locations.evaluationMetrics.html b/docs/dyn/aiplatform_v1beta1.projects.locations.evaluationMetrics.html index 831f99dacd..d4c1d6efb8 100644 --- a/docs/dyn/aiplatform_v1beta1.projects.locations.evaluationMetrics.html +++ b/docs/dyn/aiplatform_v1beta1.projects.locations.evaluationMetrics.html @@ -82,10 +82,1464 @@Instance Methods
Close httplib2 connections.
++
+create(parent, body=None, evaluationMetricId=None, x__xgafv=None)Creates an EvaluationMetric.
+ +Deletes an EvaluationMetric.
+ +Gets an EvaluationMetric.
++
+list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)Lists EvaluationMetrics.
+ +Retrieves the next page of results.
Method Details
+close()Close httplib2 connections.++ +create(parent, body=None, evaluationMetricId=None, x__xgafv=None)+Creates an EvaluationMetric. + +Args: + parent: string, Required. The resource name of the Location to create the EvaluationMetric in. Format: `projects/{project}/locations/{location}` (required) + body: object, The request body. + The object takes the form of: + +{ # EvaluationMetric is a resource that represents a reusable metric configuration. + "createTime": "A String", # Output only. The time when the EvaluationMetric was created. + "description": "A String", # Optional. A description of the EvaluationMetric. + "displayName": "A String", # Required. The user-friendly display name for the EvaluationMetric. + "gcsUri": "A String", # Optional. The Google Cloud Storage URI that stores the metric specification.. + "labels": { # Optional. Labels for the evaluation metric. + "a_key": "A String", + }, + "metric": { # The metric used for running evaluations. # Optional. The metric configuration. + "aggregationMetrics": [ # Optional. The aggregation metrics to use. + "A String", + ], + "bleuSpec": { # Spec for bleu score metric - calculates the precision of n-grams in the prediction as compared to reference - returns a score ranging between 0 to 1. # Spec for bleu metric. + "useEffectiveOrder": True or False, # Optional. Whether to use_effective_order to compute bleu score. + }, + "computationBasedMetricSpec": { # Specification for a computation based metric. # Spec for a computation based metric. + "parameters": { # Optional. A map of parameters for the metric, e.g. {"rouge_type": "rougeL"}. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the computation based metric. + }, + "customCodeExecutionSpec": { # Specificies a metric that is populated by evaluating user-defined Python code. # Spec for Custom Code Execution metric. + "evaluationFunction": "A String", # Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -> float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name]. Example: Example input: ``` instance= EvaluationInstance( response=EvaluationInstance.InstanceData(text="The answer is 4."), reference=EvaluationInstance.InstanceData(text="4") ) ``` Example converted input: ``` { 'response': {'text': 'The answer is 4.'}, 'reference': {'text': '4'} } ``` Example python function: ``` def evaluate(instance: dict[str, Any]) -> float: if instance'response' == instance'reference': return 1.0 return 0.0 ``` CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function. + }, + "exactMatchSpec": { # Spec for exact match metric - returns 1 if prediction and reference exactly matches, otherwise 0. # Spec for exact match metric. + }, + "llmBasedMetricSpec": { # Specification for an LLM based metric. # Spec for an LLM based metric. + "additionalConfig": { # Optional. Optional additional configuration for the metric. + "a_key": "", # Properties of the object. + }, + "judgeAutoraterConfig": { # The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. # Optional. Optional configuration for the judge LLM (Autorater). + "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "flipEnabled": True or False, # Optional. Default is true. Whether to flip the candidate and baseline responses. This is only applicable to the pairwise metric. If enabled, also provide PairwiseMetricSpec.candidate_response_field_name and PairwiseMetricSpec.baseline_response_field_name. When rendering PairwiseMetricSpec.metric_prompt_template, the candidate and baseline fields will be flipped for half of the samples to reduce bias. + "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "modelConfig": { # Config for model selection. # Optional. Config for model selection. + "featureSelectionPreference": "A String", # Required. Feature selection preference. + }, + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1beta1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1beta1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. + }, + "metricPromptTemplate": "A String", # Required. Template for the prompt sent to the judge model. + "predefinedRubricGenerationSpec": { # The spec for a pre-defined metric. # Dynamically generate rubrics using a predefined spec. + "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". + "metricSpecParameters": { # Optional. The parameters needed to run the pre-defined metric. + "a_key": "", # Properties of the object. + }, + }, + "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification. + "modelConfig": { # The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. # Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. + "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "flipEnabled": True or False, # Optional. Default is true. Whether to flip the candidate and baseline responses. This is only applicable to the pairwise metric. If enabled, also provide PairwiseMetricSpec.candidate_response_field_name and PairwiseMetricSpec.baseline_response_field_name. When rendering PairwiseMetricSpec.metric_prompt_template, the candidate and baseline fields will be flipped for half of the samples to reduce bias. + "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "modelConfig": { # Config for model selection. # Optional. Config for model selection. + "featureSelectionPreference": "A String", # Required. Feature selection preference. + }, + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1beta1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1beta1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. + }, + "promptTemplate": "A String", # Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements. + "rubricContentType": "A String", # The type of rubric content to be generated. + "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology. + "A String", + ], + }, + "rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. + "systemInstruction": "A String", # Optional. System instructions for the judge model. + }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, + "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. + "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. + "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. + "customOutputFormatConfig": { # Spec for custom output format configuration. # Optional. CustomOutputFormatConfig allows customization of metric output. When this config is set, the default output is replaced with the raw output string. If a custom format is chosen, the `pairwise_choice` and `explanation` fields in the corresponding metric result will be empty. + "returnRawOutput": True or False, # Optional. Whether to return raw output. + }, + "metricPromptTemplate": "A String", # Required. Metric prompt template for pairwise metric. + "systemInstruction": "A String", # Optional. System instructions for pairwise metric. + }, + "pointwiseMetricSpec": { # Spec for pointwise metric. # Spec for pointwise metric. + "customOutputFormatConfig": { # Spec for custom output format configuration. # Optional. CustomOutputFormatConfig allows customization of metric output. By default, metrics return a score and explanation. When this config is set, the default output is replaced with either: - The raw output string. - A parsed output based on a user-defined schema. If a custom format is chosen, the `score` and `explanation` fields in the corresponding metric result will be empty. + "returnRawOutput": True or False, # Optional. Whether to return raw output. + }, + "metricPromptTemplate": "A String", # Required. Metric prompt template for pointwise metric. + "systemInstruction": "A String", # Optional. System instructions for pointwise metric. + }, + "predefinedMetricSpec": { # The spec for a pre-defined metric. # The spec for a pre-defined metric. + "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". + "metricSpecParameters": { # Optional. The parameters needed to run the pre-defined metric. + "a_key": "", # Properties of the object. + }, + }, + "rougeSpec": { # Spec for rouge score metric - calculates the recall of n-grams in prediction as compared to reference - returns a score ranging between 0 and 1. # Spec for rouge metric. + "rougeType": "A String", # Optional. Supported rouge types are rougen[1-9], rougeL, and rougeLsum. + "splitSummaries": True or False, # Optional. Whether to split summaries while using rougeLsum. + "useStemmer": True or False, # Optional. Whether to use stemmer to compute rouge score. + }, + }, + "name": "A String", # Identifier. The resource name of the EvaluationMetric. Format: `projects/{project}/locations/{location}/evaluationMetrics/{evaluation_metric}` + "updateTime": "A String", # Output only. The time when the EvaluationMetric was last updated. +} + + evaluationMetricId: string, Optional. The ID to use for the EvaluationMetric, which will become the final component of the EvaluationMetric's resource name. This value should be 1-63 characters, and valid characters are /a-z-/. The first character must be a lowercase letter, and the last character must be a lowercase letter or number. + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # EvaluationMetric is a resource that represents a reusable metric configuration. + "createTime": "A String", # Output only. The time when the EvaluationMetric was created. + "description": "A String", # Optional. A description of the EvaluationMetric. + "displayName": "A String", # Required. The user-friendly display name for the EvaluationMetric. + "gcsUri": "A String", # Optional. The Google Cloud Storage URI that stores the metric specification.. + "labels": { # Optional. Labels for the evaluation metric. + "a_key": "A String", + }, + "metric": { # The metric used for running evaluations. # Optional. The metric configuration. + "aggregationMetrics": [ # Optional. The aggregation metrics to use. + "A String", + ], + "bleuSpec": { # Spec for bleu score metric - calculates the precision of n-grams in the prediction as compared to reference - returns a score ranging between 0 to 1. # Spec for bleu metric. + "useEffectiveOrder": True or False, # Optional. Whether to use_effective_order to compute bleu score. + }, + "computationBasedMetricSpec": { # Specification for a computation based metric. # Spec for a computation based metric. + "parameters": { # Optional. A map of parameters for the metric, e.g. {"rouge_type": "rougeL"}. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the computation based metric. + }, + "customCodeExecutionSpec": { # Specificies a metric that is populated by evaluating user-defined Python code. # Spec for Custom Code Execution metric. + "evaluationFunction": "A String", # Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -> float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name]. Example: Example input: ``` instance= EvaluationInstance( response=EvaluationInstance.InstanceData(text="The answer is 4."), reference=EvaluationInstance.InstanceData(text="4") ) ``` Example converted input: ``` { 'response': {'text': 'The answer is 4.'}, 'reference': {'text': '4'} } ``` Example python function: ``` def evaluate(instance: dict[str, Any]) -> float: if instance'response' == instance'reference': return 1.0 return 0.0 ``` CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function. + }, + "exactMatchSpec": { # Spec for exact match metric - returns 1 if prediction and reference exactly matches, otherwise 0. # Spec for exact match metric. + }, + "llmBasedMetricSpec": { # Specification for an LLM based metric. # Spec for an LLM based metric. + "additionalConfig": { # Optional. Optional additional configuration for the metric. + "a_key": "", # Properties of the object. + }, + "judgeAutoraterConfig": { # The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. # Optional. Optional configuration for the judge LLM (Autorater). + "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "flipEnabled": True or False, # Optional. Default is true. Whether to flip the candidate and baseline responses. This is only applicable to the pairwise metric. If enabled, also provide PairwiseMetricSpec.candidate_response_field_name and PairwiseMetricSpec.baseline_response_field_name. When rendering PairwiseMetricSpec.metric_prompt_template, the candidate and baseline fields will be flipped for half of the samples to reduce bias. + "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "modelConfig": { # Config for model selection. # Optional. Config for model selection. + "featureSelectionPreference": "A String", # Required. Feature selection preference. + }, + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1beta1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1beta1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. + }, + "metricPromptTemplate": "A String", # Required. Template for the prompt sent to the judge model. + "predefinedRubricGenerationSpec": { # The spec for a pre-defined metric. # Dynamically generate rubrics using a predefined spec. + "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". + "metricSpecParameters": { # Optional. The parameters needed to run the pre-defined metric. + "a_key": "", # Properties of the object. + }, + }, + "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification. + "modelConfig": { # The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. # Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. + "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "flipEnabled": True or False, # Optional. Default is true. Whether to flip the candidate and baseline responses. This is only applicable to the pairwise metric. If enabled, also provide PairwiseMetricSpec.candidate_response_field_name and PairwiseMetricSpec.baseline_response_field_name. When rendering PairwiseMetricSpec.metric_prompt_template, the candidate and baseline fields will be flipped for half of the samples to reduce bias. + "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "modelConfig": { # Config for model selection. # Optional. Config for model selection. + "featureSelectionPreference": "A String", # Required. Feature selection preference. + }, + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1beta1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1beta1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. + }, + "promptTemplate": "A String", # Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements. + "rubricContentType": "A String", # The type of rubric content to be generated. + "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology. + "A String", + ], + }, + "rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. + "systemInstruction": "A String", # Optional. System instructions for the judge model. + }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, + "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. + "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. + "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. + "customOutputFormatConfig": { # Spec for custom output format configuration. # Optional. CustomOutputFormatConfig allows customization of metric output. When this config is set, the default output is replaced with the raw output string. If a custom format is chosen, the `pairwise_choice` and `explanation` fields in the corresponding metric result will be empty. + "returnRawOutput": True or False, # Optional. Whether to return raw output. + }, + "metricPromptTemplate": "A String", # Required. Metric prompt template for pairwise metric. + "systemInstruction": "A String", # Optional. System instructions for pairwise metric. + }, + "pointwiseMetricSpec": { # Spec for pointwise metric. # Spec for pointwise metric. + "customOutputFormatConfig": { # Spec for custom output format configuration. # Optional. CustomOutputFormatConfig allows customization of metric output. By default, metrics return a score and explanation. When this config is set, the default output is replaced with either: - The raw output string. - A parsed output based on a user-defined schema. If a custom format is chosen, the `score` and `explanation` fields in the corresponding metric result will be empty. + "returnRawOutput": True or False, # Optional. Whether to return raw output. + }, + "metricPromptTemplate": "A String", # Required. Metric prompt template for pointwise metric. + "systemInstruction": "A String", # Optional. System instructions for pointwise metric. + }, + "predefinedMetricSpec": { # The spec for a pre-defined metric. # The spec for a pre-defined metric. + "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". + "metricSpecParameters": { # Optional. The parameters needed to run the pre-defined metric. + "a_key": "", # Properties of the object. + }, + }, + "rougeSpec": { # Spec for rouge score metric - calculates the recall of n-grams in prediction as compared to reference - returns a score ranging between 0 and 1. # Spec for rouge metric. + "rougeType": "A String", # Optional. Supported rouge types are rougen[1-9], rougeL, and rougeLsum. + "splitSummaries": True or False, # Optional. Whether to split summaries while using rougeLsum. + "useStemmer": True or False, # Optional. Whether to use stemmer to compute rouge score. + }, + }, + "name": "A String", # Identifier. The resource name of the EvaluationMetric. Format: `projects/{project}/locations/{location}/evaluationMetrics/{evaluation_metric}` + "updateTime": "A String", # Output only. The time when the EvaluationMetric was last updated. +}+++ +delete(name, x__xgafv=None)+Deletes an EvaluationMetric. + +Args: + name: string, Required. The name of the EvaluationMetric resource to be deleted. Format: `projects/{project}/locations/{location}/evaluationMetrics/{evaluation_metric}` (required) + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # This resource represents a long-running operation that is the result of a network API call. + "done": True or False, # If the value is `false`, it means the operation is still in progress. If `true`, the operation is completed, and either `error` or `response` is available. + "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # The error result of the operation in case of failure or cancellation. + "code": 42, # The status code, which should be an enum value of google.rpc.Code. + "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use. + { + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + ], + "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client. + }, + "metadata": { # Service-specific metadata associated with the operation. It typically contains progress information and common metadata such as create time. Some services might not provide such metadata. Any method that returns a long-running operation should document the metadata type, if any. + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + "name": "A String", # The server-assigned name, which is only unique within the same service that originally returns it. If you use the default HTTP mapping, the `name` should be a resource name ending with `operations/{unique_id}`. + "response": { # The normal, successful response of the operation. If the original method returns no data on success, such as `Delete`, the response is `google.protobuf.Empty`. If the original method is standard `Get`/`Create`/`Update`, the response should be the resource. For other methods, the response should have the type `XxxResponse`, where `Xxx` is the original method name. For example, if the original method name is `TakeSnapshot()`, the inferred response type is `TakeSnapshotResponse`. + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, +}+++ +get(name, x__xgafv=None)+Gets an EvaluationMetric. + +Args: + name: string, Required. The name of the EvaluationMetric resource. Format: `projects/{project}/locations/{location}/evaluationMetrics/{evaluation_metric}` (required) + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # EvaluationMetric is a resource that represents a reusable metric configuration. + "createTime": "A String", # Output only. The time when the EvaluationMetric was created. + "description": "A String", # Optional. A description of the EvaluationMetric. + "displayName": "A String", # Required. The user-friendly display name for the EvaluationMetric. + "gcsUri": "A String", # Optional. The Google Cloud Storage URI that stores the metric specification.. + "labels": { # Optional. Labels for the evaluation metric. + "a_key": "A String", + }, + "metric": { # The metric used for running evaluations. # Optional. The metric configuration. + "aggregationMetrics": [ # Optional. The aggregation metrics to use. + "A String", + ], + "bleuSpec": { # Spec for bleu score metric - calculates the precision of n-grams in the prediction as compared to reference - returns a score ranging between 0 to 1. # Spec for bleu metric. + "useEffectiveOrder": True or False, # Optional. Whether to use_effective_order to compute bleu score. + }, + "computationBasedMetricSpec": { # Specification for a computation based metric. # Spec for a computation based metric. + "parameters": { # Optional. A map of parameters for the metric, e.g. {"rouge_type": "rougeL"}. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the computation based metric. + }, + "customCodeExecutionSpec": { # Specificies a metric that is populated by evaluating user-defined Python code. # Spec for Custom Code Execution metric. + "evaluationFunction": "A String", # Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -> float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name]. Example: Example input: ``` instance= EvaluationInstance( response=EvaluationInstance.InstanceData(text="The answer is 4."), reference=EvaluationInstance.InstanceData(text="4") ) ``` Example converted input: ``` { 'response': {'text': 'The answer is 4.'}, 'reference': {'text': '4'} } ``` Example python function: ``` def evaluate(instance: dict[str, Any]) -> float: if instance'response' == instance'reference': return 1.0 return 0.0 ``` CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function. + }, + "exactMatchSpec": { # Spec for exact match metric - returns 1 if prediction and reference exactly matches, otherwise 0. # Spec for exact match metric. + }, + "llmBasedMetricSpec": { # Specification for an LLM based metric. # Spec for an LLM based metric. + "additionalConfig": { # Optional. Optional additional configuration for the metric. + "a_key": "", # Properties of the object. + }, + "judgeAutoraterConfig": { # The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. # Optional. Optional configuration for the judge LLM (Autorater). + "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "flipEnabled": True or False, # Optional. Default is true. Whether to flip the candidate and baseline responses. This is only applicable to the pairwise metric. If enabled, also provide PairwiseMetricSpec.candidate_response_field_name and PairwiseMetricSpec.baseline_response_field_name. When rendering PairwiseMetricSpec.metric_prompt_template, the candidate and baseline fields will be flipped for half of the samples to reduce bias. + "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "modelConfig": { # Config for model selection. # Optional. Config for model selection. + "featureSelectionPreference": "A String", # Required. Feature selection preference. + }, + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1beta1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1beta1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. + }, + "metricPromptTemplate": "A String", # Required. Template for the prompt sent to the judge model. + "predefinedRubricGenerationSpec": { # The spec for a pre-defined metric. # Dynamically generate rubrics using a predefined spec. + "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". + "metricSpecParameters": { # Optional. The parameters needed to run the pre-defined metric. + "a_key": "", # Properties of the object. + }, + }, + "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification. + "modelConfig": { # The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. # Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. + "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "flipEnabled": True or False, # Optional. Default is true. Whether to flip the candidate and baseline responses. This is only applicable to the pairwise metric. If enabled, also provide PairwiseMetricSpec.candidate_response_field_name and PairwiseMetricSpec.baseline_response_field_name. When rendering PairwiseMetricSpec.metric_prompt_template, the candidate and baseline fields will be flipped for half of the samples to reduce bias. + "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "modelConfig": { # Config for model selection. # Optional. Config for model selection. + "featureSelectionPreference": "A String", # Required. Feature selection preference. + }, + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1beta1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1beta1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. + }, + "promptTemplate": "A String", # Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements. + "rubricContentType": "A String", # The type of rubric content to be generated. + "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology. + "A String", + ], + }, + "rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. + "systemInstruction": "A String", # Optional. System instructions for the judge model. + }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, + "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. + "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. + "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. + "customOutputFormatConfig": { # Spec for custom output format configuration. # Optional. CustomOutputFormatConfig allows customization of metric output. When this config is set, the default output is replaced with the raw output string. If a custom format is chosen, the `pairwise_choice` and `explanation` fields in the corresponding metric result will be empty. + "returnRawOutput": True or False, # Optional. Whether to return raw output. + }, + "metricPromptTemplate": "A String", # Required. Metric prompt template for pairwise metric. + "systemInstruction": "A String", # Optional. System instructions for pairwise metric. + }, + "pointwiseMetricSpec": { # Spec for pointwise metric. # Spec for pointwise metric. + "customOutputFormatConfig": { # Spec for custom output format configuration. # Optional. CustomOutputFormatConfig allows customization of metric output. By default, metrics return a score and explanation. When this config is set, the default output is replaced with either: - The raw output string. - A parsed output based on a user-defined schema. If a custom format is chosen, the `score` and `explanation` fields in the corresponding metric result will be empty. + "returnRawOutput": True or False, # Optional. Whether to return raw output. + }, + "metricPromptTemplate": "A String", # Required. Metric prompt template for pointwise metric. + "systemInstruction": "A String", # Optional. System instructions for pointwise metric. + }, + "predefinedMetricSpec": { # The spec for a pre-defined metric. # The spec for a pre-defined metric. + "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". + "metricSpecParameters": { # Optional. The parameters needed to run the pre-defined metric. + "a_key": "", # Properties of the object. + }, + }, + "rougeSpec": { # Spec for rouge score metric - calculates the recall of n-grams in prediction as compared to reference - returns a score ranging between 0 and 1. # Spec for rouge metric. + "rougeType": "A String", # Optional. Supported rouge types are rougen[1-9], rougeL, and rougeLsum. + "splitSummaries": True or False, # Optional. Whether to split summaries while using rougeLsum. + "useStemmer": True or False, # Optional. Whether to use stemmer to compute rouge score. + }, + }, + "name": "A String", # Identifier. The resource name of the EvaluationMetric. Format: `projects/{project}/locations/{location}/evaluationMetrics/{evaluation_metric}` + "updateTime": "A String", # Output only. The time when the EvaluationMetric was last updated. +}+++ +list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)+Lists EvaluationMetrics. + +Args: + parent: string, Required. The resource name of the Location from which to list the EvaluationMetrics. Format: `projects/{project}/locations/{location}` (required) + filter: string, Optional. Filter expression that matches a subset of the EvaluationMetrics to show. For field names both snake_case and camelCase are supported. For more information about filter syntax, see [AIP-160](https://google.aip.dev/160). + orderBy: string, Optional. A comma-separated list of fields to order by, sorted in ascending order by default. Use `desc` after a field name for descending. + pageSize: integer, Optional. The maximum number of EvaluationMetrics to return. + pageToken: string, Optional. A page token, received from a previous `ListEvaluationMetrics` call. Provide this to retrieve the subsequent page. + x__xgafv: string, V1 error format. + Allowed values + 1 - v1 error format + 2 - v2 error format + +Returns: + An object of the form: + + { # Response message for EvaluationMetricService.ListEvaluationMetrics. + "evaluationMetrics": [ # List of EvaluationMetrics in the requested page. + { # EvaluationMetric is a resource that represents a reusable metric configuration. + "createTime": "A String", # Output only. The time when the EvaluationMetric was created. + "description": "A String", # Optional. A description of the EvaluationMetric. + "displayName": "A String", # Required. The user-friendly display name for the EvaluationMetric. + "gcsUri": "A String", # Optional. The Google Cloud Storage URI that stores the metric specification.. + "labels": { # Optional. Labels for the evaluation metric. + "a_key": "A String", + }, + "metric": { # The metric used for running evaluations. # Optional. The metric configuration. + "aggregationMetrics": [ # Optional. The aggregation metrics to use. + "A String", + ], + "bleuSpec": { # Spec for bleu score metric - calculates the precision of n-grams in the prediction as compared to reference - returns a score ranging between 0 to 1. # Spec for bleu metric. + "useEffectiveOrder": True or False, # Optional. Whether to use_effective_order to compute bleu score. + }, + "computationBasedMetricSpec": { # Specification for a computation based metric. # Spec for a computation based metric. + "parameters": { # Optional. A map of parameters for the metric, e.g. {"rouge_type": "rougeL"}. + "a_key": "", # Properties of the object. + }, + "type": "A String", # Required. The type of the computation based metric. + }, + "customCodeExecutionSpec": { # Specificies a metric that is populated by evaluating user-defined Python code. # Spec for Custom Code Execution metric. + "evaluationFunction": "A String", # Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -> float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name]. Example: Example input: ``` instance= EvaluationInstance( response=EvaluationInstance.InstanceData(text="The answer is 4."), reference=EvaluationInstance.InstanceData(text="4") ) ``` Example converted input: ``` { 'response': {'text': 'The answer is 4.'}, 'reference': {'text': '4'} } ``` Example python function: ``` def evaluate(instance: dict[str, Any]) -> float: if instance'response' == instance'reference': return 1.0 return 0.0 ``` CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function. + }, + "exactMatchSpec": { # Spec for exact match metric - returns 1 if prediction and reference exactly matches, otherwise 0. # Spec for exact match metric. + }, + "llmBasedMetricSpec": { # Specification for an LLM based metric. # Spec for an LLM based metric. + "additionalConfig": { # Optional. Optional additional configuration for the metric. + "a_key": "", # Properties of the object. + }, + "judgeAutoraterConfig": { # The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. # Optional. Optional configuration for the judge LLM (Autorater). + "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "flipEnabled": True or False, # Optional. Default is true. Whether to flip the candidate and baseline responses. This is only applicable to the pairwise metric. If enabled, also provide PairwiseMetricSpec.candidate_response_field_name and PairwiseMetricSpec.baseline_response_field_name. When rendering PairwiseMetricSpec.metric_prompt_template, the candidate and baseline fields will be flipped for half of the samples to reduce bias. + "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "modelConfig": { # Config for model selection. # Optional. Config for model selection. + "featureSelectionPreference": "A String", # Required. Feature selection preference. + }, + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1beta1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1beta1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. + }, + "metricPromptTemplate": "A String", # Required. Template for the prompt sent to the judge model. + "predefinedRubricGenerationSpec": { # The spec for a pre-defined metric. # Dynamically generate rubrics using a predefined spec. + "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". + "metricSpecParameters": { # Optional. The parameters needed to run the pre-defined metric. + "a_key": "", # Properties of the object. + }, + }, + "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification. + "modelConfig": { # The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. # Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation. + "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` + "flipEnabled": True or False, # Optional. Default is true. Whether to flip the candidate and baseline responses. This is only applicable to the pairwise metric. If enabled, also provide PairwiseMetricSpec.candidate_response_field_name and PairwiseMetricSpec.baseline_response_field_name. When rendering PairwiseMetricSpec.metric_prompt_template, the candidate and baseline fields will be flipped for half of the samples to reduce bias. + "generationConfig": { # Configuration for content generation. This message contains all the parameters that control how the model generates content. It allows you to influence the randomness, length, and structure of the output. # Optional. Configuration options for model generation and outputs. + "audioTimestamp": True or False, # Optional. If enabled, audio timestamps will be included in the request to the model. This can be useful for synchronizing audio with other modalities in the response. + "candidateCount": 42, # Optional. The number of candidate responses to generate. A higher `candidate_count` can provide more options to choose from, but it also consumes more resources. This can be useful for generating a variety of responses and selecting the best one. + "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly. For example, if the model detects that the user is frustrated, it may provide a more empathetic response. + "frequencyPenalty": 3.14, # Optional. Penalizes tokens based on their frequency in the generated text. A positive value helps to reduce the repetition of words and phrases. Valid values can range from [-2.0, 2.0]. + "imageConfig": { # Configuration for image generation. This message allows you to control various aspects of image generation, such as the output format, aspect ratio, and whether the model can generate images of people. # Optional. Config for image generation features. + "aspectRatio": "A String", # Optional. The desired aspect ratio for the generated images. The following aspect ratios are supported: "1:1" "2:3", "3:2" "3:4", "4:3" "4:5", "5:4" "9:16", "16:9" "21:9" + "imageOutputOptions": { # The image output format for generated images. # Optional. The image output format for generated images. + "compressionQuality": 42, # Optional. The compression quality of the output image. + "mimeType": "A String", # Optional. The image format that the output should be saved as. + }, + "imageSize": "A String", # Optional. Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. + "personGeneration": "A String", # Optional. Controls whether the model can generate people. + "prominentPeople": "A String", # Optional. Controls whether prominent people (celebrities) generation is allowed. If used with personGeneration, personGeneration enum would take precedence. For instance, if ALLOW_NONE is set, all person generation would be blocked. If this field is unspecified, the default behavior is to allow prominent people. + }, + "logprobs": 42, # Optional. The number of top log probabilities to return for each token. This can be used to see which other tokens were considered likely candidates for a given position. A higher value will return more options, but it will also increase the size of the response. + "maxOutputTokens": 42, # Optional. The maximum number of tokens to generate in the response. A token is approximately four characters. The default value varies by model. This parameter can be used to control the length of the generated text and prevent overly long responses. + "mediaResolution": "A String", # Optional. The token resolution at which input media content is sampled. This is used to control the trade-off between the quality of the response and the number of tokens used to represent the media. A higher resolution allows the model to perceive more detail, which can lead to a more nuanced response, but it will also use more tokens. This does not affect the image dimensions sent to the model. + "modelConfig": { # Config for model selection. # Optional. Config for model selection. + "featureSelectionPreference": "A String", # Required. Feature selection preference. + }, + "presencePenalty": 3.14, # Optional. Penalizes tokens that have already appeared in the generated text. A positive value encourages the model to generate more diverse and less repetitive text. Valid values can range from [-2.0, 2.0]. + "responseJsonSchema": "", # Optional. When this field is set, response_schema must be omitted and response_mime_type must be set to `application/json`. + "responseLogprobs": True or False, # Optional. If set to true, the log probabilities of the output tokens are returned. Log probabilities are the logarithm of the probability of a token appearing in the output. A higher log probability means the token is more likely to be generated. This can be useful for analyzing the model's confidence in its own output and for debugging. + "responseMimeType": "A String", # Optional. The IANA standard MIME type of the response. The model will generate output that conforms to this MIME type. Supported values include 'text/plain' (default) and 'application/json'. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. + "responseModalities": [ # Optional. The modalities of the response. The model will generate a response that includes all the specified modalities. For example, if this is set to `[TEXT, IMAGE]`, the response will include both text and an image. + "A String", + ], + "responseSchema": { # Defines the schema of input and output data. This is a subset of the [OpenAPI 3.0 Schema Object](https://spec.openapis.org/oas/v3.0.3#schema-object). # Optional. Lets you to specify a schema for the model's response, ensuring that the output conforms to a particular structure. This is useful for generating structured data such as JSON. The schema is a subset of the [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema) object. When this field is set, you must also set the `response_mime_type` to `application/json`. + "additionalProperties": "", # Optional. If `type` is `OBJECT`, specifies how to handle properties not defined in `properties`. If it is a boolean `false`, no additional properties are allowed. If it is a schema, additional properties are allowed if they conform to the schema. + "anyOf": [ # Optional. The instance must be valid against any (one or more) of the subschemas listed in `any_of`. + # Object with schema name: GoogleCloudAiplatformV1beta1Schema + ], + "default": "", # Optional. Default value to use if the field is not specified. + "defs": { # Optional. `defs` provides a map of schema definitions that can be reused by `ref` elsewhere in the schema. Only allowed at root level of the schema. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "description": "A String", # Optional. Describes the data. The model uses this field to understand the purpose of the schema and how to use it. It is a best practice to provide a clear and descriptive explanation for the schema and its properties here, rather than in the prompt. + "enum": [ # Optional. Possible values of the field. This field can be used to restrict a value to a fixed set of values. To mark a field as an enum, set `format` to `enum` and provide the list of possible values in `enum`. For example: 1. To define directions: `{type:STRING, format:enum, enum:["EAST", "NORTH", "SOUTH", "WEST"]}` 2. To define apartment numbers: `{type:INTEGER, format:enum, enum:["101", "201", "301"]}` + "A String", + ], + "example": "", # Optional. Example of an instance of this schema. + "format": "A String", # Optional. The format of the data. For `NUMBER` type, format can be `float` or `double`. For `INTEGER` type, format can be `int32` or `int64`. For `STRING` type, format can be `email`, `byte`, `date`, `date-time`, `password`, and other formats to further refine the data type. + "items": # Object with schema name: GoogleCloudAiplatformV1beta1Schema # Optional. If type is `ARRAY`, `items` specifies the schema of elements in the array. + "maxItems": "A String", # Optional. If type is `ARRAY`, `max_items` specifies the maximum number of items in an array. + "maxLength": "A String", # Optional. If type is `STRING`, `max_length` specifies the maximum length of the string. + "maxProperties": "A String", # Optional. If type is `OBJECT`, `max_properties` specifies the maximum number of properties that can be provided. + "maximum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `maximum` specifies the maximum allowed value. + "minItems": "A String", # Optional. If type is `ARRAY`, `min_items` specifies the minimum number of items in an array. + "minLength": "A String", # Optional. If type is `STRING`, `min_length` specifies the minimum length of the string. + "minProperties": "A String", # Optional. If type is `OBJECT`, `min_properties` specifies the minimum number of properties that can be provided. + "minimum": 3.14, # Optional. If type is `INTEGER` or `NUMBER`, `minimum` specifies the minimum allowed value. + "nullable": True or False, # Optional. Indicates if the value of this field can be null. + "pattern": "A String", # Optional. If type is `STRING`, `pattern` specifies a regular expression that the string must match. + "properties": { # Optional. If type is `OBJECT`, `properties` is a map of property names to schema definitions for each property of the object. + "a_key": # Object with schema name: GoogleCloudAiplatformV1beta1Schema + }, + "propertyOrdering": [ # Optional. Order of properties displayed or used where order matters. This is not a standard field in OpenAPI specification, but can be used to control the order of properties. + "A String", + ], + "ref": "A String", # Optional. Allows referencing another schema definition to use in place of this schema. The value must be a valid reference to a schema in `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring + "required": [ # Optional. If type is `OBJECT`, `required` lists the names of properties that must be present. + "A String", + ], + "title": "A String", # Optional. Title for the schema. + "type": "A String", # Optional. Data type of the schema field. + }, + "routingConfig": { # The configuration for routing the request to a specific model. This can be used to control which model is used for the generation, either automatically or by specifying a model name. # Optional. Routing configuration. + "autoMode": { # The configuration for automated routing. When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # In this mode, the model is selected automatically based on the content of the request. + "modelRoutingPreference": "A String", # The model routing preference. + }, + "manualMode": { # The configuration for manual routing. When manual routing is specified, the model will be selected based on the model name provided. # In this mode, the model is specified manually. + "modelName": "A String", # The name of the model to use. Only public LLM models are accepted. + }, + }, + "seed": 42, # Optional. A seed for the random number generator. By setting a seed, you can make the model's output mostly deterministic. For a given prompt and parameters (like temperature, top_p, etc.), the model will produce the same response every time. However, it's not a guaranteed absolute deterministic behavior. This is different from parameters like `temperature`, which control the *level* of randomness. `seed` ensures that the "random" choices the model makes are the same on every run, making it essential for testing and ensuring reproducible results. + "speechConfig": { # Configuration for speech generation. # Optional. The speech generation config. + "languageCode": "A String", # Optional. The language code (ISO 639-1) for the speech synthesis. + "multiSpeakerVoiceConfig": { # Configuration for a multi-speaker text-to-speech request. # The configuration for a multi-speaker text-to-speech request. This field is mutually exclusive with `voice_config`. + "speakerVoiceConfigs": [ # Required. A list of configurations for the voices of the speakers. Exactly two speaker voice configurations must be provided. + { # Configuration for a single speaker in a multi-speaker setup. + "speaker": "A String", # Required. The name of the speaker. This should be the same as the speaker name used in the prompt. + "voiceConfig": { # Configuration for a voice. # Required. The configuration for the voice of this speaker. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + ], + }, + "voiceConfig": { # Configuration for a voice. # The configuration for the voice to use. + "prebuiltVoiceConfig": { # Configuration for a prebuilt voice. # The configuration for a prebuilt voice. + "voiceName": "A String", # The name of the prebuilt voice to use. + }, + "replicatedVoiceConfig": { # The configuration for the replicated voice to use. # Optional. The configuration for a replicated voice. This enables users to replicate a voice from an audio sample. + "mimeType": "A String", # Optional. The mimetype of the voice sample. The only currently supported value is `audio/wav`. This represents 16-bit signed little-endian wav data, with a 24kHz sampling rate. `mime_type` will default to `audio/wav` if not set. + "voiceSampleAudio": "A String", # Optional. The sample of the custom voice. + }, + }, + }, + "stopSequences": [ # Optional. A list of character sequences that will stop the model from generating further tokens. If a stop sequence is generated, the output will end at that point. This is useful for controlling the length and structure of the output. For example, you can use ["\n", "###"] to stop generation at a new line or a specific marker. + "A String", + ], + "temperature": 3.14, # Optional. Controls the randomness of the output. A higher temperature results in more creative and diverse responses, while a lower temperature makes the output more predictable and focused. The valid range is (0.0, 2.0]. + "thinkingConfig": { # Configuration for the model's thinking features. "Thinking" is a process where the model breaks down a complex task into smaller, manageable steps. This allows the model to reason about the task, plan its approach, and execute the plan to generate a high-quality response. # Optional. Configuration for thinking features. An error will be returned if this field is set for models that don't support thinking. + "includeThoughts": True or False, # Optional. If true, the model will include its thoughts in the response. "Thoughts" are the intermediate steps the model takes to arrive at the final response. They can provide insights into the model's reasoning process and help with debugging. If this is true, thoughts are returned only when available. + "thinkingBudget": 42, # Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. This can be used to control the trade-off between response quality and latency. + "thinkingLevel": "A String", # Optional. The number of thoughts tokens that the model should generate. + }, + "topK": 3.14, # Optional. Specifies the top-k sampling threshold. The model considers only the top k most probable tokens for the next token. This can be useful for generating more coherent and less random text. For example, a `top_k` of 40 means the model will choose the next word from the 40 most likely words. + "topP": 3.14, # Optional. Specifies the nucleus sampling threshold. The model considers only the smallest set of tokens whose cumulative probability is at least `top_p`. This helps generate more diverse and less repetitive responses. For example, a `top_p` of 0.9 means the model considers tokens until the cumulative probability of the tokens to select from reaches 0.9. It's recommended to adjust either temperature or `top_p`, but not both. + }, + "samplingCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32. + }, + "promptTemplate": "A String", # Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements. + "rubricContentType": "A String", # The type of rubric content to be generated. + "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology. + "A String", + ], + }, + "rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance. + "systemInstruction": "A String", # Optional. System instructions for the judge model. + }, + "metadata": { # Metadata about the metric, used for visualization and organization. # Optional. Metadata about the metric, used for visualization and organization. + "otherMetadata": { # Optional. Flexible metadata for user-defined attributes. + "a_key": "", # Properties of the object. + }, + "scoreRange": { # The range of possible scores for this metric, used for plotting. # Optional. The range of possible scores for this metric, used for plotting. + "description": "A String", # Optional. The description of the score explaining the directionality etc. + "max": 3.14, # Required. The maximum value of the score range (inclusive). + "min": 3.14, # Required. The minimum value of the score range (inclusive). + "step": 3.14, # Optional. The distance between discrete steps in the range. If unset, the range is assumed to be continuous. + }, + "title": "A String", # Optional. The user-friendly name for the metric. If not set for a registered metric, it will default to the metric's display name. + }, + "pairwiseMetricSpec": { # Spec for pairwise metric. # Spec for pairwise metric. + "baselineResponseFieldName": "A String", # Optional. The field name of the baseline response. + "candidateResponseFieldName": "A String", # Optional. The field name of the candidate response. + "customOutputFormatConfig": { # Spec for custom output format configuration. # Optional. CustomOutputFormatConfig allows customization of metric output. When this config is set, the default output is replaced with the raw output string. If a custom format is chosen, the `pairwise_choice` and `explanation` fields in the corresponding metric result will be empty. + "returnRawOutput": True or False, # Optional. Whether to return raw output. + }, + "metricPromptTemplate": "A String", # Required. Metric prompt template for pairwise metric. + "systemInstruction": "A String", # Optional. System instructions for pairwise metric. + }, + "pointwiseMetricSpec": { # Spec for pointwise metric. # Spec for pointwise metric. + "customOutputFormatConfig": { # Spec for custom output format configuration. # Optional. CustomOutputFormatConfig allows customization of metric output. By default, metrics return a score and explanation. When this config is set, the default output is replaced with either: - The raw output string. - A parsed output based on a user-defined schema. If a custom format is chosen, the `score` and `explanation` fields in the corresponding metric result will be empty. + "returnRawOutput": True or False, # Optional. Whether to return raw output. + }, + "metricPromptTemplate": "A String", # Required. Metric prompt template for pointwise metric. + "systemInstruction": "A String", # Optional. System instructions for pointwise metric. + }, + "predefinedMetricSpec": { # The spec for a pre-defined metric. # The spec for a pre-defined metric. + "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1". + "metricSpecParameters": { # Optional. The parameters needed to run the pre-defined metric. + "a_key": "", # Properties of the object. + }, + }, + "rougeSpec": { # Spec for rouge score metric - calculates the recall of n-grams in prediction as compared to reference - returns a score ranging between 0 and 1. # Spec for rouge metric. + "rougeType": "A String", # Optional. Supported rouge types are rougen[1-9], rougeL, and rougeLsum. + "splitSummaries": True or False, # Optional. Whether to split summaries while using rougeLsum. + "useStemmer": True or False, # Optional. Whether to use stemmer to compute rouge score. + }, + }, + "name": "A String", # Identifier. The resource name of the EvaluationMetric. Format: `projects/{project}/locations/{location}/evaluationMetrics/{evaluation_metric}` + "updateTime": "A String", # Output only. The time when the EvaluationMetric was last updated. + }, + ], + "nextPageToken": "A String", # A token to retrieve the next page of results. +}+++list_next()+Retrieves the next page of results. + + Args: + previous_request: The request for the previous page. (required) + previous_response: The response from the request for the previous page. (required) + + Returns: + A request object that you can call 'execute()' on to request the next + page. Returns None if there are no more items in the collection. ++