[eyudashkin-cpu]

Draft. Do not merge or release yet. The deploy part only works once the Data Cloud platform accepts the AgentforceScorer invoke option and its feature gate is open. Until then, a deploy would be rejected. The types, template and local run work today. This PR is open now so the request/response contract can be reviewed early.

Summary

Adds AgentforceScorer, a new function feature for scoring a single Agentforce session. The scorer receives the session tracing detail view and returns open-ended scorer outputs.

datacustomcode init my_scorer --code-type function --use-in-feature AgentforceScorer
datacustomcode run my_scorer/payload/entrypoint.py --test-with my_scorer/payload/tests/test.json

Changes

• Types: new function/feature_types/agentforce_scorer.py with AgentforceScorerV1Request / AgentforceScorerV1Response, following the chunking.py conventions (V1 names, Field(description=...), extra="ignore").
  • Request: python_scorer_input: AgentforceScorerV1SessionTracingDetailView, made up of sessionState, actors, metrics and runs. Each run has messages and an agentLoop.steps.
  • Response: outputs: List[AgentforceScorerV1Output], where each output has label, value and is_passed.
  • Field names stay in the wire format (camelCase). The free-text fields (message, step input/output/attributes) are capped at 100,000 characters.
• Template: new templates/function/agentforce_scorer/ with a typed entrypoint.py, config.json and requirements.txt. test.json is generated from the request model.
• constants.py: registers the template folder and maps both type names to AgentforceScorer for deploy-time inference. Also adds FUNCTION_INVOKE_OPTIONS_FEATURES and FUNCTION_USE_IN_FEATURE_DEFAULT.
• init: unknown function features are now rejected. Before, a typo silently produced the chunking template.
• deploy: AgentforceScorer is sent as invokeOptions: ["AgentforceScorer"], which routes to the v67.0 endpoint. The legacy functionInvokeOptions field only accepts UnstructuredChunking. SearchIndexChunking behaves as before.

Testing

• New tests/test_agentforce_scorer.py (8 tests) covers:
  • parsing, defaults and serialization of the types
  • init scaffolding, test.json generation and feature inference
  • rejection of unknown features
  • running the template entrypoint
  • both deploy routes: invokeOptions for the scorer, functionInvokeOptions for chunking
• Full suite on Python 3.11: 707 passed.
• black and isort are clean on the changed files.
• Manual check: init → run --test-with returns {"outputs": [{"label": "user_turns", "value": "1", "is_passed": true}]}.
• Not tested: a real deploy against an org, which is blocked as described in the note above.

W-24740772

🤖 Generated with Claude Code

[salesforce-cla]

Thanks for the contribution! It looks like @eyudashkin-cpu is an internal user so signing the CLA is not required. However, we need to confirm this.

[eyudashkin-cpu]

@salesforce-cla recheck