Type Alias PublicEvaluatorOutputDefinition

PublicEvaluatorOutputDefinition:
    | unstable.PublicEvaluatorOutputDefinition.Numeric
    | unstable.PublicEvaluatorOutputDefinition.Boolean
    | unstable.PublicEvaluatorOutputDefinition.Categorical

Evaluator output definition returned by the public API.

This response always includes dataType and never includes an internal output-definition version. Legacy stored evaluator definitions are normalized into this shape before they are returned.

Use this response shape when deciding how to interpret future evaluation scores:

NUMERIC: expect numeric score values
BOOLEAN: expect true / false
CATEGORICAL: expect one or more values from score.categories

Example

{
 *         dataType: "NUMERIC",
 *         dataType: LangfuseAPI.unstable.EvaluatorOutputDataType.Numeric,
 *         reasoning: {
 *             description: "Explain why the answer is correct or incorrect."
 *         },
 *         score: {
 *             description: "Return a score between 0 and 1."
 *         }
 *     }

Example

{
 *         dataType: "CATEGORICAL",
 *         dataType: LangfuseAPI.unstable.EvaluatorOutputDataType.Categorical,
 *         reasoning: {
 *             description: "Explain which label best fits the output."
 *         },
 *         score: {
 *             description: "Choose the best label.",
 *             categories: ["correct", "partially_correct", "incorrect"],
 *             shouldAllowMultipleMatches: false
 *         }
 *     }

Type Alias PublicEvaluatorOutputDefinition

Example

Example

Settings