Skip to main content

Custom Metrics Overview

Leverage custom metrics to gain deeper insights and make data-driven decisions. Here are some useful custom metrics you could create in the “Reports” section.

User Engagement & Satisfaction​

User Engagement Rate​

Measures the average number of user messages per session, indicating user engagement.

  • Metric A: Number of User Messages
  • Metric B: Number of Sessions
  • Formula: “SUM(A) / SUM(B)”

Average Session Duration (in seconds)​

Provides the average duration of each session.

  • Metric A: Session Duration in Milliseconds
  • Metric B: Number of Sessions
  • Formula: “SUM(A) / SUM(B) / 60,000”

Positive Sentiment Rate​

Measures the percentage of sessions with positive user sentiment.

  • Metric A: Sessions with Positive User Sentiment
  • Metric B: Number of Sessions
  • Formula: (SUM(A) / SUM(B)) * 100

Negative Sentiment Rate​

Measures the percentage of sessions with negative user sentiment.

  • Metric A: Sessions with Negative User Sentiment
  • Metric B: Number of Sessions
  • Formula: (SUM(A) / SUM(B)) * 100

Sentiment Ratio (Positive to Negative)​

This ratio helps understand the balance of user sentiment.

  • Metric A: Sessions with Positive User Sentiment
  • Metric B: Sessions with Negative User Sentiment
  • Formula: “SUM(A) / SUM(B)”

Content & Language Quality​

Language Accuracy​

Indicates the percentage of sessions where the assistant responded in the expected language.

  • Metric A: Sessions with Assistant Responding in the Expected Language(s)
  • Metric B: Number of Sessions
  • Formula: “(SUM(A) / SUM(B)) * 100”

Gender Bias Rate​

Measures the percentage of sessions containing gender bias in the assistant's response.

  • Metric A: Sessions with Gender Bias in Assistant's Response
  • Metric B: Number of Sessions
  • Formula: “(SUM(A) / SUM(B)) * 100”

Toxic Content Rate​

Measures the percentage of sessions containing toxic content in the assistant's response.

  • Metric A: Sessions with Toxic Content in Assistant's Response
  • Metric B: Number of Sessions
  • Formula: “(SUM(A) / SUM(B)) * 100”

Cost & Operational Performance​

Cost per Session​

This metric helps understand the average cost incurred for each session.

  • Metric A: Cost
  • Metric B: Number of Sessions
  • Formula: “SUM(A) / SUM(B)”

Success Rate​

This percentage metric shows the rate of successful requests.

  • Metric A: Number of Requests
  • Metric B: Number of Failed Requests
  • Formula: ((SUM(A) - SUM(B)) / SUM(A)) * 100

Average Latency per Request​

This metric provides the average time taken to process each request.

  • Metric A: Latency in Milliseconds
  • Metric B: Number of Requests
  • Formula: “SUM(A) / SUM(B)”

Token Utilization Efficiency​

Indicates the average number of tokens used per request, helping to track token usage.

  • Metric A: Number of Tokens (Input and Output)
  • Metric B: Number of Requests
  • Formula: “SUM(A) / SUM(B)”

Average Input Tokens per Request​

Provides the average number of input tokens per request.

  • Metric A: Number of Input Tokens
  • Metric B: Number of Requests
  • Formula: “SUM(A) / SUM(B)”

Average Output Tokens per Request​

Provides the average number of output tokens per request.

  • Metric A: Number of Output Tokens
  • Metric B: Number of Requests
  • Formula: “SUM(A) / SUM(B)”

Assistant Response Efficiency​

Evaluates how responsive the assistant is compared to user messages.

  • Metric A: Number of Assistant Messages
  • Metric B: Number of User Messages
  • Formula: “SUM(A) / SUM(B)”