Out-of-the-box Metrics Overview

The metrics listed below are out-of-the-box metrics already available on the LogSpend platform and we constantly add new metrics. However, you are not limited to these metrics and new metrics can be easily configured and created using custom formulas in the “Reports” section.

Below is a list of some commonly used metrics and their definitions:

• Sessions: Represents the number of user sessions. All interactions within a conversation or part of a task within a given time period. A new session is created when there is more than 30 minutes of inactivity.
• Session Duration: Represents the session duration in milliseconds.
• User Messages: The number of user messages in a session.
• Assistant Messages: The number of assistant messages in a session.
• Function Calls: The number of function calls in a session.
• Gender Bias: The number of sessions where the assistant's response contains hurtful content or there is a difference between groups (e.g. genders, sexual orientations, etc.).
• Language Match: The number of sessions where the assistant responds in the expected language(s).
• Toxicity: The number of sessions where the assistant's response contains problematic content, such as hate speech.
• User Sentiment: The number of sessions where users are expressing the corresponding user sentiment: positive, negative, or neutral.
• Cost: How much was spent to process user interactions. This is directly computed based on LLM provider pricing but custom cost can also be provided as a custom property.
• Requests/Interactions: Represents the number of user interactions or LLM requests.
• Failed Requests: Represents the number of times an interaction couldn’t be processed due to system errors.
• Input Tokens: Represents the number of input tokens for each LLM request.
• Output Tokens: Represents the number of output tokens for each LLM request.
• Latency: Represents the latency in milliseconds for each LLM request.