Configuring KPI Metrics

Configuring KPI Metrics KPI Metrics are a set of use cases you care about for your specific bot or VA. These are user-defined metrics, which must be explicitly configured.
An example use case is shown below.
All the data shown in a use case is driven by a set of predetermined goals.
It can be beneficial to set goals as a user enters a use case, completes a use case, and fails a use case.
It is better to define use case failure as a single discrete goal, and annotate the reasons for that failure via “properties” on the use case (e.g. “reason” : “payment_failed”). The same is true for defining success - as one single goal.
The combination of success and failure can then be used to define success, abandonment and failure rates, as the example below shows.
The KPI page supports the definition of as many use cases as you’d like.
Often, a use case maps neatly to an intent with some complexity in its fulfillment.

kpi-metrics

Here is a complete definition of a bot’s dashboard. This includes:

  • 1x section of additional Usage Metrics
  • 1x sample usecase
{
  "alias": "test-dashboard", // required
  "title": "the best dashboard ever", // required
  "resourceName": "ClassicFlow", // required
  "resourceType": "Bot", // required
  "labels": {},
  "businessHours": {
    //optional
    "monday": "09-17",
    "tuesday": "09-17",
    "wednesday": "09-17",
    "thursday": "09-17",
    "friday": "09-17",
    "saturday": "09-17",
    "sunday": "09-17"
  },
  "useCases": [
    // required - min one item
    {
      "title": "start ues case", // required
      "summary": {
        "start": "conversation_start", // required
        "success": "conversation_engaged", // required
        "failure": "dead" // required
      },
      "metrics": [
        // optional
        {
          "title": "start count", // required
          "type": "bucket", // required
          "properties": {
            "goal": "conversation_start"
          }
        }
      ]
    }
  ],
  "additionalUsageMetrics": [
    // optional
    {
      "title": "start ues case", // required
      "summary": {
        "start": "conversation_start", // required
        "success": "conversation_engaged", // required
        "failure": "dead" // required
      },
      "metrics": [
        // optional
        {
          "title": "start count", // required
          "type": "bucket", // required
          "properties": {
            "field": "Reason",
            "goal": "conversation_start"
          }
        }
      ]
    }
  ]
}

Feel free to use the above example as a starting point.

We will now take a closer look at defining useCases:
(Note the following examples are a subset of the entire object needed to define a dashboard)

{
  "useCases": [
    // required - min one item
    {
      "title": "start use case", // required
      "summary": {
        "start": "conversation_start", // required
        "success": "conversation_engaged", // required
        "failure": "dead" // required
      },
      "metrics": [
        // optional
        {
          "title": "start count", // required
          "type": "bucket", // required
          "properties": {
            "field": "Reason",
            "goal": "conversation_start"
          }
        }
      ]
    }
  ]
}

Certain parts of this payload are required, like the summary section. This is what gives us the tile at the top in the section labeled summary.

Under metrics, each array item will result in a tile in the detail section.
There are 3 types of metric:

Simple Count - is a simple, total count of a goal
Bucket - is a table view with counts and percentages
Ratio - is a percentage

The properties differ depending on the type of metric chosen - an example of each follows.

Simple Count

A count appears as a single, raw number.

sample-count

For simple, we can either:
Select a total count of times a goal was reached, OR
Filter that count by number of times the goal was reached with a certain property set (optional “field” and “value” properties)

{
  "title": "# Requested",
  "type": "simple",
  "properties": {
    "goal": "Usecase_goal",
    "field": "Usecase", // optional
    "value": "Requested" // optional
  }
}

Goal - The name of the goal we want to query
Field (Optional filter) - The property we are filter on.
Value (Optional filter) - Is the value we are looking for in the above field

Bucket

A bucket tile appears like this:

bucket

An example bucketed count, splitting on the property “Reason” follows:

{
  "title": "# Reason Failed",
  "type": "bucket",
  "properties": {
    "field": "Reason",
    "goal": "Reason_Failed"
  }
}

Field - is the goal property we want to split the table rows on
Goal - is the name of the goal we want to use

Ratio

An example ratio tile:

ratio

A ratio is determined by dividing two goal counts:
% Ratio = [numeratorGoal / divisorGoal]

{
  "title": "Rate",
  "type": "ratio",
  "properties": {
    "numeratorGoal": "Rate_Success",
    "divisorGoal": "Rate_Start"
  }
}

numeratorGoal - A goal name
divisorGoal - A goal name