Get A/B test

Get an A/B test information and results.

This page documents an earlier version of the API client. For the latest version, see Retrieve A/B test details.

Required ACL: analytics

Examples

C#

AnalyticsClient analytics = new AnalyticsClient("YourApplicationID", "YourWriteAPIKey");
ABTest abtest = analytics.GetABTest(42);

Parameters

abTestID · integer · required

Unique identifier of the A/B test.

Response

abTestID · integer

Unique identifier of the A/B test.

addToCartSignificance · number

A/B test significance based on addToCart conversions. Should be greater than 0.95 to be considered significant, no matter which variant is winning.

clickSignificance · number

A/B test significance based on click data. Should be greater than 0.95 to be considered significant, no matter which variant is winning.

configuration · object

Configuration parameters for the A/B test.

Show child attributes

emptySearch · object

Show child attributes

exclude · boolean

Whether empty searches are excluded when calculating A/B test results.

outliers · object

Show child attributes

exclude · boolean

Whether outliers are excluded when calculating A/B test results.

conversionSignificance · number

A/B test significance based on conversion data. Should be greater than 0.95 to be considered significant, no matter which variant is winning.

createdAt · string

Time at which the A/B test has been created. Format: Y-m-d\TH:i:s\Z

endAt · string

Time at which the A/B test will automatically stop. Format: Y-m-d\TH:i:s\Z

name · string

Name of the A/B test.

purchaseSignificance · number

A/B test significance based on purchase conversions. Should be greater than 0.95 to be considered significant, no matter which variant is winning.

revenueSignificance · object

A/B test revenue significance for each currency. This object has currency codes as keys and the significance scores as values.

status · string

Current status of the A/B test.

variants · object[]

The two A/B test variants:

• The base index
• The index to test against

Show child attributes

addToCartCount · integer

Distinct number of converted clicks with the eventSubtype set to addToCart and clickAnalytics set to true.

addToCartRate · number

Add-to-cart rate is based on conversion events that have eventSubType set to addToCart, and the total number of tracked searches within the specified timeframe. The rates are computed as the number of add-to-cart conversion events divided by the number of tracked searches.

averageClickPosition · integer

Average click position for the variant.

clickCount · integer

Distinct click count for the variant.

clickThroughRate · number

Click through rate for the variant.

conversionCount · integer

Distinct conversion count for the variant.

conversionRate · number

Conversion rate for the variant.

currencies · object

Revenue metrics for each currency. Each key is a ISO 4217 currency code. Each value is an object with these properties:

Show child attributes

currency · string

Currency code in ISO 4217 format.

mean · number

Mean revenue for this currency.

revenue · number

Revenue made in this currency.

standardDeviation · number

Standard deviation for the revenue made in this currency.

description · string

Description of the variant.

filterEffects · object

Effects of the outlier or empty search exclusion. Each effect has these properties:

• usersCount: number of users excluded by this filter
• trackedSearchesCount: number of tracked searches excluded by this filter

Example:

JSON

{
  "outliers": {
        "usersCount": "...",
        "trackedSearchesCount": "..."
  },
  "emptySearch": {
        "usersCount": "...",
        "trackedSearchesCount": "..."
    }
}

index · string

Index name of the variant.

noResultCount · integer

Number of tracked searches without any results.

purchaseCount · integer

Distinct number of converted clicks with the eventSubtype set to purchase and clickAnalytics set to true.

purchaseRate · number

The purchase rate is based on conversion events with the eventSubType set to purchase, and the total number of tracked searches within the specified timeframe. The rates are computed as the number of purchase conversion events divided by the number of tracked searches.

searchCount · integer

Total number of searches for this variant, including untracked searches.

trackedSearchCount · integer

Number of tracked searches for this variant. Tracked searches are searches with clickAnalytics enabled.

trackedUserCount · integer

Number of users who searched this variant with at least one search that had clickAnalytics enabled.

userCount · integer

Total number of users who searched on this variant, including untracked users.

Response as JSON

This section shows the JSON response returned by the API. Each API client wraps this response in language-specific objects, so the structure may vary. To view the response, use the getLogs method. Don’t rely on the order of properties—JSON objects don’t preserve key order.

JSON

{
  "abTestID": 78,
  "clickSignificance": 1,
  "conversionSignificance": 0.9937,
  "createdAt": "2018-05-15T17:52:15.644906Z",
  "endAt": "2018-05-17T23:59:59Z",
  "name": "Custom Ranking salesRank Test",
  "status": "stopped",
  "configuration": {
    "outliers": {
      "exclude": true
    }
  },
  "variants": [
    {
      "averageClickPosition": 0,
      "clickCount": 17115,
      "clickThroughRate": 0.19839107906664039,
      "conversionCount": 11508,
      "conversionRate": 0.13339670101658765,
      "description": "",
      "index": "atis-abtest-default",
      "noResultCount": 0,
      "searchCount": 86269,
      "trackedSearchCount": 70365,
      "trafficPercentage": 70,
      "userCount": 55501,
      "trackedUserCount": 44197,
      "filterEffects": {
          "outliers": {
            "usersCount": 0,
            "trackedSearchesCount": 0
          }
       }
    },
    {
      "averageClickPosition": 0,
      "clickCount": 7716,
      "clickThroughRate": 0.20869847452125934,
      "conversionCount": 5129,
      "conversionRate": 0.13872660391647734,
      "description": "",
      "index": "atis-abtest-salesRank",
      "noResultCount": 0,
      "searchCount": 36972,
      "trackedSearchCount": 30459,
      "trafficPercentage": 30,
      "userCount": 22694,
      "trackedUserCount": 21356,
      "filterEffects": {
          "outliers": {
            "usersCount": 1,
            "trackedSearchesCount": 300
          }
       }
    }
  ]
}