Estimate the sample size and duration of an A/B test

Given the traffic percentage and the expected effect size, this endpoint estimates the sample size and duration of an A/B test based on historical traffic.

POST /3/abtests/estimate

curl

cURL

curl --request POST \
  --url https://analytics.us.algolia.com/3/abtests/estimate \
  --header 'accept: application/json' \
  --header 'content-type: application/json' \
  --header 'x-algolia-api-key: ALGOLIA_API_KEY' \
  --header 'x-algolia-application-id: ALGOLIA_APPLICATION_ID' \
  --data '{"configuration":{"filters":[{"domain":"abtesting","name":"isOutlier","trackEffects":true,"includes":true}],"minimumDetectableEffect":{"size":0,"metric":"addToCartRate"}},"variants":[{"index":"delcourt_production","trafficPercentage":60,"description":"Current production index"},{"index":"delcourt_production","trafficPercentage":60,"description":"Current production index"}]}'

200

{
  "durationDays": 21,
  "sampleSizes": [
    23415
  ]
}

Required ACL: analytics

Authorizations

x-algolia-application-id · string · header · required

Your Algolia application ID.

x-algolia-api-key · string · header · required

Your Algolia API key with the necessary permissions to make the request. Permissions are controlled through access control lists (ACL) and access restrictions. The required ACL to make a request is listed in each endpoint's reference.

Body

application/json

configuration · estimateConfiguration · object · required

A/B test configuration for estimating the sample size and duration using minimum detectable effect.

Show child attributes

configuration. minimumDetectableEffect · object · required

Configuration for the smallest difference between test variants you want to detect.

Show child attributes

configuration.minimumDetectableEffect. size · number<double> · required

Smallest difference in an observable metric between variants. For example, to detect a 10% difference between variants, set this value to 0.1.

Required range: 0 <= x <= 1

configuration.minimumDetectableEffect. metric · enum<string> · required

Metric for which you want to detect the smallest relative difference.

Available options: addToCartRate, clickThroughRate, conversionRate, purchaseRate, noResultsRate

configuration. filters · object[]

List of metric filters applied to the test population.

Show child attributes

configuration.filters. domain · string · required

Metric domain (for example abtesting, personalization).

Example:

"abtesting"

configuration.filters. name · string · required

Public metric name.

Example:

"isOutlier"

configuration.filters. trackEffects · boolean

Whether the experiment should record the effects of this filter.

configuration.filters. includes · boolean

If true, keep items that match the filter; if false, exclude them.

variants · (Variant · object | Variant with search parameters · object)[] · required

A/B test variants.

Minimum array length: 2

Variant

Show child attributes

variants. index · string · required

Index name of the A/B test variant (case-sensitive).

Example:

"delcourt_production"

variants. trafficPercentage · integer · required

Percentage of search requests each variant receives.

Required range: 1 <= x <= 99

Example:

60

variants. description · string

Description for this variant.

Example:

"Current production index"

Response

200

application/json

OK

durationDays · integer<int64>

Estimated number of days needed to reach the sample sizes required for detecting the configured effect. This value is based on historical traffic.

Example:

21

sampleSizes · integer<int64>[]

Sample size estimates for each variant. The first element is the control variant. Each element is the estimated number of searches required to achieve the desired statistical significance.

Number of tracked searches needed to be able to detect the configured effect for the control variant.