Crawler API

The Crawler API lets you manage and run your crawlers.

Base URL

The base URL for making requests to the Crawler API is:

• https://crawler.algolia.com/api

All requests must use HTTPS.

Availability and authentication

To authenticate your API requests, use the basic authentication header:

• Authorization: Basic <credentials>

Where <credentials> is a base64-encoded string <user-id>:<api-key>.

• <user-id>. The Crawler user ID.
• <api-key>. The Crawler API key.

You can find both on the Crawler settings page in the Algolia dashboard. The Crawler credentials are different from your regular Algolia credentials.

Request format

Request bodies must be JSON objects.

Parameters

Parameters are passed as query parameters for GET requests, and in the request body for POST and PATCH requests.

Query parameters must be URL-encoded. Non-ASCII characters must be UTF-8 encoded.

Response status and errors

The Crawler API returns JSON responses. Since JSON doesn’t guarantee any specific ordering, don’t rely on the order of attributes in the API response.

Successful responses return a 2xx status. Client errors return a 4xx status. Server errors are indicated by a 5xx status. Error responses have a message property with more information.

Version

The current version of the Crawler API is version 1, indicated by the /1/ in each endpoint’s URL.