requestOptions

Options for all crawler requests.

• Type: object

The Crawler only accepts headers from the list of allowed headers. The Cookie header is replaced by the cookie fetched in login.

Examples

JavaScript

{
  requestOptions: {
    proxy: 'http://algolia:password@1.1.1.1:8080',
    timeout: 10000,
    retries: 2,
    headers: {
      Authorization: 'Bearer Aerehdf==',
      Cookie: 'session=1234;',
    }
  }
}

Parameters

proxy · string

URL of a proxy server for all crawler requests.

headers · object

Headers to add to all requests.

Supported headers are:

• Accept-Language
• Authorization
• Cookie

retries · number · default: 2

Maximum number of retries for all crawler requests.

timeout · number · default: 30000

Maximum time in milliseconds before canceling the request.