Indexing

Learn how the indexing works in the Algolia extension for Magento Open Source and Adobe Commerce

To provide fast and relevant search, the Algolia extension takes your products, categories, and pages from your store and transforms them into searchable data. This process is called indexing.

You can index your data through these methods:

• Automatically with an indexing queue
• Manually from the Magento console
• Manually on the command-line

Explore related content in the Algolia Academy

Algolia for Magento Open Source and Adobe Commerce

If you encounter any issues with indexing, or the queue, see Troubleshooting data and indexing issues.

To provide the best search experience for your users, the extension automatically keeps all your data up to date, offering two indexing mechanisms in Magento:

• Section reindex. An entire section of the catalog (products, categories, pages) is uploaded to Algolia’s servers and reindexed.
• Single item reindex. A single resource (product, category, page) is uploaded to the servers and reindexed. Single item reindexing happens when a resource is updated.

By default, the indexing operations run synchronously, and you must wait until the indexing process finishes before you can continue. This is inconvenient and can have unexpected side effects. Therefore, you should instead use the indexing queue, which processes all index operations in the background and avoids some sources of errors.

Enable indexing to Algolia

The first thing to do is to ensure that indexing to Algolia is enabled. Go to Stores > Algolia Search > Indexing Manager > Enable Indexing and verify that Yes is selected.

Section reindex with an indexing queue

The indexing queue uses temporary indices to reindex products. Instead of updating the production index directly, Algolia creates a copy and replaces the production index only when the update is complete.

This approach has several advantages:

• Higher reindexing speed
• No inaccuracies with deleted products
• Fewer indexing operations

Changes to the index will be visible once the temporary index completely replaces the production index.

Section reindex without an indexing queue

Without the indexing queue, reindexing the complete catalog is synchronous: everything has to wait until the process completes.

Processing large indices synchronously may trigger PHP timeouts.

Without a queue, reindexing takes more time and resources. It’s also less reliable: if products are updated while the reindexing is in process, the data on Algolia’s servers may not be up to date.

Enable the indexing queue for any full reindexing, especially if you have large catalogs.

Automatic indexing

The Algolia extension for Magento sends every change and deletion on products or categories to Algolia’s servers to keep all data up to date.

Manual indexing

To index your data manually, first turn off the indexing queue: set Stores > Configuration > Algolia Search > Indexing Queue/Cron > Enable Indexing Queue to No in the Magento dashboard.

You can now use the command-line to send manual updates to Algolia’s server. For example, to reindex all products:

Extension version 3.16 and higher

php path/to/magento/bin/magento algolia:reindex:products

Use the same command for all other indices created by the extension:

Command (version 3.16 and higher)	Command (version 3.15.x and lower)	Action
algolia:reindex:all	Not available	Reindex all entities
algolia:reindex:products	indexer:reindex algolia_products	Reindex all products
algolia:reindex:categories	indexer:reindex algolia_categories	Reindex all categories
algolia:reindex:pages	indexer:reindex algolia_pages	Reindex all CMS pages
algolia:reindex:suggestions	indexer:reindex algolia_suggestions	Reindex all query suggestions
algolia:reindex:additional_sections	indexer:reindex algolia_additional_sections	Reindex all additional sections
algolia:reindex:process_queue	indexer:reindex algolia_queue_runner	Process jobs in the indexing queue
algolia:reindex:delete_products	indexer:reindex algolia_delete_products	Remove inactive products from Algolia indices

Manual indexing with the indexing queue

To reindex with the indexing queue turned on, manually trigger the queue runner, or let the cron job process it when your data is updated. To do this, and add all your products to the indexing queue, run:

Extension version 3.16 and higher

php path/to/magento/bin/magento algolia:reindex:products
PROCESS_FULL_QUEUE=1 php path/to/magento/bin/magento algolia:reindex:process_queue

You should only implement this as a troubleshooting activity, not as a regular task.

Indexing Manager

Extension version 3.16.0 introduces the Indexing Manager, which includes a new configuration section and a dedicated page in the Magento admin for triggering full entity indexing.

Configuration

To configure the Indexing Manager, go to Stores > Configuration > Algolia Search > Indexing Manager.

From extension version 3.16.0, the legacy Magento indexers can be turned off for full indexing. However, the extension still uses them for incremental updates triggered by Magento’s materialized views or individual entity save events.

For earlier versions of the extension, Magento indexers handle both full and incremental indexing.

For full reindexing, run the new CLI commands or use the Indexing Manager to run a full reindex .

Full reindexing with the Indexing Manager

To reindex, go to Stores > Indexing Manager.

This page consists of two sections:

• At the top, you can select specific Entities and Store(s) for reindexing.
• Below, there’s a list of indices managed by the extension (linked to their Algolia dashboard entry).

Index products

It’s essential for your business to have exact and up-to-date product data in your search.

Full reindexing

To fully reindex all your products, run:

Extension version 3.16 and higher

php path/to/magento/bin/magento algolia:reindex:products

Indexable products

To prevent too many indexed products and save indexing operations, Algolia only indexes products that actually appear in the online store.

Algolia only indexes products that are:

• Visible—either in the catalog, the search, or both
• Enabled
• Not deleted
• In stock—unless Magento is configured to show out-of-stock products, too.

If you miss a product in your search, make sure the product meets all four criteria.

For troubleshooting information, see Troubleshooting data, indexes, and queues

Missing products

If search can’t find some products, go the Algolia dashboard, select the index that contains your products and search there.

• If you can find the product, confirm that is has the correct visibility setting. If products are only visible in the catalog, they won’t show up while searching. They will only show up on the InstantSearch page (in the category section).

• If you can’t find the product, it hasn’t been indexed. Make sure it adheres to the indexing requirements. If the product meets the requirements, Reindex it.

If the problem persists, turn on logging and investigate the log files to see what’s going on during the reindexing process.

Price indexing

When creating product records for the corresponding index, Algolia extracts pricing information from Magento’s core “Product Price” index.

The Product Price index must be up-to-date each time the Algolia indexer runs.

Review the state of the Product Price index from the Magento admin panel or the Magento CLI.

• Magento admin panel, under System > Index Management

  

• Magento CLI, identified as catalog_product_price, by running: bin/magento indexer:status

  

If the Product Price index is invalid or is missing records, the Algolia indexer can’t sync the related products.

If products are missing on your storefront, even after confirming that they meet the criteria for visibility, this could be the result of missing Product Price index records. For details of how to resolve this issue, see the troubleshooting guide.

Searchable attributes

You can configure which attributes should be searched when users type their queries. To configure the list of searchable attributes, go to Stores > Configuration > Algolia Search > Products.

In the Products tab, you can set each attribute as searchable, retrievable, or ordered. By default, all attributes are set to be searched as unordered. This is usually the best choice, and you shouldn’t change it without specific reasons.

For more information, see Configuring searchable attributes the right way

Default indexed attributes

Some attributes are always indexed, regardless of your configuration. These attributes aren’t all searchable but can be used for filtering, sorting, customizing the ranking, and building the results page.

These product attributes are always indexed:

Attribute	Description
name	Name
url	URL
visibility_search	Visibility in the search
visibility_catalog	Visibility in the catalog
categories	Categories, formatted as a tree path
categories_without_path	Categories, without the tree path
thumbnail_url	Thumbnail image
image_url	Main image
in_stock	Stock availability
price	Price
type_id	Type (simple, configurable, bundled, etc.)

image_url start with /pub

This error can occur when your app has the pub/ directory as the root directory for the Magento installation. This is usually considered a good practice, but it has some drawbacks. The extension can’t know which directory is the root folder for Magento, so it always assume the base directory is your root folder.

The best way to handle this problem is to make your server ignore /pub in the URL.

If it’s not possible to change the server configuration, a setting is provided in Stores > Configuration > Algolia Search > Advanced > Remove /pub/ from image URLs. When this setting is changed, reindex the catalog.

Facets

Facets are attributes used to filter search results. Common facets include price, color, categories, and brand. You may need to adjust the facets of the products you sell and how your users search for these products.

For each facet, you can specify:

• The attribute
• The label - this will be displayed above the filter
• The facet type

Select one of the modifiers from the Options drop-down menu to make facet values searchable, not searchable, or filterable (not “facetable”).

Use the filterOnly modifier if you only want to use the filtering feature. This will help optimize search speed and reduce index size.

You can attach facets to rules. If you set Create Query rule to Yes, Algolia uses the attribute as a filter. Suppose you create a rule for the color attribute and users search for products with a specific color, for example, “red shorts”. In that case, Algolia filters all results with “red” as the color attribute.

When a rule is attached to an attribute, it’s applied to both the Autocomplete menu and InstantSearch results page.

Rules are only available on Grow and Premium plans.

By default, the extension provides facets on the price, categories, and color attributes. Any numeric attribute, for example, price, is shown as a slider in the user interface.

Attributes specified as facets are automatically indexed as retrievable but not searchable. You don’t need to add them manually to the searchable attributes.

How to add numeric attributes as non-castable

To access the configuration page for Algolia in your Magento project, go to Stores > Algolia Search > Credentials and Basic Setup > Advanced in the Admin panel.

From there, you can specify which text fields or non-castable attributes to include under Product numeric textfield attributes that should not be converted into a numeric field by Magento.

‘No’ values in filters

If a product doesn’t have any value assigned to an attribute, the value of this attribute will be automatically set to ‘No’ by Magento. To turn off indexing for ‘No’ values, set the Index empty value setting in your searchable attributes to No for the attributes you want to ignore ‘No’ values for.

Changing any index setting requires a full reindex of all data.

Sorting strategies

Sorting is only available on the InstantSearch Results page. By default, the Algolia Magento extension uses the exhaustive sorting strategy with Standard Replicas, but you can switch to the relevant sorting strategy using virtual replicas.

When searching for products, users may expect multiple ways to sort the result set. For example, they want to sort by relevance, popularity, price, or date.

The default sorting strategy when searching is sorting by relevance. Any other sorting strategy needs to be defined in the Sort Settings.

For each strategy, define an attribute, sort order (ascending or descending), and label.

You can also specify whether to create a virtual replica (instead of a standard replica) for the sorting strategy.

By default, there are three sorting strategies:

1. From lowest price to highest price
2. From highest price to lowest price
3. From newest to oldest

If you’ve chosen to use standard replicas, each sorting strategy creates a new standard replica index, which increases the number of records used by your plan.

Attributes configured for sorting are automatically indexed as retrievable but not searchable. You don’t need to add them manually to the searchable attributes.

Replicas are configured in Algolia whenever you save changes to the sorting configuration in your store’s Admin. If a sorting attribute is deleted from the configuration in your store’s Admin, the corresponding replica index will be deleted in Algolia.

If you experience issues with your replica configuration in Algolia or can’t save changes from your store’s Admin, refer to the Troubleshooting guidance.

You can also apply sorting strategies to your InstantSearch results pages in your store’s Admin with Merchandising Studio. With this method, you can add rules that apply alternative sorting strategies when certain conditions are met. The extension doesn’t manage these indices.

Number of records created

To provide fast results, Algolia pre-computes a part of the order at indexing time. This means only one sorting is available for a single index. Each additional sort will result in an additional index. Sorting indices may take up significantly increased storage depending on whether standard (default) or virtual replicas are enabled for the corresponding sorting attribute configuration.

In Magento, this means that by default these indices are created:

• One per store
• One per store per additional sort order (such as by price or by date)

To calculate the amount of records this generates, take a look at the following example:

A Magento app has two stores (for two languages) which both have 100 products. Both stores have additional sorts on price and date, resulting in a total of four additional sorts.

Using this scenario, you would need to index:

Sort	Records	Total
Default sorting	100×2	200
Custom sorting	100×2×4	800
Total records		1000

If the customer group feature is enabled, each customer group will result in an extra index on the pricing sorting. Following the preceding example, but with a total of 5 customer groups:

Sort	Records	Total
Default sorting	100×2	200
Pricing sorting	100×2×2×5	2000
Custom sorting	100×2×2	400
Total records		2600

The number of indexed records can be lowered by either removing some sorting strategies or by converting them to using virtual replicas. (Virtual replicas do require some additional storage but this is marginal relative to the number of records.)

Sorts can be configured in the extensions’ administration panel:

Switch to virtual replicas

Virtual replicas are not available on the Grow plan.

To configure Virtual Replicas for Sorting, go to Stores > Configuration > Algolia Search > InstantSearch Results Page > Sorting.

In the grid, enable virtual replicas for each desired attribute by setting Enable Virtual Replica? to Yes.

When you switch a sorting attribute to a virtual replica, Algolia deletes the standard replica and creates a virtual replica index in its place. If you reset this value to No, the opposite happens.

Virtual replicas slightly increase the primary index’s size (less than 10 MB per million records), but they don’t affect the number of records. For more information, see Standard and virtual replicas.

You can enable up to 20 virtual replicas per index. The extension informs you if you exceed this limit.

Scoped configuration

The extension creates a product index for each Algolia-enabled store (or “store view”) in your Magento instance. To keep within the virtual replica limits on a single index, use the store view selector to configure your sorting attributes for each store independently.

Choose the website or store view where you wish to apply the desired changes.

For a website scope, remove the Use Default check mark and configure your sorting attributes accordingly:

For a store scope, remove the Use Website check mark:

For more information, see Adobe’s documentation.

Index settings

Configure these product index settings in the Magento dashboard:

• searchableAttributes
• customRanking
• unretrievableAttributes
• attributesForFaceting
• maxValuesPerFacet
• removeWordsIfNoResults

Any changes in the Algolia dashboard override these settings until you perform a full reindex or make updates to your Algolia configuration from the Magento dashboard.

Manage additional index settings in the Algolia dashboard.

To set default values from Magento, configure this under “Advanced Algolia indices settings”.

To change the settings programmatically, you can hook into the algolia_products_index_before_set_settings event provided by the extension. For more information, see Custom backend events.

Index from the Magento products grid

Version 3.16.0 introduced new features to the Magento products grid:

• You can select products in the grid and reindex them with the Actions drop-down menu in the top-left corner: select Reindex to Algolia.
• You can trigger full product reindexing by clicking Reindex All Products to Algolia in the main toolbar at the top of the page.

Index categories

To keep the number of records and indexing operations as low as possible, Algolia only indexes active categories. To show all categories in the Autocomplete menu and InstantSearch Results page, set Show categories that are not included in the navigation menu to Yes.

Full reindexing

Extension version 3.16 and higher

php path/to/magento/bin/magento algolia:reindex:categories

Searchable attributes

You can configure which attributes should be searched when users type their queries. To configure the list of searchable attributes, go to Stores > Algolia Search > Categories.

In the Categories tab, you can set each attribute as searchable, retrievable, or ordered. By default, all attributes are set to be searched as unordered. This is usually the best choice, and you shouldn’t change it without specific reasons.

For more information, see Configuring searchable attributes the right way.

Default searchable category attributes

Some attributes are always indexed, regardless of your configuration. These attributes aren’t all searchable but can be used for filtering, sorting, customizing the ranking, and building the results page.

These category attributes are always indexed:

Attribute	Description
name	Name
url	URL
path	Path (parent categories)
level	Level in the category tree
include_in_menu	Visibility in the menu
_tags	Filled automatically by the extension
popularity	Popularity
product_count	Amount of products

Index settings

Configure these category index settings in the Magento dashboard:

• searchableAttributes
• customRanking
• unretrievableAttributes

Any changes in the Algolia dashboard override these settings until you perform a full reindex or make updates to your Algolia configuration from the Magento dashboard.

Manage additional index settings in the Algolia dashboard.

To set default values from Magento, configure this under “Advanced Algolia indices settings”.

To change the settings programmatically, you can hook into the algolia_categories_index_before_set_settings event provided by the extension. For more information, see Custom backend events.

Index pages

To turn indexing of content-management-system (CMS) pages on or off, go to Stores > Algolia Search > Indexing Manager > Enable Pages Index. When set to Yes the Algolia extension automatically indexes CMS pages, allowing users to search for pages in the Autocomplete menu. All active pages are indexed by default.

You can exclude specific pages, like error pages, so they don’t appear in the search results.

Turn off this indexing of pages in Additional Sections.

Full reindexing

Extension version 3.16 and higher

php path/to/magento/bin/magento algolia:reindex:pages

Searchable attributes

You can’t configure the searchable attributes for pages in the Magento dashboard. However, you can change them programmatically, by hooking into the algolia_after_create_page_object event provided by the extension. For more information, see Custom backend events.

Default searchable page attributes

These attributes are always indexed and aren’t all searchable. You can use them for filtering, sorting, custom ranking, and building the search results page.

These page attributes are always indexed:

Attribute	Description
name	Name
url	URL
slug	Slug
content	Content

Any records larger than your plan’s limit won’t be indexed. In this case, only the page’s name would be searchable.

Index settings

The following page index settings are always set and can’t be changed through the extension’s interface:

• searchableAttributes: unordered(slug), unordered(name) and unordered(content)
• attributesToSnippet: content:7

Any changes in the Algolia dashboard override these settings until you perform a full reindex or make updates to your Algolia configuration from the Magento dashboard.

Manage additional index settings in the Algolia dashboard.

To set default values from Magento, configure this under “Advanced Algolia indices settings”.

You can also change the settings programmatically, by hooking into the algolia_pages_index_before_set_settings event provided by the extension. For more information, see Custom Backend Events.

Index with the Magento pages grid

Version 3.16.0 introduced new features to the Magento pages grid:

• Users can now select pages in the grid and trigger a reindex for them with the Actions drop-down menu on the top-left corner by selecting Reindex to Algolia.
• Users can now trigger a full pages reindexing by clicking Reindex All Pages to Algolia in the main toolbar on top of the page.

Index suggestions

Magento stores every query, the number of results, and the number of searches in the catalogsearch_query table in the database.

Magento only stores backend searches. As-you-type searches aren’t stored.

Algolia’s Magento extension lets you index queries that are performed regularly. In the settings, you can filter relevant queries, for example, by the minimum number of results or minimum popularity.

To turn this behavior on or off, go to Stores > Configuration > Algolia Search > Indexing Manager > Enable Query Suggestions Index.

With these options enabled, the extension processes backend searches. Since the data in catalogsearch_query is also updated, the queries in this table become more relevant over time.

By default, suggestions aren’t indexed. When enabling the indexing of suggestions, you need to trigger a manual reindex. Another way to start the indexing of suggestions is by adding a recurring job to the cron table:

Extension version 3.16 and higher

1 * * * * php path/to/magento/bin/magento algolia:reindex:suggestions

Instead of Magento suggestions, you can also use Algolia’s Query Suggestions feature. For more information, see Algolia Query Suggestions.

Full reindexing

Extension version 3.16 and higher

php path/to/magento/bin/magento algolia:reindex:suggestions

Searchable attributes

You can’t configure searchable attributes for suggestions in the Magento dashboard. However, you can change them programmatically by hooking them into the algolia_after_create_suggestion_object event provided by the extension. For more information, see Custom backend events.

Default searchable query attributes

These attributes are always indexed and aren’t all searchable. You can use these attributes for filtering, sorting, custom ranking, and building the search results page.

These query attributes are always indexed:

Attribute	Description
query	Value
number_of_results	Number of results
popularity	Number of searches
updated_at	Last update timestamp

Index settings

The following query suggestion index settings are always set and can’t be changed in the Magento dashboard:

• searchableAttributes: unordered(query)
• customRanking: desc(popularity), desc(number_of_results)
• typoTolerance: false
• attributesToRetrieve: query
• removeWordsIfNoResults: lastWords

Any changes in the Algolia dashboard override these settings until you perform a full reindex or make updates to your Algolia configuration from the Magento dashboard.

Manage additional settings in the Algolia dashboard.

To set default values from Magento, configure this under “Advanced Algolia indices settings”.

To change the settings programmatically, you can hook into the algolia_suggestions_index_before_set_settings event provided by the extension. For more information, see Custom backend events.

Index additional sections

The Autocomplete menu lets you display other sections from attributes, such as colors and brands.

For this feature to work, you must enable the InstantSearch Results Page.

The attributes used for the additional sections have to be set as attributes for faceting.

Extension version 3.16 and higher

php path/to/magento/bin/magento algolia:reindex:additional_sections

Searchable attributes

Manage additional index settings in the Algolia dashboard. It’s also possible to change the settings programmatically, by hooking into the algolia_additional_sections_index_before_set_settings event provided by the extension.

For more information, see Custom Backend Events.

Default searchable attributes

These attributes are indexed by default and aren’t all searchable. They can be used for filtering, sorting, customizing the ranking and building the results page.

The attributes that are always indexed:

Attribute	Description
value	The attributes’ value, for example, Red, XL, Nike.

Index settings

The following additional section index settings are always set and can’t be changed through the extension’s interface:

• searchableAttributes: unordered(value)

Any changes in the Algolia dashboard override these settings until you perform a full reindex or make updates to your Algolia configuration from the Magento dashboard.

Manage additional index settings in the Algolia dashboard.

To set default values from Magento, configure this under “Advanced Algolia indices settings”.

It’s also possible to change the settings programmatically, by hooking into the algolia_additional_sections_index_before_set_settings event provided by the extension. For more information, see Custom backend events.

Remove inactive products

To remove products from your Algolia index, use the algolia_delete_products indexer:

Extension version 3.16 and higher

php path/to/magento/bin/magento algolia:reindex:delete_products

This indexer removes all products from Algolia indices that you don’t want to include in your search. This can be useful when you delete or deactivate products directly in Magento’s database, and the extension could reindex and remove them regularly.

The reindexer doesn’t delete products from your Magento database. Products are only removed from Algolia indices.

Incorrect links

Sometimes, links of the Magento installation can show up with an additional /magento/ in the URL. Most likely, this is because web server rewrites are turned off. Go to Stores > Configuration > General > Web > Search Engine Optimization and set the field Use Web Server Rewrites to Yes. When this setting is updated, a full reindex is required for the changes to take effect.