Algolia supports multiple languages
Algolia is language agnostic and matches the text in the search box with the text in the index: this is called textual matching. For example, suppose you have an index with only English text, and a user searches in Japanese. In that case, Algolia won’t return any results because the Latin alphabet doesn’t match Japanese characters. If your users search in Japanese, your index should contain Japanese text. If you want to support multiple languages, the most common solution is to create one index per language. Algolia uses a wide array of natural language techniques, ranging from general, such as finding words or using Unicode, to specific, including distinguishing letters from logograms, breaking down compound words, and using single-language dictionaries for vocabulary. The following is split into several natural language understanding strategies: Some language-based techniques (such as normalization) play an integral role and are performed with every indexing and search operation. These aren’t configurable. Other techniques rely on specialized dictionaries, which facilitate word and word-root detection, and these do come with several configurable options. Depending on the use case, Algolia offers additional techniques (like typo tolerance and Rules) that you can turn on, turn off, or fine-tune. These are also configurable using Algolia’s API settings. For more information, see:Normalization
Algolia performs normalization during indexing and at query time, ensuring consistency in how your data is represented and matched. You can’t globally turn off normalization, but you can turn it off for certain special characters. The normalization process is language-agnostic and applies to all supported languages.What does normalization mean?
- Turns all characters to lowercase
- Removes special characters (diacritics) such as accents, umlauts, and Arabic short vowels. However, you can keep diacritics with the
keepDiacriticsOnCharactersindex setting - Removes punctuation within words, for example, apostrophes
- Manages punctuation between words
- Uses word separators, such as spaces or other characters
- Includes or excludes non-alphanumeric characters (separatorsToIndex)
- Transforms traditional Chinese into modern
Add language specificity using dictionaries
Some of these automated techniques only work in some languages.No automatic language detection
Algolia doesn’t attempt to detect the language of your or your users as they type in queries. Therefore, to benefit from language-specific algorithms, you need to tell Algolia what language you want your records to be interpreted.- If you don’t pick a language, Algolia assumes you want to cover all supported languages. The drawback here is that you create ambiguities by mixing every language’s peculiarities. For example, plurals in Italian are applied to plurals in English, causing problems such as the following: “paste”, the plural of “pasta” in Italian, will also be considered the plural of “pasta” in English, which isn’t the case, as “paste” in English is a word in its own right (to spread).
- It’s okay to mix two or three languages in a single index and specify them in your settings. However, you should prepare your indices and records appropriately. For more on this, refer to the multiple languages tutorial.
Because the default language of an index is all supported languages,
enabling the
removeStopWords
or ignorePlurals
parameters without setting an index’s language will ignore the wrong plurals and remove the wrong stop words.
It’s, therefore, essential to set the query languages of all your indices.Dictionaries
Several language-related methods require the use of dictionaries. With dictionaries, Algolia can apply language-specific, word-based logic to your data and your user’s queries. Algolia maintains separate, language-specific dictionaries for:- Removing stop words
- Detecting pluralized and other declined forms (alternative forms of words due to number, case, or gender)
- Splitting compound words (also known as decompounding)
- Handling Asian logograms (CJK)
- Normalization
- Language-specific configurations
- Customize stop words
- Customize plurals and other declensions
- Customize word segmentation
Typo tolerance and languages
What’s a typo?
- A missing letter in a word, “hllo” → “hello”
- An extraneous letter, “heello” → “hello”
- Inverted letters: “hlelo” → “hello”
- Substituted letter: “heilo” → “hello”
Other spelling errors
Extra or missing spaces and punctuation doesn’t count as typos. Algolia only handles them iftypoTolerance is enabled (set to true, min, or strict). For example:
- Missing spaces between two words is handled by splitting: “helloworld” → “hello world”
- Extra spaces or punctuation is handled by concatenation: “hel lo” → “hello”
Typos as language-dependent
To illustrate the principle, English is a suitable language because it’s phonemic: it uses single characters to represent sounds to form a word. It makes spelling errors possible. Algolia doesn’t support typo tolerance for logogram-based languages (like Chinese and Japanese), as these languages use pictorial characters to represent partial or complete words instead of single letters to represent sounds. For alphabet-based and phonemic languages (like English, French, and Russian), you can configure Algolia in these ways to improve typo tolerance:typoToleranceminWordSizefor1TypominWordSizefor2TyposdisableTypoToleranceOnAttributesdisableTypoToleranceOnWordsallowTyposOnNumericTokens
Turn off typo tolerance and prefix search on specific words
TheadvancedSyntax parameter lets you turn off typo tolerance on specific words in a query by using double quotes. For example, the query “foot problems” is typo tolerant on both query words, while “foot” problems” is only typo tolerant on “problems”.
This parameter also disables prefix searching on words inside the double quotes.