How to create Multilingual Recommended System

Today, E-commerce thrives on personalization but expanding in global market is challenging because of language barriers.

Creating multilingual recommended system ensures a more tailored shopping experience for all users.

Why does your Business need the Multilingual Recommended System?

Language diversity can make or break a global user experiences. It ensure that every customer can understood, regardless of their native language.

This approach boosts engagement, increases conversions, and builds loyalty across different markets.

Building Multilingual Recommended System

Collecting products data for Multilingual Recommended System

• Gather all product data, with their title, and text descriptions.
• Ensure the dataset covers all the languages your target audience speaks.
• Example: A T-shirt’s description is available in English, Hindi, and Polish, or multiple products in different languages, catering to diverse customers.

Select right embedding model

• Selecting the right embedding model is very important. Model should support multilingual.
• Multilingual embedding models are trained on multiple language data.
• Two similar products but have been written in different languages, their multilingual models generate similar context embeddings.
• Some multilingual embeddings models are openai – text-embedding-3-large, cohere embed-multilingual-light-v3.0, embed-multilingual-v3.0, etc.

Creating products embeddings

Example, Generating embeddings from an openai

from openai import OpenAI
client = OpenAI()

def get_embedding(text, model="text-embedding-ada-002"):
    text = text.replace("\n", " ")
    return client.embeddings.create(input=[text], model=model).data[0].embedding

product_embedding = get_embedding("""
Wireless Noise-Canceling Headphones

Experience superior sound quality with our wireless noise canceling headphones.
Designed for comfort,durability and they offer up to 30 hours of battery life and seamless bluetooth connectivity.
""")

The output is a vector representation:

[
  -0.006929283495992422,
  -0.005336422007530928,
  ... (truncated for brevity)
]

Save embeddings in VectorDB

Store the generated embeddings in a vector database with metadata such as product ID, language, and category. Popular VectorDB include:

• ChromaDB
• Pinecone
• Milvus

Recommending Products

Step 1: Calculate Similarity

When a user views a product, API generates embeddings for its title and description.

Query the vector database using these embeddings to find similar products based on cosine similarity or other distance metrics.

Step 2: Generate Results

The vector database retrieves products with the most similar embeddings, providing recommendations.

Even if the recommended products are in different languages, the multilingual model ensures they share the same contextual meaning.

Benefits of a Multilingual Recommender System

• Seamless User Experience: Users see recommendations in their preferred language.
• Global Reach: Cater to diverse markets without language limitations.
• Increased Sales: Personalized suggestions drive higher conversions.
• Efficient Scaling: Multilingual embeddings reduce the need for separate models per language.

You can also check out the Magento 2 Recommendation System. It works with a multilingual embeddings model and supports dynamic models.

This is a pair of wireless headphones with the product description in English.

The recommended products are available in different languages: the first in French, the second in Spanish, and the last in Italian.

Our recommendation system suggests products even in different languages.

Conclusion

A multilingual recommender system is vital for any e-commerce business targeting global markets.

By leveraging multilingual embeddings, vector databases, and similarity metrics, you can create an inclusive and effective recommendation engine that resonates with users worldwide.

Start your Artificial Intelligence development with Webkul.