[Shopify] Why is my indexing slow?

Indexing with the app works in two steps:

1. Complete reindexing: this consists in creating a completely new index, by listing all of your products, collections, blog posts and pages.
2. Real-time updating: this relies on Shopify’s webhooks to keep your index up to date.

Complete reindexing

When doing a complete reindexing, we browse through your updated list of products and save them in a temporary index. Once your data is fully uploaded, we overwrite the live index with the temporary, up-to-date index. As a result, your search is always up while you’re adding new products, collections, posts, or pages.

This step can be slow; this is especially the case when your objects use metafields, because we need to run a Shopify API request to retrieve each product. See our metafields section for more information.

Note: Blog posts and pages do not have update webhooks, so we periodically perform a complete reindexing instead.

Real-time indexing

Changes to your Shopify catalogue are immediately processed and stored in your Algolia index: this is called real-time indexing.

Here’s the full lifecycle of a product update:

1. When you save a product in Shopify, it’s added to a Shopify internal queue, where it waits for previous jobs to execute before it can be processed. When it reaches the front of the queue, Shopify loads your updated product into your store. Then, when Shopify has finished processing the job, it forwards the changes to the Algolia Shopify plugin via a webhook.
2. We receive the webhook call, and add an indexing job to our internal queue. Once this job is processed, it forwards the relevant update to the Algolia API.
3. The Algolia API also has an indexing queue, to which we add forwarded jobs from our plugin. Once the job is fully processed, search results return up to date versions of your product.

Both the Shopify and the Algolia API can become bottlenecks if they’re heavily loaded. Both APIs are optimized, so this should be fairly rare.

The main bottleneck will often be our plugin, which, since it’s free, offers limited resources. We’ve done a lot of optimization since our first release, but if you have frequent updates, we might process them slower than you’re sending them to Shopify.

If you think your indexing is stuck, make sure you have no error when opening the app. If you don’t, please contact us and we will investigate!

Last reviewed 16-11-22.