Internally, Query Categorization currently operates in one of the following “hidden” modes:
MODE 1
An index has enough events to make relevant predictions
- You’ll see predictions for 1000s of queries in the dashboard
- No warning
MODE 2
An index has some events, but not enough events to make relevant predictions and so Query Categorization falls back to a “degraded mode”
- Instead of displaying no predictions, we internally fallback on a more naive approach that works with less events but only on the head
- You’ll see far fewer queries with predictions
- We display the warning "Limited number of queries with predictions" in the dashboard
MODE 3
There are few or no events
- There are no predictions
- We display an error message.
Fluctuations in traffic/queries can result in a scenario in which some days Query Categorization operates in Mode 1 (above the thresholds to make relevant predictions), while on some other days it’s below the threshold and operates in Mode 2.
This threshold is not a specific number of queries or events. Basically, we consider the last 90 days of traffic and feed it to the Query Categorization model. If the model confidence metrics are too low after training is complete, we fall back on Mode 2.
Even with consistent traffic/number of events, training can push the predictions into Mode 2 if the traffic is split too thin across too many unique searches, or if for each unique query, the records that receive clicks/conversions belong to categories that have no relationship in the category tree.