If one variant has significantly more searches, despite having roughly the same number of users, that variant will usually lose the test by a large margin.
What's happening
Algolia A/B tests use a user token provided by your front end to keep track of which bucket - A or B - the user is in so they see the same variant each time.
Sometimes you might want assign an anonymous ID to users until they accept cookies or sign in or for another reason.
To Algolia A/B tests, all these users will appear to be one user who performs many thousands of searches. Usually, these users who didn't accept cookies are less engaged and less likely to convert so they skew the test results.
How to fix it
You have a couple of choices, firstly you can find a better way to assign user tokens, something like random UUIDs in the session, but this can be tricky and users may be using the tokens for something else.
The easiest option is to append the enableABTest:false parameter to all Algolia queries for users who have the anonymous token. This will exclude users who have the anonymous token from the A/B tests.
Getting Ahead
You can check if you are affected by this issue before running proper tests by running a dummy or A/A test.
The easiest way to do this is by setting up a test with a rule context added to the variant, this won't do anything (unless you create rules) so in theory the two variants should perform identically. If you see a large difference in searches as above you know the issue is present.