Why does the crawler block reindexing with "The difference between the number of records is too large"

This is a safety check. maxLostRecordsPercentage is the relevant value in the config. If you are getting this error after making changes to your site and not changing the crawler config, that is likely the source of the error. You need to update your config to match your new DOM.