Skip to main content

Algolia DevCon - June 28-29, 2023

Discover the future of the AI Stack with Algolia, be the first to see new platform demos and announcements, and get the most out of Algolia through interactive workshops.

REGISTER FOR FREE DON'T SHOW AGAIN
×

Why can’t the Algolia Crawler extract content from complex PDF files?

  • Updated

This article explains why some PDF files can’t be fully indexed by the Algolia Crawler and how to check whether a PDF’s content is accessible.

Many PDF documents use complex layouts or non-standard structures that differ significantly from HTML pages. To index these files, the Algolia Crawler relies on an Apache Tika Server, which extracts a document’s text and converts it into a simplified HTML format. This process works only when the text in the PDF is accessible and selectable.

If a PDF contains scanned images, restricted text, or inaccessible layers, the crawler may only extract part of the content or fail to extract it entirely.

To check whether a PDF is accessible:

  • Open the PDF in a viewer.

  • Select all content (for example, Ctrl + A or Cmd + A).

  • If only part of the document can be selected, the remaining content is not accessible to text extraction tools.

Limited selection usually indicates one of the following:

  • The PDF is image-based (scanned) rather than text-based.

  • The document has restricted permissions.

  • The PDF was generated in a way that does not preserve selectable text.

If the PDF was originally created in a tool such as Microsoft Word, review how it was exported and ensure it meets accessibility standards. Making the text fully selectable improves both accessibility and crawler extraction.

Helpful resources:

Related articles