Via the Crawler configuration: we provide an option to boost the weight of a page by adding a pageRank field to the extractor (see documentation) directly in the Crawler config.
You can programmatically detect the page you’d like to boost by either:
- Analyzing the URL that is being crawled
- The URL object is available in the parameters of the recordExtractor, see documentation
- Adding some extra informations in your DOM (e.g. a latest meta tag in the head), and retrieve it using the available cheerio instance
Article is closed for comments.