Tools / Crawler / APIs / Configuration

Crawler: SafetyChecks

Type: object
Parameter syntax 
safetyChecks: {
  beforeIndexPublishing: {
    maxLostRecordsPercentage: number
  }
}
See code examples

About this parameter

A configurable collection of safety checks to make sure the crawl was successful.

This configuration describes all the checks the Crawler can perform to ensure data is correct. For example, the number of records from one crawl to another.

Examples

Copy 
1
2
3
4
5
6
7

{
  safetyChecks: {
    beforeIndexPublishing: {
      maxLostRecordsPercentage: 10
    }
  }
}

Parameters

Parameter Description
beforeIndexPublishing
type: object
Optional

Checks triggered after the Crawler finishes, and before pushing the records to Algolia into the final index.

beforeIndexPublishing ➔ maxLostRecordsPercentage

Parameter Description
maxLostRecordsPercentage
type: number
Optional

Defines the limit of records difference between the new and the last crawl as a percentage of total records (inclusive).

Default: 10.

Minimum: 0
Maximum: 100.

If the new number of records is less than last number of records * (1 - maxLostRecordsPercentage / 100), the process throws a SafeReindexingError, blocking the Crawler until manual restart.
Previous
login
Next
actions
Did you find this page helpful?