Optional filters

Unlike filters, optional filters don’t remove records from your search results when your query doesn’t match them. Instead, they divide your records into two sets: the results that match the optional filter and those that don’t.

Use this to boost results you want to show before others or “bury” results you want to show last.

Filter scoring

You can add extra nuance to your ranking by specifying scores for different optional filters. For example, you want to display your matching products in the following order:

• Apple products first
• Samsung products second
• Huawei products last

In this scenario, products from any other brand appear after Apple and Samsung products and before Huawei products. You can achieve this by using scored optional filters to boost Apple and Samsung products and negative filters to bury Huawei products.

1
2
3
4
5
6
7
$index->search('phone', array(
    'optionalFilters' => array(
        'brand:Apple<score=3>',
        'brand:Samsung<score=2>',
        'brand:-Huawei',
    )
));

1
2
3
4
5
6
7
index.search('phone', {
    optionalFilters: [
      'brand:Apple<score=3>',
      'brand:Samsung<score=2>',
      'brand:-Huawei',
    ]
  })

1
2
3
4
5
6
7
index.search('phone', {
  optionalFilters: [
    'brand:Apple<score=3>',
    'brand:Samsung<score=2>',
    'brand:-Huawei',
  ],
});

1
2
3
4
5
6
7
index.search("phone", {
    "optionalFilters": [
        "brand:Apple<score=3>",
        "brand:Samsung<score=2>",
        "brand:-Huawei",
    ],
})

1
2
3
4
5
6
7
8
9
index.search(query: Query("phone")
  .set(\.optionalFilters, to: ["brand:Apple<score=3>",
                               "brand:Samsung<score=2>",
                               "brand:-Huawei"])
) { result in
  if case .success(let response) = result {
     print("Response: \(response)")
  }
}

1
2
3
4
5
6
7
8
9
10
val query = query("phone") {
    optionalFilters {
        and {
            facet("brand", "Apple", score = 3)
            facet("brand", "Samsung", score = 2)
            facet("brand", "-Huawei")
        }
    }
}
index.search(query = query)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
index.Search<Contact>(new Query {
    SearchQuery = "phone",
    OptionalFilters = new List<List<string>> {
        new List<string> {
            "brand:Apple<score=3>",
            "brand:Samsung<score=2>",
            "brand:-Huawei"
        }
    }
});

// Asynchronous
await index.SearchAsync<Contact>(new Query {
    SearchQuery = "phone",
    OptionalFilters = new List<List<string>> {
        new List<string> {
            "brand:Apple<score=3>",
            "brand:Samsung<score=2>",
            "brand:-Huawei"
        }
    }
});

1
2
3
4
5
6
7
8
9
List<List<String>> optionalFilters = Collections.singletonList(
    Arrays.asList("brand:Apple<score=3>", "brand:Samsung<score=2>", "brand:-Huawei"));
Query query = new Query("phone")
    .setOptionalFilters(optionalFilters);

index.search(query);

// Asynchronous
index.searchAsync(query);

1
2
3
4
5
6
7
8
res, err := index.Search(
  "phone",
  opt.OptionalFilterAnd(
    "brand:Apple<score=3>", 
    "brand:Samsung<score=2>", 
    "brand:-Huawei",
  ),
)

1
2
// Creating Rules isn't currently possible with this client.
// Please use the dashboard instead.

When you don’t specify a score for an optional filter, it defaults to 1.

For performance reasons, you shouldn’t use filter scoring on searches that may return more than 100,000 results.

Score calculation

The engine has three ways of calculating scores:

• Calculate the sum of optional filters
• Calculate the maximum score for optional filters
• A combination of both.

By default, the engine sums the score of each optional filter. The engine determines this score by taking the maximum score of each optional filter.

Simple optional filters

Here’s an example of how the engine calculates an optionalFilters score:

1
2
3
4
5
6
$index->search('', array(
    'optionalFilters' => array(
        'brand:Apple<score=2>',
        'type:tablet'
    )
));

1
2
3
4
5
6
index.search('', {
    optionalFilters: [
      'brand:Apple<score=2>',
      'type:tablet'
    ]
  })

1
2
3
index.search('', {
  optionalFilters: ['brand:Apple<score=2>', 'type:tablet'],
});

1
2
3
4
5
6
index.search("phone", {
    "optionalFilters": [
        "brand:Apple<score=2>",
        "type:table",
    ],
})

1
2
3
4
5
6
7
8
index.search(query: Query("phone")
  .set(\.optionalFilters, to: ["brand:Apple<score=2>",
                               "type:Tablet"])
) { result in
  if case .success(let response) = result {
     print("Response: \(response)")
  }
}

1
2
3
4
5
6
7
8
9
val query = query {
    optionalFilters {
        and {
            facet("brand", "Apple", score = 2)
            facet("type", "tablet")
        }
    }
}
index.search(query = query)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
index.Search<Contact>(new Query {
    SearchQuery = "phone",
    OptionalFilters = new List<List<string>> {
        new List<string> {
            "brand:Apple<score=2>",
            "type:table"
        }
    }
});

// Asynchronous
await index.SearchAsync<Contact>(new Query {
    SearchQuery = "phone",
    OptionalFilters = new List<List<string>> {
        new List<string> {
            "brand:Apple<score=2>",
            "type:table"
        }
    }
});

1
2
3
4
5
6
7
8
9
10
List<List<String>> optionalFilters = Collections.singletonList(
    Arrays.asList("brand:Apple<score=2>", "type:tablet"));

Query query = new Query()
    .setOptionalFilters(optionalFilters);

index.search(query);

// Asynchronous
index.searchAsync(query);

1
2
3
4
5
6
7
res, err := index.Search(
  "",
  opt.OptionalFilterAnd(
    "brand:Apple<score=2>",
    "type:tablet>",
  ),
)

1
2
// Creating Rules isn't currently possible with this client.
// Please use the dashboard instead.

The engine starts by calculating the maximum score for each optional filter you send. In the preceding example, the engine first checks if a record’s brand attribute has Apple as its value. If this is the case, the record gets a score of 2 on this filter and 0 otherwise. Finally, it checks the type attribute to see if it matches tablet and assigns a score of 1.

In the following records, the first record matches both the brand:Apple<score=2> and type:tablet optional filters, which score 2 and 1 (for a total score of 3). However, the second record only matches the brand:Apple<score=2> optional filter, so the record gets a total score of 2. Therefore, the iPad Pro record shows up higher than the iPhone one.

1
2
3
4
5
6
7
8
9
10
11
12
[
  {
    "name": "iPad Pro",
    "brand": "Apple",
    "type": "tablet"
  },
  {
    "name": "iPhone 11",
    "brand": "Apple",
    "type": "phone",
  }
]

With these optional filters, the engine ranks results as follows:

1. Apple tablets (score: 3)
2. Other Apple products (score: 2)
3. Tablets by other brands (score: 1)
4. Other products by other brands (score: 0)

Complex optional filters

The engine calculates the maximum score for each element of your optional filters, so you can combine filters to perform more complex score calculations.

For example, you want to promote red and blue products but avoid over-promoting products that are both red and blue. To do so, use the following optional filters:

1
[["color:red<score=2>", "color:blue"], ["type:-jeans"]]

And the following record:

1
2
3
4
5
{
  "name": "Long-sleeved shirt",
  "type": "shirt",
  "color": ["red", "blue"]
}

1. The engine calculates the maximum score for the first optional filter. In this case, since it’s an array (["color:red<score=2>", "color:blue"]), the engine checks all nested optional filters and uses the highest matching score.
2. Since the record contains red and blue colors, it scores 2 (match on red) and 1 (match on blue). The engine uses the highest matching score instead of a sum, meaning this record gets a score of 2 on this filter (not 3).
3. The engine matches the second optional filter (type:-jeans). The record also matches (it’s not “jeans”), so it scores 1. Adding this to the previous optional filter’s score, the total score for this record is 3.

With these optional filters, the engine ranks results as follows:

1. Products that have “red” listed as color and aren’t jeans (score: 3)
2. Jeans that have “red” listed as color (score: 2), and products that have “blue” listed as color but aren’t jeans (score: 2)
3. Jeans that have “blue” listed as color (score: 1), and products that aren’t jeans (score: 1)
4. Jeans in colors other than “red” and “blue” (score: 0)

When several records have a similar score, the engine ranks them according to the ranking formula, as with any other search.

sumOrFiltersScores

The previous example used filter scoring to boost products that are either red or blue and not jeans. Records that are both red and blue are also prevented from getting a higher score than other records.

You can change the engine’s behavior by adding the sumOrFiltersScores parameter to a search. This makes the engine add all scores together, regardless of the structure of your filters.

In the preceding example, this means that products that are both “red” and “blue” would show up even higher:

1. Products that have “red” and “blue” listed as color and aren’t jeans (score: 4)
2. Products that have “red” and “blue” listed as color and are jeans (score: 3)
3. Jeans that have only “red” listed as color (score: 2), and products that have only “blue” listed as color and aren’t jeans (score: 2)
4. Jeans that have only “blue” listed as color (score: 1), and products that aren’t jeans (score: 1)
5. Jeans in colors other than “red” and “blue” (score: 0)

The sumOrFiltersScores parameter only changes the scoring for OR optional filters, which you create with a nested array of optional filters (optionalFilters: [["attr:val"]]). The engine always sums AND filter scores (optionalFilters: ["attr:val1", "attr:val2"]).

Performance

Optional filters are powerful but have some significant performance limitations. Here are a few recommendations:

• Define optionalFilters with the filterOnly modifier, which tells the engine to treat the attribute as a filter instead of a facet. For example:

1
2
3
4
5
$index->setSettings([
  'attributesForFaceting' => [
    "filterOnly(brand)"
  ]
]);

1
2
3
4
5
index.set_settings({
  attributesForFaceting: [
    "filterOnly(brand)"
  ]
})

1
2
3
4
5
6
7
index.setSettings({
  attributesForFaceting: [
    "filterOnly(brand)"
  ]
}).then(() => {
  // done
});

1
2
3
4
5
index.set_settings({
    'attributesForFaceting': [
        'filterOnly(brand)'
    ]
})

1
2
3
4
5
6
7
let settings = Settings()
  .set(\.attributesForFaceting, to: [.filterOnly("brand")])
index.setSettings(settings) { result in
  if case .success(let response) = result {
     print("Response: \(response)")
  }
}

1
2
3
4
5
6
7
val settings = settings {
    attributesForFaceting {
        +"filterOnly(brand)"
    }
}

index.setSettings(settings)

1
2
3
4
5
6
7
8
9
10
11
12
IndexSettings settings = new IndexSettings
{
    AttributesForFaceting = new List<string>
    {
      "filterOnly(brand)"
    }
};

index.SetSettings(settings);

// Asynchronous
await index.SetSettingsAsync(settings);

1
index.setSettings(new IndexSettings().setAttributesForFaceting(Collections.singletonList("allergens")));

1
2
3
res, err := index.SetSettings(search.Settings{
  AttributesForFaceting: opt.AttributesForFaceting("filterOnly(brand)"),
})

1
2
3
4
5
6
7
client.execute {
  setSettings of "myIndex" `with` IndexSettings(
    attributesForFaceting = Some(Seq(
      "filterOnly(brand)"
    ))
  )
}

• Prefer attributes with a low cardinality for your optionalFilters. For example, it’s preferable to use it with attributes like “brand” rather than “product_name”, as there are usually more unique product names than brands.
• Minimize the number of optionalFilters as much as possible.
• Aim for queries to return a smaller result set.
• Depending on your plan, it may be possible to apply sharding and other custom settings to optimize your implementation. For more information, contact the support team.