Searching

SearchService.search(query, language:, limit:) returns the hits of a query, best first. A hit has its document, its score, the stretches of each field that matched, and a snippet of a field around the first match, ready for a list of results.

Why

A search box is judged in the first few letters: it must find bean from bea, forgive contxet, and show why a result matched. The service does that with one call, and the hit carries what a result list needs to highlight.

Example

final hits = await search.search('"bean factory" scop', language: 'en', limit: 10);

for (final hit in hits) {
  final snippet = hit.snippet('body', length: 140);
  print('${hit.score.toStringAsFixed(2)} ${hit.document.title}');
  print('  ${hit.document.link}');
  print('  ${snippet.text}');           // "…the bean factory creates a scoped…"
  print('  ${snippet.ranges}');         // [[4, 16), [29, 35)]
}

The query

Written Meaning
bean factory Both words (operator: and), or either (operator: or)
"bean factory" The two words next to each other, in order
bea (the last word, no space after it) Also the start of longer words: bean, beans
contxet Also words one typo away (context), when fuzzy allows

The ranking

Each field is scored with BM25 and weighted by spring.search.fields: by default title 8, heading 4, body 1, and 1 for a field not named. Then the SearchRanker beans change the scores (see Analyzers and rankers), and with collapse-by only the best hit of each value of that attribute stays (page: one section per page). At most limit hits are returned, spring.search.max-hits (20) by default.

The hit

Member Meaning
document The document found
score How well it matches; higher first
highlights The matches of each field, in order: a List<SearchRange> (start, end) per field name
snippet(field, {length: 160}) About length characters of the field around its first match (from the start when it has none), cut at spaces, … where cut, with the matches in it (SearchSnippet.text, SearchSnippet.ranges)
withScore(score) The same hit with another score (for a ranker)

In Flutter, turn the ranges into TextRanges and show them with HighlightedText or ThemeText(highlights:) (see Search in a Flutter app).

The index of a language

SearchService.index(language) gives the SearchIndex of a language (or the default language's when it has none), for an app that searches it directly. invalidate() forgets the indexes; the next search builds them again.

The documents are read once; the index of a language is built the first time that language is searched, so a search waits for its own language's index only. Reading and building pause every few milliseconds, so an app keeps drawing its frames meanwhile (the web has one thread for both). When:

prepare() without a language builds every index, the default language's first. prepared tells whether the documents are read or being read, isIndexed(language) whether that index is built or being built.

The index the build writes

For the pages of spring.search.markdown the build writes the index itself, a file per language in lib/resources/spring_gen/search/ (en.json, ja.json): the pages cut into sections, their terms analyzed and counted. The SearchService reads the file of a language instead of reading every page and building the index, so the first search costs one download and a fraction of a second instead of hundreds of files and seconds of work on the thread that draws.

flutter:
  assets:
    - lib/resources/spring_gen/search/

The properties

Key Type Default Meaning
spring.search.operator and, or and and: a document has every word of the query; or: one of them
spring.search.fuzzy number 1 The typos a word may have: one for a word of five to seven letters, up to this many for a longer one; 0: none
spring.search.fields.<field> number title 8, heading 4, body 1 The weight of a field; a field not named weighs 1
spring.search.max-hits number 20 The most hits a search gives
spring.search.collapse-by attribute name none Only the best hit of each value of this attribute stays
spring.search.indexing first-search, start, background first-search When the indexes are built: at the first search (or prepare()), while the application starts, or right after it while the app runs
spring.search.prebuilt boolean true The build writes the index of spring.search.markdown, a file per language, and the search reads it (The index the build writes)
spring.search.default-language language tag en The language of a search that names none, or one without documents

See also