Document Search Engine

A term index needs frequencies; a phrase index additionally needs positions. Phrase starts must preserve consecutive token order, including repeated-token overlaps. In a a a, phrase a a starts twice, so skipping forward by phrase length after a match undercounts.

using Posting=std::map<std::string,std::vector<std::size_t>>;
// One term maps each document ID to every token position in that document.

An update must remove all old postings before installing new ones; merely adding new terms leaves stale matches and wrong totals. Keep original text separately from normalized tokens so get() can return exact content. The token rule lowercases Unicode, then retains ASCII letter/digit runs. Text makes code points explicit; U+0130 lowers to i plus a combining separator, and Kelvin sign lowers to k. ASCII-only tolower would silently change this rule. Sort results by decreasing score and then ID, and return detached value pairs.

C++ contract conventions: numeric values use signed Int (long long); all inputs and intermediate arithmetic are within its representable range. Opt<T> means std::optional<T> and absence is std::nullopt; Vec<T> means std::vector<T>. Invalid documented arguments throw std::invalid_argument. Inputs passed by const reference remain caller owned. Returned containers and records own their contents. Calls use the declared C++ parameter types. Implicit conversions performed by the caller are outside the runtime validation contract. Unless a lesson states otherwise, string ordering is lexicographic over Unicode code points (Text) or UTF-8 identifiers (std::string).

Text and query parameters use Text (UTF-32 code points), rather than UTF-8 bytes. Token output is ASCII std::string; document IDs remain std::string. Tokenization applies Unicode lowercase then ASCII [a-z0-9] runs, including the two non-ASCII lowercase mappings that emit ASCII. No locale-dependent cctype behavior is used.