Home/Low-level design/Document Search Engine

Document Search Engine

Tokenization, positional indexes, and stale updates

An inverted index maps a token to documents; a positional index also records token offsets. Positions let phrase matching check adjacency and overlapping starts. Matching whole tokens avoids false hits such as finding art inside cart.

go
positions := []int{0, 1, 2}
fmt.Println(positions[1]-positions[0] == 1) // adjacent token positions

Replacing a document must remove all old frequencies and positions before adding its new content. Deleting an ID and later reusing it must not revive stale postings. Keep the original text separately from normalized tokens. Search ranking combines numeric score descending with ID ascending; map iteration cannot supply that order. Statistics include empty documents even though they have no postings.

Go models this service with a struct and pointer-receiver methods. Constructor functions return a pointer; validation failures return errors rather than panicking. Scalars in snapshot structs are copied by value. Maps, slices, and optional pointers still require detached copies when crossing the service boundary. Go maps do not have key iteration order; use ordered slices for ranked results. There are no threads, persistence, wall-clock reads, or external side effects in this exercise.

Put it into practice

Implement the exercise in a full-screen workspace, then run its tests.

Open practice
Next lessonTask Scheduler