Return a stored answer for a similar enough question, but only inside its tenant, language and prompt version, never for personal answers, and purge it when a source changes.
A semantic cache embeds the question, finds the nearest past question, and returns its stored answer if it's similar enough, skipping the model entirely. Day 5 (section 3.2) lists how that goes wrong: "opened" and "unopened" earphones sit close in embedding space but flip the answer; a personal answer leaks to the next customer; a policy change leaves the cache citing the old rule. This lab builds the cache with the lesson's four rules:
Implement:
| Name | Does |
|---|---|
make_scope(tenant, language, prompt_version) |
returns the tuple (tenant, language, prompt_version). Any part missing or empty: ValueError |
SemanticCache(threshold, ttl_s, clock) |
|
store(scope, question, vector, answer, sources, personal=False) |
stores an entry and returns True; a personal answer is not stored (False). A scope that isn't a valid 3-part tuple: ValueError |
lookup(scope, vector) |
the stored answer of the most similar live entry in the same scope if its cosine similarity is >= threshold, else None. Ties: the newest entry |
invalidate_source(source_id) |
removes every entry whose sources include it; returns how many |
hit_rate() |
hits / lookups so far; 0.0 before any lookup |
Rules the tests check:
clock.now() - stored_at >= ttl_s; expired entries never hit.cosine(a, b) (given). Compare with >=: the threshold is the lowest similarity you accept.lookup returns only the answer string, and every call counts as a hit or a miss.In JavaScript and TypeScript the names are makeScope(tenant, language, promptVersion), which returns the array [tenant, language, promptVersion], and new SemanticCache(threshold, ttlSeconds, clock) with store(scope, question, vector, answer, sources, personal = false), lookup(scope, vector) (null on a miss), invalidateSource(sourceId) and hitRate(). A ValueError is a TypeError in JavaScript (Node's convention for an invalid argument): throw it for a missing or empty part and for a scope that isn't a 3-element array. Arrays compare by reference, so compare scopes part by part, or turn them into one string with JSON.stringify. The given code has the same FakeClock, cosine, unit and toy vectors V.
Write your code, then run 3 example tests and 8 hidden tests. Each run is limited to 8 seconds: an infinite loop gets stopped and the runtime restarts, your code stays.