Status: IN
The architecture applies dedup at three independent system boundaries with accuracy adapted to cost: exact key-based dedup at API boundaries (idempotency keys for hotel, payment, ad-click), exact event-based dedup at stream processing boundaries (coordinated dedup with watermark finalization), and approximate content-based dedup at crawling boundaries (Bloom filter + SimHash) — each boundary uses the mechanism whose accuracy-memory tradeoff fits its domain's scale and failure cost.
Three dedup mechanisms at three boundaries — API (exact/key), stream (exact/event), crawl (approximate/content)
Depends on (SL): idempotency-keys-ignore-payload-content, forward-only-stream-processing-is-exactly-once, probabilistic-dedup-trades-memory-for-coverage