{"results":[{"id":"ad-click-dedup-global-not-per-ad","text":"Ad click deduplication keys on `event_id` alone (global registry), not per-ad, converting at-least-once delivery into exactly-once aggregation.","truth_value":"IN","justification_count":0,"dependent_count":1,"challenges":[],"last_reviewed":null,"review_result":null,"source_type":""},{"id":"approximation-spans-counting-windowing-and-similarity","text":"The codebase's approximation strategies form a complete accuracy-cost spectrum: probabilistic structures (HLL for cardinality, Morris for counting, SimHash for similarity) trade accuracy for space, and the sliding window counter trades exact window boundaries for bounded resource consumption via current/previous window weighting — together covering counting, rate limiting, and content dedup.","truth_value":"IN","justification_count":1,"dependent_count":0,"challenges":[],"last_reviewed":"2026-06-06T06:26:57","review_result":"pass","source_type":""},{"id":"ch-get-nodes-deduplicates-physical","text":"`get_nodes` walks clockwise and skips virtual nodes belonging to already-collected physical nodes, guaranteeing distinct physical nodes for replication.","truth_value":"IN","justification_count":0,"dependent_count":1,"challenges":[],"last_reviewed":null,"review_result":null,"source_type":""},{"id":"chat-dm-conversation-dedup","text":"DM conversations use a deterministic sorted-pair ID (`dm:{min}:{max}`) guaranteeing exactly one conversation per user pair regardless of who messages first.","truth_value":"IN","justification_count":0,"dependent_count":1,"challenges":[],"last_reviewed":null,"review_result":null,"source_type":""},{"id":"consistent-hashing-is-a-stateless-topology-abstraction","text":"The consistent hash ring is a pure topology abstraction with no data-plane state: node operations are idempotent (add/remove on existing/missing nodes are no-ops), migration tracking is optional (callers pass keys in, the ring reports movements), and preference list construction automatically deduplicates physical nodes — making the ring a stateless function from key to node list that is safe to use concurrently and compose freely.","truth_value":"IN","justification_count":1,"dependent_count":1,"challenges":[],"last_reviewed":"2026-06-05T18:21:49","review_result":"insufficient","source_type":""},{"id":"crawler-layered-dedup-bloom-then-simhash","text":"Deduplication is two-tiered: the Bloom filter rejects exact-URL revisits first (O(k) per check), then SimHash rejects near-duplicate content (linear scan over seen hashes) — content dedup only runs for URLs that pass the Bloom filter.","truth_value":"IN","justification_count":0,"dependent_count":1,"challenges":[],"last_reviewed":null,"review_result":null,"source_type":""},{"id":"crawler-three-layer-dedup","text":"The web crawler achieves comprehensive deduplication through three coordinated layers: URL normalization at ingestion boundaries, Bloom filter for O(k) exact-URL rejection, and SimHash for near-duplicate content detection.","truth_value":"IN","justification_count":1,"dependent_count":1,"challenges":[],"last_reviewed":"2026-06-06T06:26:57","review_result":"unnecessary","source_type":""},{"id":"dedup-and-finalization-are-coordinated","text":"Dedup retention outlives the aggregation window by 2×, and finalized results are irrevocable — this is coordinated design: since emitted results cannot be retracted, the system must ensure duplicates are caught before finalization, requiring dedup coverage to extend beyond the window boundary as a correctness invariant.","truth_value":"IN","justification_count":1,"dependent_count":2,"challenges":[],"last_reviewed":"2026-06-06T06:26:57","review_result":"pass","source_type":""},{"id":"dedup-is-global-not-per-ad","text":"The `seen_events` dedup registry keys on `event_id` alone; the same event ID arriving for different `ad_id`s will be deduplicated.","truth_value":"OUT","justification_count":0,"dependent_count":1,"challenges":[],"last_reviewed":null,"review_result":null,"source_type":""},{"id":"dedup-is-stratified-across-boundaries-and-accuracy-levels","text":"The architecture applies dedup at three independent system boundaries with accuracy adapted to cost: exact key-based dedup at API boundaries (idempotency keys for hotel, payment, ad-click), exact event-based dedup at stream processing boundaries (coordinated dedup with watermark finalization), and approximate content-based dedup at crawling boundaries (Bloom filter + SimHash) — each boundary uses the mechanism whose accuracy-memory tradeoff fits its domain's scale and failure cost.","truth_value":"IN","justification_count":1,"dependent_count":1,"challenges":[],"last_reviewed":"2026-06-06T06:26:57","review_result":"pass","source_type":""},{"id":"dedup-outlives-aggregation-window","text":"The dedup registry retains entries for 2× the allowed lateness, ensuring late-arriving duplicates are still caught even after their aggregation window has been finalized by the watermark — the dedup horizon intentionally exceeds the processing horizon.","truth_value":"IN","justification_count":1,"dependent_count":1,"challenges":[],"last_reviewed":"2026-06-06T06:26:57","review_result":"pass","source_type":""},{"id":"dedup-pruning-uses-2x-lateness","text":"The dedup registry evicts entries older than `2 * allowed_lateness` on watermark advance, meaning dedup coverage extends beyond the late-event acceptance window.","truth_value":"IN","justification_count":0,"dependent_count":2,"challenges":[],"last_reviewed":null,"review_result":null,"source_type":""},{"id":"duplicate-prevention-is-complete-from-api-to-storage","text":"The architecture achieves complete duplicate prevention from external API boundary to internal storage through complementary forward-only mechanisms: idempotency keys extend forward-only semantics to the client boundary (making duplicate submissions return cached results permanently), while stratified dedup covers internal processing with accuracy adapted to cost (exact key-based at API boundaries, coordinated window-based at stream boundaries, probabilistic at crawl boundaries).","truth_value":"IN","justification_count":1,"dependent_count":0,"challenges":[],"last_reviewed":"2026-06-06T06:26:57","review_result":"pass","source_type":""},{"id":"event-and-task-processing-share-forward-only-correctness","text":"Event stream processing (exactly-once via coordinated dedup and finalization) and task graph processing (video pipeline forward progress via control-data separation and failure cascading) achieve their correctness guarantees through the same forward-only principle despite operating on fundamentally different processing models — both prevent regression and maximize useful work through irrevocable transitions.","truth_value":"IN","justification_count":1,"dependent_count":1,"challenges":[],"last_reviewed":"2026-06-06T06:26:57","review_result":"pass","source_type":""},{"id":"eviction-timing-has-no-codebase-convention","text":"Data eviction timing varies across modules with no consistent strategy: message queue trims eagerly on every publish, rate limiter replaces its counter dict on each allowed request, URL shortener checks expiration lazily at read time, and the dedup registry prunes on watermark advance — each module independently decides when to reclaim old data, unlike other cross-cutting patterns (soft delete, time injection) where conventions exist.","truth_value":"IN","justification_count":1,"dependent_count":1,"challenges":[],"last_reviewed":"2026-06-06T06:26:57","review_result":"insufficient","source_type":""},{"id":"forward-only-is-universal-processing-primitive","text":"Forward-only correctness is the codebase's universal processing primitive, applying uniformly across all three processing paradigms: event streams (exactly-once via coordinated dedup and finalization), task orchestration (DAG forward progress via control-data separation), and message delivery (bounded retry with escalation to permanent failure).","truth_value":"IN","justification_count":1,"dependent_count":2,"challenges":[],"last_reviewed":null,"review_result":null,"source_type":""},{"id":"forward-only-spans-data-domain-and-execution-layers","text":"Forward-only design enforces no-regression guarantees independently at three architectural layers: domain-level state machines contain temporal gaps through irrevocable transitions, execution-level stream processing achieves exactly-once through coordinated dedup and finalization, and execution-level task pipelines maximize forward progress through control-data separation — the no-regression principle is layered across the architecture, not singular.","truth_value":"IN","justification_count":1,"dependent_count":2,"challenges":[],"last_reviewed":null,"review_result":null,"source_type":""},{"id":"forward-only-stream-processing-is-exactly-once","text":"Stream processing achieves exactly-once delivery per entity: dedup outlives finalization windows, finalized results are irrevocable, and the pipeline never backtracks — producing correct per-event semantics when event IDs are globally unique.","truth_value":"IN","justification_count":1,"dependent_count":3,"challenges":[],"last_reviewed":null,"review_result":null,"source_type":""},{"id":"kv-150-vnodes-per-node","text":"Consistent hash ring uses 150 virtual nodes per physical node; `_get_preference_list` deduplicates by physical node ID when walking the ring","truth_value":"IN","justification_count":0,"dependent_count":0,"challenges":[],"last_reviewed":null,"review_result":null,"source_type":""},{"id":"probabilistic-dedup-trades-memory-for-coverage","text":"The crawler's multi-layer dedup combines deterministic normalization with probabilistic structures (Bloom filter for O(k) exact-URL rejection, SimHash for O(1) near-content detection), achieving coverage across three dimensions while maintaining bounded memory through tunable accuracy-space tradeoffs.","truth_value":"IN","justification_count":1,"dependent_count":2,"challenges":[],"last_reviewed":null,"review_result":null,"source_type":""}],"count":22,"limit":20,"offset":0}