{"id":"bloom-filter-prevents-frontier-duplicates","text":"Every URL is added to the Bloom filter before being pushed into the frontier, so the frontier never enqueues a URL that has already been seen (modulo false positives, which cause URLs to be permanently skipped).","truth_value":"IN","source":"entries/2026/06/05/web-crawler-web_crawler.md","source_url":"","source_hash":"","justifications":[],"dependents":["crawler-three-layer-dedup"],"metadata":{},"created_at":"","updated_at":"","reviewed_at":"","verified_at":"","retracted_at":"","explanation":{"steps":[{"node":"bloom-filter-prevents-frontier-duplicates","truth_value":"IN","reason":"premise"}]}}