BBOT

The talk traces the development of BBOT, an open-source recursive internet scanner aimed at pentesters and bug bounty hunters. The speaker begins with a consulting story: external assessments kept missing subdomains that internal testing later exposed, which revealed flaws in a conventional chain of enumeration, port scanning and web discovery. Spiderfoot's design, where each module takes one data type, emits another and feeds results back into the loop, became the core idea.

Recursion brought its own failures. Cloud provider CNAMEs pulled in whole provider ranges, DNS wildcards created false positives, infinite HTTP redirects and CNAME loops made scans run forever, and any scanner that follows links effectively becomes a web spider. The fix was a scope distance counter on every finding that caps how far a scan wanders out of bounds, along with cloud tagging and stronger wildcard detection. Moving from threads to asyncio simplified the code, and fuzzing the async stack exposed bugs in DNS and HTTP libraries and in Python itself, which were fixed upstream.

On a large eBay target, BBOT found nearly twice as many subdomains as the other tools tested, while some of those tools ran for hours or crashed. The main takeaway for practitioners is to treat all internet data as untrusted input and to build an explicit exit condition into any recursive process. The tool installs from PyPI and can also be used as a Python library, including in small chat-bot integrations.