Skip to content

BBOT 3.0 - blazed_elijah#2007

Merged
TheTechromancer merged 606 commits intodevfrom
3.0
Apr 16, 2026
Merged

BBOT 3.0 - blazed_elijah#2007
TheTechromancer merged 606 commits intodevfrom
3.0

Conversation

@TheTechromancer
Copy link
Copy Markdown
Collaborator

@TheTechromancer TheTechromancer commented Nov 22, 2024

Summary

BBOT 3.0 "blazed_elijah" contains changes needed to store BBOT data in a persistent database. The idea is to release it alongside BBOT server, a tiny CLI-only database. This will be paired with a series of blog posts showing how BBOT server can be used on the command line to script out bug bounty hunting, threat intel, and ASM (i.e. running scheduled scans, exporting to CSV, diffing results over time, etc.).

Together, BBOT 3.0 and BBOT server will give us a solid foundation to build a bunch of other useful tooling, like asset inventory. Sometime in the future, it may also be useful to frontend.

Breaking changes

1. .data and .data_json event fields

The main breaking change in BBOT 3.0 is that the name of the .data field is different based on whether it's a str or dict.

  • .data: string
  • .data_json: dictionary

The siem_friendly option has been removed, since BBOT data is now SIEM-friendly by default.

2. Changes to vulnerabilities

The VULNERABILITY event type has been removed in favor of FINDING, which now has several improvements:

  • A name field which holds a generic description common to all findings of the same type. This makes it easier to collapse and categorize them.
  • A confidence field
  • A severity field

Features

Potential changes

@TheTechromancer TheTechromancer changed the base branch from stable to dev November 22, 2024 01:58
@TheTechromancer TheTechromancer self-assigned this Nov 22, 2024
@codecov
Copy link
Copy Markdown

codecov bot commented Nov 22, 2024

Codecov Report

❌ Patch coverage is 84.04848% with 658 lines in your changes missing coverage. Please review.
✅ Project coverage is 91%. Comparing base (0f16c56) to head (d8f056e).
⚠️ Report is 628 commits behind head on dev.

Files with missing lines Patch % Lines
bbot/core/event/base.py 76% 78 Missing ⚠️
bbot/modules/azure_tenant.py 76% 66 Missing ⚠️
...ot/test/benchmarks/test_event_memory_benchmarks.py 19% 52 Missing ⚠️
bbot/test/benchmarks/_scan_memory_web_crawl.py 0% 49 Missing ⚠️
...test/benchmarks/test_scan_throughput_benchmarks.py 37% 28 Missing ⚠️
...bot/test/benchmarks/_scan_memory_subdomain_enum.py 0% 27 Missing ⚠️
bbot/modules/trajan.py 85% 26 Missing ⚠️
bbot/core/helpers/misc.py 67% 21 Missing ⚠️
bbot/core/helpers/asn.py 62% 18 Missing ⚠️
bbot/modules/base.py 74% 17 Missing ⚠️
... and 58 more
Additional details and impacted files
@@           Coverage Diff           @@
##             dev   #2007     +/-   ##
=======================================
- Coverage     92%     91%     -0%     
=======================================
  Files        416     440     +24     
  Lines      34690   37685   +2995     
=======================================
+ Hits       31648   34104   +2456     
- Misses      3042    3581    +539     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@liquidsec
Copy link
Copy Markdown
Contributor

@TheTechromancer we should now add to the list of breaking changes, the removal of vulnerability event type and the addition of severity and confidence as attributes on finding types

Comment thread bbot/test/test_step_1/test_presets.py Dismissed
Comment thread bbot/test/test_step_1/test_presets.py Dismissed
Comment thread bbot/test/test_step_1/test_presets.py Dismissed
Comment thread bbot/test/test_step_1/test_presets.py Dismissed
Comment thread bbot/test/test_step_1/test_presets.py Dismissed
Comment thread bbot/test/test_step_1/test_scan.py Dismissed
Comment thread bbot/test/test_step_1/test_target.py Dismissed
Comment thread bbot/test/test_step_1/test_target.py Dismissed
Comment thread bbot/test/test_step_1/test_target.py Dismissed
Comment thread bbot/test/test_step_1/test_target.py Dismissed
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 27, 2026

🚀 Performance Benchmark Report

ℹ️ No baseline benchmark data available

Showing current results for 3.0 only.

📊 Current Results (3.0) - Click to expand

Results

Test Name Mean Time Ops/sec Min Max
Bloom Filter Dns Mutation Tracking Performance 4.20ms 238.3 ops/sec 4.14ms 4.28ms
Bloom Filter Large Scale Dns Brute Force 17.14ms 58.3 ops/sec 16.98ms 17.46ms
Large Closest Match Lookup 353.49ms 2.8 ops/sec 349.03ms 357.71ms
Realistic Closest Match Workload 191.62ms 5.2 ops/sec 190.68ms 192.86ms
Event Memory Medium Scan 2.222s 0.5 ops/sec 2.031s 2.548s
Event Memory Large Scan 10.589s 0.1 ops/sec 10.509s 10.633s
Event Validation Full Scan Startup Small Batch 408.98ms 2.4 ops/sec 402.38ms 412.39ms
Event Validation Full Scan Startup Large Batch 571.34ms 1.8 ops/sec 566.27ms 573.36ms
Make Event Autodetection Small 30.18ms 33.1 ops/sec 30.09ms 30.35ms
Make Event Autodetection Large 310.40ms 3.2 ops/sec 309.01ms 311.86ms
Make Event Explicit Types 13.65ms 73.3 ops/sec 13.61ms 13.75ms
Excavate Single Thread Small 3.948s 0.3 ops/sec 3.868s 4.019s
Excavate Single Thread Large 9.488s 0.1 ops/sec 9.379s 9.538s
Excavate Parallel Tasks Small 4.141s 0.2 ops/sec 4.067s 4.212s
Excavate Parallel Tasks Large 7.161s 0.1 ops/sec 7.114s 7.216s
Is Ip Performance 3.19ms 313.7 ops/sec 3.15ms 4.76ms
Make Ip Type Performance 11.55ms 86.6 ops/sec 11.47ms 11.88ms
Mixed Ip Operations 4.51ms 221.8 ops/sec 4.43ms 4.94ms
Memory Use Web Crawl 1.03µs 969.0K ops/sec 1.03µs 1.03µs
Memory Use Subdomain Enum 561ns 1782.5K ops/sec 561ns 561ns
Scan Throughput 100 8.170s 0.1 ops/sec 7.781s 8.473s
Scan Throughput 1000 41.319s 0.0 ops/sec 39.323s 42.782s
Typical Queue Shuffle 63.73µs 15.7K ops/sec 59.13µs 862.59µs
Priority Queue Shuffle 724.01µs 1.4K ops/sec 679.48µs 2.26ms

🐍 Python Version 3.11.15

liquidsec and others added 14 commits March 2, 2026 12:05
Fix omitted event types leaking to console output
- Add submodule auto-filter: disable submodules whose max severity/confidence
  is below configured thresholds (avoids running expensive submodules for nothing)
- Create baddns.yml base preset (CNAME, MX, TXT) and baddns-heavy.yml (all submodules)
- Rename spider-intense→spider-heavy, baddns-intense→baddns-heavy
- Fix baddns_zone default min_severity to INFORMATIONAL (NSEC/zonetransfer need it)
- Update kitchen-sink.yml, remove stale enable_references v1.x config
- Fix baddns_zone NSEC test (bad.dns→bad.com for tldextract compatibility)
- Fix baddns_direct test (updated signature matcher for baddns 2.0)
- Update all preset warning messages and docs references
- Add submodule auto-filter: disable submodules whose max severity/confidence
  is below configured thresholds (avoids running expensive submodules for nothing)
- Create baddns.yml base preset (CNAME, MX, TXT) and baddns-heavy.yml (all submodules)
- Rename spider-intense→spider-heavy, baddns-intense→baddns-heavy
- Fix baddns_zone default min_severity to INFORMATIONAL (NSEC/zonetransfer need it)
- Update kitchen-sink.yml, remove stale enable_references v1.x config
- Fix baddns_zone NSEC test (bad.dns→bad.com for tldextract compatibility)
- Fix baddns_direct test (updated signature matcher for baddns 2.0)
- Update all preset warning messages and docs references
@aconite33
Copy link
Copy Markdown
Contributor

recheck

liquidsec and others added 20 commits April 3, 2026 13:35
Show status code, URL, title, and redirect target instead of raw JSON dict.
…tatus-line

Show FINDING severity breakdown in scan status line
CDNs and reverse proxies return 502/503/504 for upstream failures, not
as an IIS shortname signal. Treating them as status-code differences
caused false positives on CDN-fronted non-IIS hosts.
Add human-readable stdout display for all event types
…y-check

Fix non-functional DNS brute-force canary check
Convert websocket URLs to HTTP instead of raising findings
…-gateway-fp

Skip gateway error codes in IIS shortnames detection
"evilcorp.com # the main domain",
],
)
assert "evilcorp.com" in target.seeds

Check failure

Code scanning / CodeQL

Incomplete URL substring sanitization High test

The string
evilcorp.com
may be at an arbitrary position in the sanitized URL.

Copilot Autofix

AI 1 day ago

Copilot could not generate an autofix suggestion

Copilot could not generate an autofix suggestion for this alert. Try pushing a new commit or if the problem persists contact support.

)

result = chain_lists([str(target_file)], try_files=True, _strip_comments=True)
assert "evilcorp.com" in result

Check failure

Code scanning / CodeQL

Incomplete URL substring sanitization High test

The string
evilcorp.com
may be at an arbitrary position in the sanitized URL.

Copilot Autofix

AI 1 day ago

Copilot could not generate an autofix suggestion

Copilot could not generate an autofix suggestion for this alert. Try pushing a new commit or if the problem persists contact support.


result = chain_lists([str(target_file)], try_files=True, _strip_comments=True)
assert "evilcorp.com" in result
assert "othercorp.com" in result

Check failure

Code scanning / CodeQL

Incomplete URL substring sanitization High test

The string
othercorp.com
may be at an arbitrary position in the sanitized URL.

Copilot Autofix

AI 1 day ago

Copilot could not generate an autofix suggestion

Copilot could not generate an autofix suggestion for this alert. Try pushing a new commit or if the problem persists contact support.

@TheTechromancer
Copy link
Copy Markdown
Collaborator Author

recheck

@TheTechromancer TheTechromancer merged commit 233c86c into dev Apr 16, 2026
17 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants