AFDocs fixes: refresh known-exceptions reference after 2026-07-31 audit - #429
Closed
rachaelrenk wants to merge 1 commit into
Closed
AFDocs fixes: refresh known-exceptions reference after 2026-07-31 audit#429rachaelrenk wants to merge 1 commit into
rachaelrenk wants to merge 1 commit into
Conversation
Ran the AFDocs scorecard against docs.warp.dev twice (afdocs@0.18.7). Score 79/100 (C+): 16 pass, 4 fail, 1 warn, 2 skip. Every failure traced back to an allowlisted platform limitation, a checker false positive, or page-sampling variance, so no site changes were needed. The exceptions reference had drifted from reality, which risks future audits either re-attempting fixes that are already committed or treating allowlisted noise as a regression. Updated it to match what the audit actually reports now: - content-start-position, markdown-url-support: now pass. Documented why, and kept the entries so a re-appearance is recognized as known noise. - content-negotiation: recorded that both remediations the afdocs-fix skill suggests (src/middleware.ts and the vercel.json Accept-header rewrites) are already committed and still never fire on Vercel's static hosting, so they are not re-added. - llms-txt-links-markdown: replaced the vague "YAML/JSON are classified as non-markdown" note with the real mechanism. 13 of 15 same-origin llms.txt links serve markdown (87%) against a 90% threshold; the only two that don't are openapi.yaml and openapi.json, which toMdUrls() can never satisfy. Noted that neither deleting a link nor padding llms.txt is an acceptable fix. - llms-txt-directive-html / -md: new entry. The intermittent "2 missing" warn comes from AFDocs pulling openapi.yaml and openapi.json into its page sample via llms.txt. Verified against all 352 sitemap pages, using AFDocs' own detection logic, that the directive is present on every page and buried on none. - page-size-*: refreshed measurements. The three pages over 100K markdown chars are open-source-licenses (~1.37M), changelog/2026 (~154K), and privacy (~112K). Dropped the stale all-settings warning; it is ~40K now. - markdown-content-parity: refreshed the affected-page list and kept the prior sample for history. - Added a sampling-variance section to both the skill and the reference. llms-txt-directive-html and page-size-html flipped between the two runs with no site changes, so a single run is not enough to call a regression. Co-Authored-By: Oz <oz-agent@warp.dev>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Superseded by a bot-authored replacement PR from the same branch. Closing this human-authored draft. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ran the AFDocs scorecard against
docs.warp.devand found no genuine, fixable site issues. This PR refreshes the audit skill's exceptions reference, which had drifted from what the audit actually reports.Audit result: 79/100 (C+) — 23 checks: 16 pass, 4 fail, 1 warn, 2 skip.
No site source changed (
src/,astro.config.mjs,vercel.json, and content are all untouched), so there is nothing to rebuild or redeploy.Why no site fixes
Every failure and the one warning traced back to an allowlisted platform limitation, a checker false positive, or page-sampling variance. I ran the audit twice and read the AFDocs check implementations to confirm each one:
llms-txt-links-markdownopenapi.yamlandopenapi.json, andtoMdUrls()returns no.mdcandidates for non-page extensions, so they can never satisfy the check.content-negotiationafdocs-fixskill suggests are already committed (src/middleware.tsand theAccept: text/markdownrewrites invercel.json) and still never fire.markdown-content-parity### 2\. …). All flagged diffs are numbered heading text; content is intact.page-size-htmlllms-txt-directive-htmlopenapi.yaml/openapi.jsoninto its page sample via llms.txt; neither is HTML.For the directive warning I crawled all 352 sitemap pages using AFDocs' own detection logic and confirmed the llms.txt directive is present on every page and buried past the 50% threshold on none.
I deliberately did not clear
llms-txt-links-markdown. The only ways to reach 90% are deleting an OpenAPI link or padding llms.txt with extra links — both degrade the file to satisfy a scorecard rather than improving the agent experience. Every link that represents documentation prose already serves markdown (13/13).Changes
references/known-exceptions.md:content-start-positionandmarkdown-url-supportnow pass — documented why, and kept the entries so a re-appearance is recognized as known noise rather than a new bug.content-negotiation— recorded that the suggested middleware and rewrites are already in place, so a future agent doesn't re-add them.llms-txt-links-markdown— replaced the vague "YAML/JSON are classified as non-markdown" note with the actual mechanism and the 13/15 threshold math.llms-txt-directive-html/-md— new entry covering the OpenAPI page-sample artifact, with the full-crawl verification.page-size-*— refreshed measurements. The three pages over 100K markdown chars areopen-source-licenses(~1.37M),changelog/2026(~154K), andprivacy(~112K). Dropped the staleall-settingswarning; it is ~40K now.markdown-content-parity— refreshed the affected-page list, kept the prior sample for history.SKILL.md:llms-txt-directive-htmlandpage-size-htmlflipped between the two runs with no site changes, so a single run isn't enough to call a regression.Testing
node .agents/skills/afdocs-audit/scripts/afdocs_audit.mjstwice; preflight confirmed no Vercel Firewall challenge, so both audits are valid..mdavailability across the sitemap, directive presence on all 352 pages, and markdown character counts for the oversized pages..agents/, which is outside the Astro content pipeline.Conversation: https://app.warp.dev/conversation/2597d1b1-9c4f-466e-b6af-80ced35eb448
Run: https://oz.warp.dev/runs/019fb730-0148-7ef8-ae20-b0f4e7eb4a1c
This PR was generated with Oz.