Skip to content

✨ NEW: Add section reference plugin (section_ref)#144

Merged
chrisjsewell merged 4 commits into
masterfrom
claude/mdit-section-reference-plugin-yfvmtz
Jul 19, 2026
Merged

✨ NEW: Add section reference plugin (section_ref)#144
chrisjsewell merged 4 commits into
masterfrom
claude/mdit-section-reference-plugin-yfvmtz

Conversation

@chrisjsewell

@chrisjsewell chrisjsewell commented Jul 16, 2026

Copy link
Copy Markdown
Member

What

Adds a new section_ref inline plugin that captures section-sign references — §1, §1.1, §2.3.4 — the syntax LLMs commonly use to refer to numbered headings in documents.

See §1, §1.1 and §2.3.4 for details.

Each reference becomes a dedicated section_ref token:

  • content: the matched text (e.g. "§1.1")
  • markup: "§"
  • meta["numbers"]: the parsed section number as a list of ints (e.g. [1, 1]), so consumers don't re-parse the dotted string (a list rather than a tuple so Token.as_dict()/JSON round-trips are type-stable; leading zeros are normalized, with the original text preserved in content)

so downstream renderers (notably MyST-Parser) can resolve them to heading cross-references. The default rendering is a neutral <span class="section-ref">§1.1</span>, overridable via md.add_render_rule("section_ref", ...).

Initial use case

executablebooks/MyST-Parser#1170 (draft) consumes this plugin as a new section_ref MyST extension, resolving the references to internal links targeting the correspondingly numbered heading of the document. That PR is blocked on this one being merged and released (it needs mdit-py-plugins >= 0.7).

Matching rules

  • § must be immediately followed by ASCII digits ([0-9], never unicode digits), optionally in further dot-separated groups — § 1 and bare § stay literal text
  • trailing punctuation is not consumed: see §1. captures §1
  • a directly-following ASCII letter rejects the match (§1a, §1.2b), while non-ASCII letters do not block it (见§3章 captures §3)
  • standard CommonMark backslash-escaping applies: \§1 stays literal
  • inline code, code fences, autolinked URLs, and link destinations/titles are unaffected

Implementation notes

§ is not one of markdown-it's default text-rule terminator characters, so a plain inline rule would never fire mid-paragraph. The plugin therefore uses md.inline.add_terminator_char("§"), the same mechanism as the gfm_autolink plugin, and raises a RuntimeError on markdown-it-py < 4.1.0 (package dependency range unchanged).

Testing

  • 24 new tests: token-contract test, version-guard test, and 22 rendering fixtures (tests/fixtures/section_ref.md) covering emphasis/link/code/heading contexts, boundary and escaping edge cases
  • full suite passes (507 tests); ruff + mypy --strict clean under both markdown-it-py 3.x and 4.x
  • adversarially probed against a MyST-style plugin stack (attrs, myst_role, dollarmath, gfm_autolink, footnote, deflist, anchors), GFM autolink URLs containing §, link-label posMax slices, the typographer, and cross-instance state leakage

Captures section-sign references such as §1, §1.1 and §2.3.4 — the
syntax LLMs commonly use to refer to numbered headings — into dedicated
`section_ref` inline tokens (meta["number"] holds the number string),
so downstream renderers such as MyST-Parser can resolve them to heading
cross-references. Default rendering is a `<span class="section-ref">`.

Uses `md.inline.add_terminator_char("§")` (markdown-it-py >= 4.1.0),
since § is not a default text-rule terminator character.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwRkZLzpbHdb3pgmctmn6H
@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.79%. Comparing base (d11bdaf) to head (6a4040d).
⚠️ Report is 14 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #144      +/-   ##
==========================================
- Coverage   92.80%   92.79%   -0.02%     
==========================================
  Files          31       40       +9     
  Lines        1835     2289     +454     
==========================================
+ Hits         1703     2124     +421     
- Misses        132      165      +33     
Flag Coverage Δ
pytests 92.79% <100.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

claude added 3 commits July 16, 2026 09:48
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwRkZLzpbHdb3pgmctmn6H
meta["numbers"] now holds e.g. [1, 1] for §1.1, so downstream renderers
do not need to re-parse the dotted number string. A list (not tuple) is
used so Token.as_dict()/JSON round-trips are type-stable. The original
matched text remains available in token.content.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwRkZLzpbHdb3pgmctmn6H
@chrisjsewell
chrisjsewell merged commit c58aaaa into master Jul 19, 2026
12 checks passed
@chrisjsewell
chrisjsewell deleted the claude/mdit-section-reference-plugin-yfvmtz branch July 19, 2026 18:06
chrisjsewell added a commit that referenced this pull request Jul 19, 2026
Bumps the version to 0.7.0 and dates the changelog for release.

## 0.7.0 - 2026-07-19

- ✨ NEW: Add section reference plugin (`section_ref`) (#144)
- 🐛 FIX: `myst_role` no longer mis-rejects a role at the start of inline
content when the content ends with a backslash (#62, #145)

Also included since 0.6.1: the switch to PyPI trusted publishing (#142)
— this will be the first release through the OIDC publish job.

After merge, tagging `v0.7.0` on master triggers the publish job.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01LwRkZLzpbHdb3pgmctmn6H)_

Co-authored-by: Claude <noreply@anthropic.com>
chrisjsewell added a commit that referenced this pull request Jul 19, 2026
## What

Fixes #62: `myst_role` misparsed inline content that **starts** with a
role and **ends** with a backslash:

```python
md.render("{foo}`ar`\\")
# before: <p>{foo}<code>ar</code>\</p>          (role not captured)
# after:  <p><code class="myst role">{foo}[ar]</code>\</p>
```

## Why

The "starting backslash escape" guard read `state.src[state.pos - 1]`.
At `state.pos == 0` that is Python negative indexing — it reads the
**last** character of the inline source (the `except IndexError` never
fires), so a role at the start of the inline was rejected whenever the
content ended with `\`.

The guard is removed entirely rather than bounds-checked, because it was
wrong in both remaining respects too:

- For its intended case (`\{foo}`x``) it was dead code: `{` is
CommonMark-escapable, so the core `escape` rule (which runs before this
rule) consumes `\{` wholesale — `myst_role` is never invoked there.
Verified unchanged before/after.
- For an escaped backslash (`\\{foo}`x``) it wrongly suppressed a live
role; it now renders a literal `\` followed by the parsed role, matching
the CommonMark analogy (`\\*a*` → `\<em>a</em>`). This is the "edge case
`\\{`" the guard's own comment flagged as broken.

This mirrors the analysis done for the `section_ref` plugin in #144,
where the same guard pattern was dropped.

## Testing

- 3 new fixture cases: the #62 repro, `\{foo}`x`` (still escaped, via
the core rule), and `\\{foo}`x`` (now a live role).
- Adversarially probed old-vs-new behavior across commonmark/default/gfm
presets, table cells, link labels, image alts, and emphasis adjacency:
the only differences are the two intended fixes. Full suite 511 passed;
ruff and strict mypy clean.
- Sole caveat: in a `zero`-preset config *without* the core `escape`
rule, `\{foo}`x`` now parses the role (previously
suppressed-but-still-literal-backslash). No realistic consumer runs
`myst_role` without the escape rule; a code comment now documents the
assumption.

Note: the same latent `state.src[state.pos - 1]` wraparound exists in
`dollarmath` (only reachable with the non-default `allow_digits=False`)
— left for a separate PR.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01LwRkZLzpbHdb3pgmctmn6H)_

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants