Skip to content

Commit 018283b

Browse files
authored
fix(rich-md-editor): raw-HTML-block fragmentation + styling follow-up (#5440)
* fix(rich-md-editor): fix raw-HTML-block fragmentation and styling - RawHtmlBlock previously delegated to marked's built-in block-HTML tokenizer, which (per CommonMark's HTML-block-type-6 rule) stops at the first blank line. A real <details> with a paragraph inside — the common case — fragmented into a raw chip, an ordinary rendered paragraph, and a second raw chip, stranding real content in between. - Fixed with a custom block-level tokenizer that scans to the tag's matching close (reusing the balanced open/close depth-tracking already built for nested inline HTML), spanning blank lines, restricted to CommonMark's own block-tag whitelist (details, div, table, section, etc.) so tags that can legitimately start a paragraph (em, a, span, code, kbd) are left untouched. - Dropped the warning-colored tint on raw HTML/footnote blocks (color-mix with --warning read as an error state) in favor of the same neutral surface as existing code blocks — the hover badge alone now signals "this is raw, unrendered text". * fix(rich-md-editor): fix indented HTML, quoted attributes, and code-mention edge cases - CommonMark allows up to 3 leading spaces before a block-HTML opening line; the new block tokenizer required column 0, so an indented <details>/<div> still fragmented across blank lines. Fixed by splitting off the leading indent, matching against the rest, and stitching the indent back onto raw. - The open-tag and balanced-close regexes stopped at the first `>`, even inside a quoted attribute value (e.g. data-example="a > b"), producing wrong match lengths and miscounting a same-tag mention inside the quoted value as a real nested tag. Replaced with an attribute-aware pattern that treats a full quoted value (including any interior >) as one unit. - The balanced-tag scan couldn't distinguish real markup from a tag name mentioned inside inline code or a fenced code block. Now masks code regions (same-length filler, positions preserved) before scanning, so a properly-escaped mention (backticks or a fenced example) is never miscounted. A genuinely bare, unescaped mention outside code remains a known, inherent limitation of regex-based tag matching (shared by real HTML parsers given the same ambiguous input) — verified this can never lose data or hang, only reflow to a stable fixpoint on save. * fix(rich-md-editor): fix blockquoted-fence masking and void-tag balance scan - maskCodeRegions's fenced-code regex required fence markers at column 0, so a fence quoted inside a markdown blockquote (each line prefixed with `> `) wasn't recognized, leaving tag mentions inside it visible to the balance scanner. Extended the fence pattern to tolerate an optional blockquote prefix on both the opening and closing fence line. - BLOCK_HTML_TAG_NAMES includes several void elements (link, meta, base, hr, ...) that have no closing tag at all. The block tokenizer only treated an explicit self-closing `/>` as complete, so a void tag without one would scan the rest of the document for a `</meta>` that will never appear, risking a false match on a later same-name mention. Now reuses the existing VOID_TAGS set (already used by the inline tokenizer) to treat these as complete right after the open tag. Also restored `hr` to the whitelist, which was accidentally dropped when transcribing marked's tag list. * fix(rich-md-editor): tolerate an indented (non-blockquoted) fence in code masking maskCodeRegions's fence pattern handled a blockquoted fence (`> \`\`\``) but still required the fence marker at column 0 otherwise, missing CommonMark's independent up-to-3-space fence indent tolerance this codebase already relies on elsewhere (FENCE_OPEN/FENCE_CLOSE in markdown-parse.ts). An indented fence's tag-name mention could still end a whitelisted block early. Fixed by combining both allowances in one prefix pattern (zero-or-more blockquote levels, each independently followed by up to 3 more spaces of indent). * fix(rich-md-editor): mask HTML comments in balanced-tag scan A tag-name mention inside an HTML comment (e.g. `<!-- see </div> below -->`) inside a whitelisted raw HTML block could still be matched by the balance scan and end the block early, fragmenting it. maskCodeRegions already masked fenced/inline code the same way; extend it to mask comments too.
1 parent dee814b commit 018283b

3 files changed

Lines changed: 390 additions & 43 deletions

File tree

apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/raw-markdown-snippet.test.ts

Lines changed: 161 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,17 @@
66
* and reach a fixpoint on a second pass (see `serializeMarkdownDocument` in `./markdown-parse.ts`).
77
*/
88
import { describe, expect, it } from 'vitest'
9-
import { serializeMarkdownDocument } from './markdown-parse'
9+
import { parseMarkdownToDoc, serializeMarkdownDocument } from './markdown-parse'
1010

1111
function roundTrip(input: string): string {
1212
return serializeMarkdownDocument(input).trim()
1313
}
1414

15+
/** Top-level node type names of the parsed doc, for structural (not just string) assertions. */
16+
function topLevelTypes(input: string): (string | undefined)[] {
17+
return (parseMarkdownToDoc(input).content ?? []).map((n) => n.type)
18+
}
19+
1520
describe('raw markdown snippet nodes', () => {
1621
it('preserves a standalone HTML comment', () => {
1722
const input = '<!-- a note -->\n\ntext'
@@ -96,3 +101,158 @@ describe('raw markdown snippet nodes', () => {
96101
expect(roundTrip(input)).toBe(input)
97102
})
98103
})
104+
105+
describe('raw HTML block: does not fragment across blank lines', () => {
106+
it('a <details><summary> block with a blank-line-separated body is ONE node, not three', () => {
107+
const input =
108+
'<details>\n<summary>Click to expand</summary>\n\nThis is inside a details/summary block.\n\n</details>'
109+
expect(topLevelTypes(input)).toEqual(['rawHtmlBlock'])
110+
expect(roundTrip(input)).toBe(input)
111+
expect(roundTrip(roundTrip(input))).toBe(roundTrip(input))
112+
})
113+
114+
it('a <div> with multiple blank-line-separated paragraphs inside is ONE node', () => {
115+
const input = '<div>\n\nfirst paragraph\n\nsecond paragraph\n\n</div>'
116+
expect(topLevelTypes(input)).toEqual(['rawHtmlBlock'])
117+
expect(roundTrip(input)).toBe(input)
118+
})
119+
120+
it('nested same-tag block HTML balances depth across blank lines', () => {
121+
const input = '<div>\nouter\n\n<div>\n\ninner\n\n</div>\n\nstill outer\n</div>'
122+
expect(topLevelTypes(input)).toEqual(['rawHtmlBlock'])
123+
expect(roundTrip(input)).toBe(input)
124+
expect(roundTrip(roundTrip(input))).toBe(roundTrip(input))
125+
})
126+
127+
it('a paragraph starting with a non-block-list inline tag is NOT captured as a raw block', () => {
128+
// `em`/`a` aren't in the CommonMark block-HTML tag whitelist — they can legitimately start an
129+
// ordinary paragraph, and must keep parsing as real marks, not freeze as raw source.
130+
expect(topLevelTypes('<em>hi</em> there, this is a normal paragraph')).toEqual(['paragraph'])
131+
expect(roundTrip('<em>hi</em> there')).toBe('*hi* there')
132+
})
133+
134+
it('a stray inline-only tag alone on its own line is left to the stock (non-whitelisted) path', () => {
135+
// `<span>` isn't in the block whitelist, so the new block tokenizer must not claim it — it falls
136+
// through to marked's own (stricter) block-HTML detection, unaffected by this change.
137+
const input = '<span>\n\nnot a block-html tag\n\n</span>'
138+
expect(() => roundTrip(input)).not.toThrow()
139+
})
140+
141+
it('an unterminated block tag falls back gracefully (no crash, no infinite loop)', () => {
142+
const input = '<details>\n<summary>never closed</summary>\n\nbody'
143+
expect(() => roundTrip(input)).not.toThrow()
144+
})
145+
146+
it('a block comment spanning blank lines still round-trips via the new shared tokenizer path', () => {
147+
const input = '<!--\n\nmulti-line comment\n\n-->\n\ntext after'
148+
expect(topLevelTypes(input)[0]).toBe('rawHtmlBlock')
149+
expect(roundTrip(input)).toBe(input)
150+
})
151+
152+
it('a table and code block adjacent to a fragmenting-prone details block still coexist correctly', () => {
153+
const input =
154+
'<details>\n<summary>s</summary>\n\nbody\n\n</details>\n\n| a | b |\n| --- | --- |\n| 1 | 2 |\n\n```js\nconst x = 1\n```'
155+
expect(roundTrip(input)).toBe(input)
156+
})
157+
158+
it('preserves an indented (up to 3 spaces) block-HTML opening line', () => {
159+
// `roundTrip`'s `.trim()` would strip the very leading indent this test verifies, so check the
160+
// parsed node's own text (and the untrimmed serialization) instead of the trimmed helper.
161+
for (const indent of [' ', ' ', ' ']) {
162+
const input = `${indent}<details>\n<summary>x</summary>\n\nbody\n\n</details>`
163+
const doc = parseMarkdownToDoc(input)
164+
expect(doc.content?.map((n) => n.type)).toEqual(['rawHtmlBlock'])
165+
expect(doc.content?.[0].content?.[0].text).toBe(input)
166+
expect(serializeMarkdownDocument(input)).toBe(`${input}\n`)
167+
}
168+
})
169+
170+
it('preserves a quoted attribute value containing a literal >', () => {
171+
const input = '<div data-example="a > b">\n\ncontent\n\n</div>'
172+
expect(topLevelTypes(input)).toEqual(['rawHtmlBlock'])
173+
expect(roundTrip(input)).toBe(input)
174+
expect(roundTrip(roundTrip(input))).toBe(roundTrip(input))
175+
})
176+
177+
it('a quoted attribute containing a nested same-tag mention does not confuse the balance scan', () => {
178+
// Without attribute-aware matching, `<div>` inside the quoted value below would be miscounted as
179+
// a real nested open tag, throwing off the depth count entirely.
180+
const input = '<div title="a <div> b">\n\ncontent\n\n</div>'
181+
expect(topLevelTypes(input)).toEqual(['rawHtmlBlock'])
182+
expect(roundTrip(input)).toBe(input)
183+
})
184+
185+
it('does not mistake a tag name mentioned inside an inline code span for a real closing tag', () => {
186+
const input = '<details>\n<summary>x</summary>\n\nSee `</details>` in the docs.\n\n</details>'
187+
expect(topLevelTypes(input)).toEqual(['rawHtmlBlock'])
188+
expect(roundTrip(input)).toBe(input)
189+
})
190+
191+
it('does not mistake a tag name mentioned inside a fenced code block for a real closing tag', () => {
192+
const input = '<div>\n\nExample:\n\n```html\n<div>example</div>\n```\n\nmore body\n\n</div>'
193+
expect(topLevelTypes(input)).toEqual(['rawHtmlBlock'])
194+
expect(roundTrip(input)).toBe(input)
195+
})
196+
197+
it('does not mistake a tag name mentioned inside an HTML comment for a real closing tag', () => {
198+
const input = '<div>\n\n<!-- see </div> below for the closing tag -->\n\nmore body\n\n</div>'
199+
expect(topLevelTypes(input)).toEqual(['rawHtmlBlock'])
200+
expect(roundTrip(input)).toBe(input)
201+
})
202+
203+
it('a bare (unescaped, un-fenced) tag-name mention never crashes and always converges to a stable save', () => {
204+
// Known, inherent limitation of regex-based (non-DOM) tag matching, shared by any HTML-block
205+
// scanner (and by real HTML parsers given the same ambiguous input) — a bare mention outside
206+
// code can still be misread as the real closer. The bar this file holds itself to is: never
207+
// crash, never lose text, and always settle to a fixpoint after one save (isRoundTripSafe's own
208+
// documented tolerance for single-pass normalization) — not a perfect, DOM-aware parse.
209+
const input =
210+
'<details>\n<summary>x</summary>\n\nSee the literal text </details> in docs.\n\nmore body\n\n</details>'
211+
expect(() => roundTrip(input)).not.toThrow()
212+
const once = roundTrip(input)
213+
const twice = roundTrip(once)
214+
expect(once).toBe(twice)
215+
// No word from the original is dropped, even though the structure/whitespace may be reflowed.
216+
for (const word of ['See', 'the', 'literal', 'text', 'in', 'docs', 'more', 'body']) {
217+
expect(once).toContain(word)
218+
}
219+
})
220+
221+
it('does not mistake a tag name mentioned inside a blockquoted fenced code block for a real closing tag', () => {
222+
const input =
223+
'<div>\n\n> Example:\n>\n> ```html\n> <div>example</div>\n> ```\n\nmore body\n\n</div>'
224+
expect(topLevelTypes(input)).toEqual(['rawHtmlBlock'])
225+
expect(roundTrip(input)).toBe(input)
226+
})
227+
228+
it('does not mistake a tag name mentioned inside an indented (no blockquote) fenced code block for a real closing tag', () => {
229+
const input =
230+
'<div>\n\nExample:\n\n ```html\n <div>example</div>\n ```\n\nmore body\n\n</div>'
231+
expect(topLevelTypes(input)).toEqual(['rawHtmlBlock'])
232+
expect(roundTrip(input)).toBe(input)
233+
})
234+
235+
it('treats a void block tag (no closing tag exists) as complete right after the open tag', () => {
236+
// `link`/`meta`/`base`/`hr` are in the CommonMark block-HTML whitelist but are void elements —
237+
// scanning for a `</meta>` that will never legitimately appear would risk grabbing unrelated
238+
// later content (or a stray same-name mention) into the block.
239+
for (const input of [
240+
'<link rel="stylesheet" href="x.css">\n\nafter',
241+
'<meta charset="utf-8">\n\nafter',
242+
'<hr>\n\nafter',
243+
]) {
244+
const doc = parseMarkdownToDoc(input)
245+
expect(doc.content?.[0].type).toBe('rawHtmlBlock')
246+
expect(roundTrip(input)).toContain('after')
247+
}
248+
})
249+
250+
it('a void block tag does not swallow a later, unrelated mention of its own tag name', () => {
251+
const input = '<meta charset="utf-8">\n\nSee the `<meta>` tag in docs.\n\nmore body'
252+
const doc = parseMarkdownToDoc(input)
253+
// The <meta> is its own complete block; the later mention (in code) stays in a separate paragraph.
254+
expect(doc.content?.[0].type).toBe('rawHtmlBlock')
255+
expect(doc.content?.[0].content?.[0].text).toBe('<meta charset="utf-8">')
256+
expect(roundTrip(input)).toContain('more body')
257+
})
258+
})

0 commit comments

Comments
 (0)