Skip to content

Ignore blank lines when reading JSONL#1479

Open
biefan wants to merge 1 commit intoAzure:mainfrom
biefan:fix-jsonl-blank-lines
Open

Ignore blank lines when reading JSONL#1479
biefan wants to merge 1 commit intoAzure:mainfrom
biefan:fix-jsonl-blank-lines

Conversation

@biefan
Copy link
Contributor

@biefan biefan commented Mar 16, 2026

Summary

This makes read_jsonl() tolerate blank lines in JSONL input.

Problem

read_jsonl() currently does a direct json.loads(line) for every line in the file.

That means a JSONL file with an empty line or a whitespace-only separator fails immediately with JSONDecodeError, even when every non-empty line is valid JSON.

In practice, blank lines are common in generated or hand-edited JSONL files, especially at the end of a file or between entries during review.

Fix

Skip blank / whitespace-only lines before decoding:

json.loads(line) for line in file if line.strip()

This preserves the existing behavior for valid JSONL records while making the reader robust to common formatting noise.

Tests

Added regression coverage for a JSONL stream containing a blank line between two valid entries.

Validation command:

.venv/bin/pytest tests/unit/common/test_json_helper.py -q

Validation result:

1 passed in 0.01s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants