GH-47417: [C++] Fix JSON parser losing nulls in a null-typed list by advitrocks9 · Pull Request #51108 · apache/arrow

advitrocks9 · 2026-08-31T18:32:55Z

Rationale for this change

{"a": [null, 1]} silently parses to [[1]] in release builds, and aborts in debug.

This was introduced in b7054c21aab (0.14.0).

What changes are included in this PR?

AppendNull looks up its child from the parent and returns the updated BuilderPtr, instead of taking a copy from the caller. Null() writes it back to builder_, so builder_ no longer drifts from the parent's copy.

Are these changes tested?

Two new tests, BlockParser.NullsInList (inferred schema) and BlockParserWithSchema.NullsInList (explicit schema), parse lists of nulls, including {"a": [{"b": null}, null]}. Both fail in debug without the fix.

Are there any user-facing changes?

{"a": [null, 1]} now parses to [[null, 1]].

This PR contains a "Critical Fix". Well-formed JSON silently loses list elements.

GitHub Issue: [C++] Assertion fires when trying to parse json #47417

Was AI used for this PR?

In accordance to the AI generation guidelines, please disclose below whether and how AI was used in this PR.

PR code and description written by:

Human
AI

Reviewed before submission by:

Human
AI
Not reviewed

…a list

github-actions · 2026-08-31T18:33:20Z

⚠️ GitHub issue #47417 has been automatically assigned in GitHub to PR creator.

Copilot

Pull request overview

Fixes a long-standing JSON parsing bug in the C++ BlockParser where null elements could be silently dropped (or trigger DCHECKs in debug) when appending nulls to a null-typed list, by ensuring the inline null-count state in BuilderPtr is updated in-place.

Changes:

Change RawBuilderSet::AppendNull to take a BuilderPtr* so increments to the inline null-count for Kind::kNull persist to the caller.
Update all AppendNull call sites to pass builder pointers accordingly (including nested object-field null propagation).
Add a regression test covering nulls inside lists (null-only and promoted-from-null cases).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
cpp/src/arrow/json/parser.cc	Fixes null appends by mutating the `BuilderPtr` in-place, preventing lost null counts in null-typed list builders.
cpp/src/arrow/json/parser_test.cc	Adds a regression test to ensure nulls inside lists are preserved and don’t break list offsets.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

…ror and Ignore

github-actions · 2026-09-03T05:51:59Z

⚠️ GitHub issue #47417 has been automatically assigned in GitHub to PR creator.

Copilot

🟢 Approval recommended

The fix is narrowly scoped, updates all relevant call sites, and adds targeted regression tests exercising both the all-null and promotion scenarios described.

Review details

Files reviewed: 2/2 changed files
Comments generated: 0 new
Review effort level: Lite

advitrocks9 · 2026-09-18T12:09:35Z

@pitrou mind taking a look? the current red CI is from an existing known repowide issue

# Conflicts: # cpp/src/arrow/json/parser.cc # cpp/src/arrow/json/parser_test.cc

Copilot

🟢 Approval recommended

The focused fix addresses the reported state-loss bug and has regression coverage for both parsing paths.

1 open finding

The new regression test exercises the inferred-schema path (InferType) but doesn’t cover the…

🧠 Review effort: Lite

github-actions · 2026-10-08T19:39:36Z

⚠️ GitHub issue #47417 has been automatically assigned in GitHub to PR creator.

advitrocks9 · 2026-10-08T20:22:42Z

Merged main to fix the conflict with the simdjson rewrite. The fix itself is unchanged.

pitrou · 2026-10-21T08:09:57Z

@Reranko05 @taepper Since you both recently worked on the JSON reader, would you like to review this?

taepper · 2026-10-21T08:28:46Z

@advitrocks9 Thank you for the contribution! Would you mind filling out the AI disclosure section of the PR template?

### Was AI used for this PR?

In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR.

**PR code and description written by:**

- [ ] Human
- [ ] AI

**Reviewed before submission by:**

- [ ] Human
- [ ] AI
- [ ] Not reviewed

Reranko05

Does it make sense to also test a null struct with a null-typed child, like {"a": [{"b": null}, null]}? Existing tests don't cover the kNull case inside the recursive kObject branch of AppendNull.

Copilot

🟢 Approval recommended

No unresolved blocking issues were identified, and all supplied assessments indicate approval readiness.

No open issues 🎉

1 resolved since last review

The new regression test exercises the inferred-schema path (InferType) but doesn’t cover the…

_{Review effort: Lite}

github-actions · 2026-10-22T01:47:27Z

⚠️ GitHub issue #47417 has been automatically assigned in GitHub to PR creator.

advitrocks9 · 2026-10-22T01:51:24Z

@taepper @Reranko05 thanks for the review! I've gone ahead and addressed your concerns, and updated the PR body (:

taepper · 2026-10-22T06:18:34Z

@@ -966,7 +972,7 @@ class ParseImpl : public BlockParser {
      if (ARROW_PREDICT_FALSE(!field_builder.nullable)) {
        return ParseError("a required field was absent");
      }


We could use

if (ARROW_PREDICT_FALSE(!Cast<Kind::kObject>(parent)->field_builder(i).nullable)) { return ParseError("a required field was absent"); }

to avoid the variable field_builder

Done, thanks!

taepper · 2026-10-22T06:18:45Z

+    ARROW_ASSIGN_OR_RAISE(builder_,
+                          builder_set_.AppendNull(builder_stack_.back(), field_index_));
+    return Status::OK();


I like this a lot! :)

Copilot

🟢 Approval recommended

The reviewed changes address the null-list parsing defect and include comprehensive regression coverage with no unresolved review issues.

No open issues 🎉

_{Review effort: Lite}

apacheGH-47417: [C++] Fix JSON parser dropping nulls at the start of …

a5db8c8

…a list

advitrocks9 requested a review from pitrou as a code owner August 31, 2026 18:32

Copilot AI lite review requested due to automatic review settings August 31, 2026 18:32

github-actions Bot added Component: C++ awaiting review Awaiting review labels Aug 31, 2026

Copilot started reviewing on behalf of advitrocks9 August 31, 2026 18:33

View session

Copilot AI reviewed Aug 31, 2026

View reviewed changes

Comment thread cpp/src/arrow/json/parser_test.cc

apacheGH-47417: [C++] Cover the explicit list(null()) schema under Er…

609dc09

…ror and Ignore

Copilot AI review requested due to automatic review settings September 3, 2026 05:50

Copilot started reviewing on behalf of advitrocks9 September 3, 2026 05:51

View session

Copilot AI reviewed Sep 3, 2026

View reviewed changes

github-actions Bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Sep 3, 2026

Merge remote-tracking branch 'origin/main' into fix-apachegh-47417

1c376e9

# Conflicts: # cpp/src/arrow/json/parser.cc # cpp/src/arrow/json/parser_test.cc

Copilot AI lite review requested due to automatic review settings October 8, 2026 19:32

Copilot started reviewing on behalf of advitrocks9 October 8, 2026 19:32

View session

Copilot AI reviewed Oct 8, 2026

View reviewed changes

Reranko05 reviewed Oct 21, 2026

View reviewed changes

taepper reviewed Oct 21, 2026

View reviewed changes

Comment thread cpp/src/arrow/json/parser.cc Outdated

apacheGH-47417: [C++] Return the updated builder from AppendNull

2bc1be2

Copilot AI lite review requested due to automatic review settings October 22, 2026 01:33

Copilot started reviewing on behalf of advitrocks9 October 22, 2026 01:34

View session

Copilot AI reviewed Oct 22, 2026

View reviewed changes

advitrocks9 requested review from Reranko05 and taepper October 22, 2026 01:53

taepper reviewed Oct 22, 2026

View reviewed changes

apacheGH-47417: [C++] Inline the nullable check for absent fields

d29f49e

Copilot AI lite review requested due to automatic review settings October 24, 2026 13:58

Copilot started reviewing on behalf of advitrocks9 October 24, 2026 13:58

View session

Copilot AI reviewed Oct 24, 2026

View reviewed changes

Conversation

advitrocks9 commented Aug 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Was AI used for this PR?

Uh oh!

github-actions Bot commented Aug 31, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

github-actions Bot commented Sep 3, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

advitrocks9 commented Sep 18, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

github-actions Bot commented Oct 8, 2026

Uh oh!

advitrocks9 commented Oct 8, 2026

Uh oh!

pitrou commented Oct 21, 2026

Uh oh!

taepper commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reranko05 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

github-actions Bot commented Oct 22, 2026

Uh oh!

advitrocks9 commented Oct 22, 2026

Uh oh!

taepper Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

advitrocks9 Oct 24, 2026

Choose a reason for hiding this comment

Uh oh!

taepper Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

advitrocks9 commented Aug 31, 2026 •

edited

Loading

taepper commented Oct 21, 2026 •

edited

Loading