Issues

• Reduce per-batch allocation in broadcast serialization

  requires-triage
  apache/datafusion-comet#7309 · Filter by author sunchao opened on Oct 28, 2026

• Executor JVM abort (NMT footer canary broken) when the in-memory cache is enabled on Spark 3.4

  area:scanbugrequires-triage
  apache/datafusion-comet#7305 · Filter by author andygrove opened on Oct 28, 2026
  1

  

• JVM columnar shuffle writers use Comet defaults in map stages of jobs outside a SQL execution

  bugrequires-triage
  apache/datafusion-comet#7303 · Filter by author comphead opened on Oct 28, 2026

• Map stages of RDD-level jobs run Comet with default settings instead of the session's

  bugrequires-triage
  apache/datafusion-comet#7301 · Filter by author comphead opened on Oct 28, 2026

• [EPIC] Native Spark expressions: run as many Spark built-in expressions as possible natively, by default

  area:expressionsenhancementEPICrequires-triage
  apache/datafusion-comet#7300 · Filter by author andygrove opened on Oct 28, 2026 · 2.0.0
  

• perf: report vectorTypes from the Arrow cached batch serializer

  area:scanenhancementperformancerequires-triage
  apache/datafusion-comet#7298 · Filter by author andygrove opened on Oct 28, 2026 · 1.3.0

• perf: store dictionary-encoded columns as dictionaries in the in-memory cache

  area:scanenhancementperformancerequires-triage
  apache/datafusion-comet#7297 · Filter by author andygrove opened on Oct 28, 2026 · 1.3.0

• perf: decode cached batches in native code instead of on the JVM

  area:scanenhancementperformancerequires-triage
  apache/datafusion-comet#7296 · Filter by author andygrove opened on Oct 28, 2026 · 1.3.0

• [EPIC] Iceberg work for 1.3.0: native writes on by default and scan correctness

  area:IcebergEPICrequires-triage
  apache/datafusion-comet#7293 · Filter by author andygrove opened on Oct 28, 2026 · 1.3.0
  

• [EPIC] Comet Native Executors: run Spark queries on native executors built on DataFusion Ballista

  enhancementEPICrequires-triage
  apache/datafusion-comet#7288 · Filter by author andygrove opened on Oct 28, 2026 · 2.0.0
  

• make_date under a codegen-dispatched parent falls back to Spark since #7261

  requires-triage
  apache/datafusion-comet#7284 · Filter by author dwsmith1983 opened on Oct 28, 2026
  2

  

• Offset overflow - Filter merge buffers are not tracked by the memory pool

  bug
  apache/datafusion-comet#7279 · Filter by author parthchandra opened on Oct 27, 2026
  

• Offset overflow - MergeRows and aggregate emit build batches larger than the batch size

  bug
  apache/datafusion-comet#7278 · Filter by author parthchandra opened on Oct 27, 2026
  

• Offset overflow - In-memory table cache ignores spark.comet.batchSize

  bug
  apache/datafusion-comet#7277 · Filter by author parthchandra opened on Oct 27, 2026
  

• Offset overflow - Broadcast exchange merges every batch into one

  bug
  apache/datafusion-comet#7276 · Filter by author parthchandra opened on Oct 27, 2026
  

• Offset overflow - Read large string/binary columns as view types

  bug
  apache/datafusion-comet#7275 · Filter by author parthchandra opened on Oct 27, 2026
  

• Offset overflow - BoundedWindowAggExec concatenates the unfinished partition buffer

  bug
  apache/datafusion-comet#7274 · Filter by author parthchandra opened on Oct 27, 2026
  

• Offset overflow - Hash join and nested loop join build side concatenated into one batch (upstream)

  bug
  apache/datafusion-comet#7273 · Filter by author parthchandra opened on Oct 27, 2026
  

• Offset overflow - TopK emit interleaves all K rows into one batch (upstream)

  bug
  apache/datafusion-comet#7272 · Filter by author parthchandra opened on Oct 27, 2026
  

• Offset overflow - Byte limit in arrow BatchCoalescer and DataFusion LimitedBatchCoalescer (upstream)

  bug
  apache/datafusion-comet#7271 · Filter by author parthchandra opened on Oct 27, 2026
  

• Offset overflow - WindowAggExec concatenates the whole task input into one batch

  bug
  apache/datafusion-comet#7270 · Filter by author parthchandra opened on Oct 27, 2026
  

• Offset overflow- Native shuffle writer: retry with fewer rows on offset overflow

  requires-triage
  apache/datafusion-comet#7269 · Filter by author parthchandra opened on Oct 27, 2026
  

• Offset overflow - Pass the session batch size to native FilterExec

  bug
  apache/datafusion-comet#7268 · Filter by author parthchandra opened on Oct 27, 2026
  

• [EPIC] Native operators overflow Arrow's 2 GiB offset limit on large string/binary columns

  bug
  apache/datafusion-comet#7267 · Filter by author parthchandra opened on Oct 27, 2026
  

• perf: coalesce small native shuffle batches before broadcast serialization

  requires-triage
  apache/datafusion-comet#7257 · Filter by author sunchao opened on Oct 27, 2026