Skip to content

inspect.entries() raises TypeError when a data file has no column_sizes #4186

Open

Description

@RyanLin5967

Apache Iceberg version

0.12.0 (latest release)

Please describe the bug 🐞

Currently, entries() builds column_sizes with dict(entry.data_file.column_sizes), so an entry without that optional field raises TypeError. files() handles it, and #1902 added or {} to the three count columns beside it for #1884.

import os, tempfile
import pyarrow as pa, pyarrow.parquet as pq
from pyiceberg.catalog.sql import SqlCatalog
from pyiceberg.manifest import DataFile, DataFileContent, FileFormat
from pyiceberg.typedef import Record

d = tempfile.mkdtemp()
cat = SqlCatalog("c", uri="sqlite:///:memory:", warehouse=f"file://{d}")
cat.create_namespace("ns")
t = cat.create_table("ns.t", schema=pa.schema([("x", pa.int64())]))

pq.write_table(pa.table({"x": [1]}), f"{d}/a.parquet")
f = DataFile.from_args(content=DataFileContent.DATA, file_path=f"file://{d}/a.parquet", file_format=FileFormat.PARQUET,
                       partition=Record(), record_count=1, file_size_in_bytes=os.path.getsize(f"{d}/a.parquet"))
with t.transaction() as tx:
    with tx.update_snapshot().fast_append() as fa:
        fa.append_data_file(f)

t.inspect.entries()
# TypeError: 'NoneType' object is not iterable, should be one row with column_sizes None

DuckDB 1.5.6 leaves column_sizes out of the data and delete files it writes, so entries() fails on tables it wrote.

tested on pyiceberg 0.12.0 and main

Willingness to contribute

  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time

Activity

  1. aaryasb3101 commented on Oct 31, 2026

    @aaryasb3101

    Hi! I'd be interested in contributing a fix for this issue.

    I noticed that the other optional count columns already appear to handle missing values, while column_sizes does not.

    I'd like to investigate making the handling consistent and add a regression test for data files where column_sizes is absent.

    Is anyone already working on this, or would a contribution be welcome?

  2. RyanLin5967 commented on Oct 31, 2026

    @RyanLin5967
    Author

    I'm not working on it, go ahead. On main it's still dict(entry.data_file.column_sizes) in entries() in pyiceberg/table/inspect.py. files() already uses dict(...) if ... is not None else None there, which gives the None the repro expects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions