pyiceberg/manifest.py holds a module-level cache:
_manifest_cache = _ManifestCache()
Its docstring describes it accurately: "Process-wide ManifestFile cache keyed by manifest_path." The key is the path string alone — no catalog, table, session, or tenant forms part of it.
def get_or_cache(self, manifest_file: ManifestFile) -> ManifestFile:
...
manifest_path = manifest_file.manifest_path
if manifest_path in self._cache:
return self._cache[manifest_path]
self._cache[manifest_path] = manifest_file
return manifest_fileOn a hit it returns the cached object and discards the one just read. Since read_manifest_list constructs ManifestFile objects from the entries of a manifest list rather than by reading each manifest file, the cached object reflects whatever the first manifest list to name that path declared — including its partition summaries, counts and sequence numbers.
Two tables in the same process that reference the same manifest_path therefore share one ManifestFile, and the first read wins. Because those fields drive scan pruning, a stale or mismatched entry changes which files a scan considers.
Issue investigation generated via claude, reviewed by Sung, Kevin, Fokko.
pyiceberg/manifest.pyholds a module-level cache:Its docstring describes it accurately: "Process-wide ManifestFile cache keyed by manifest_path." The key is the path string alone — no catalog, table, session, or tenant forms part of it.
On a hit it returns the cached object and discards the one just read. Since
read_manifest_listconstructsManifestFileobjects from the entries of a manifest list rather than by reading each manifest file, the cached object reflects whatever the first manifest list to name that path declared — including its partition summaries, counts and sequence numbers.Two tables in the same process that reference the same
manifest_paththerefore share oneManifestFile, and the first read wins. Because those fields drive scan pruning, a stale or mismatched entry changes which files a scan considers.Issue investigation generated via claude, reviewed by Sung, Kevin, Fokko.