You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
How to migrate to 0.3 for APIs that expect PathLike #460
importupathimportpolarsp=upath.UPath("s3://bucket/file", anon=True)
object_store_storage_options= ... # convert your fsspec p.storage_options to the object store stylepolars.scan_parquet(p.as_uri(), storage_options=object_store_storage_options)
Unless it's a bug and pyarrow should handle UPaths?
Incidentally I thought path.as_uri() was same as str(path) but looking at the code there is a guard for relative paths which is probably worth documenting too
Activity
ap-- commentedon Oct 27, 2025
Hi @Zan-L
cross-referencing: pola-rs/polars#24921
In the example you provided, they converted remote UPaths to a string via indirectly calling os.fspath() via os.path.expanduser() see here: https://github.com/pola-rs/polars/blob/77254fce44d7b848aa9aa8bb3a0f01f5d2e63d1c/py-polars/src/polars/_utils/various.py#L235
It's better to be explicit about how you're calling this API, instead of relying on the UPath being cast to string (and losing its storage options).
Your example code is effectively equivalent to:
Warning
But be aware !! you're dropping all storage_options in the general case (in your example there are none).
I would recommend to use it like this, if you want to use fsspec and s3fs as the read backend (probably slower than polars default):
Or if you want to use the polars rust backends
Warning
Because polars uses object-store instead of fsspec and s3fs, the storage options have different names and you have to convert: https://docs.rs/object_store/latest/object_store/aws/enum.AmazonS3ConfigKey.html
I'll add this to the migration guide when I find some time.
Cheers,
Andreas
cas-- commentedon Dec 10, 2025
Ah this explains why I am encountering new errors upgrading to latest version of UPath passed to
pandas.read_parquetandpyarrow.Including the error here to help anyone else encountering it:
Considering that pyarrow is fsspec compatible I assume this would be the recommended approach:
Unless it's a bug and pyarrow should handle UPaths?
Incidentally I thought
path.as_uri()was same asstr(path)but looking at the code there is a guard for relative paths which is probably worth documenting toouniversal_pathlib/upath/core.py
Lines 1955 to 1961 in 63b6f1c