feat: delegate ReadRowsPage.to_arrow to pandas_gbq.arrow

[shuoweil]

Working in progress branch

Delegates ReadRowsPage.to_arrow() in google-cloud-bigquery-storage to pandas_gbq.arrow.from_read_rows_response when pandas-gbq is installed, decoupling protobuf record batch parsing into pandas-gbq.

Changes Made

1. pandas-gbq (packages/pandas-gbq):
   • Added pandas_gbq.arrow.from_read_rows_response(message, arrow_schema=None) to parse ReadRowsResponse protobuf messages into pyarrow.RecordBatch instances via IPC stream reading.
   • Exported arrow submodule in pandas_gbq.__all__.
   • Added unit test suite in packages/pandas-gbq/tests/unit/test_arrow.py.
2. google-cloud-bigquery-storage (packages/google-cloud-bigquery-storage):
   • Updated ReadRowsPage.to_arrow() in google/cloud/bigquery_storage_v1/reader.py to delegate to pandas_gbq.arrow.from_read_rows_response.
   • Added a PendingDeprecationWarning informing users of direct pandas-gbq usage.
   • Retained inline stream parser fallback when pandas-gbq is not present.
   • Added explicit unit tests in packages/google-cloud-bigquery-storage/tests/unit/test_reader_v1_arrow.py.

Fixes #<526614511> 🦕

[gemini-code-assist]

Code Review

This pull request deprecates direct retrieval of Arrow record batches via google-cloud-bigquery-storage and delegates this functionality to a new arrow submodule in pandas-gbq. The review feedback highlights several critical issues: 1) pyarrow.RecordBatch does not have a from_batches method, so from_arrays should be used instead; 2) importing pyarrow at the top level of pandas_gbq/arrow.py will break imports for users who do not have this optional dependency installed, requiring a conditional import and string type annotations; and 3) reader.read_next_batch() can raise a StopIteration exception if the stream is empty, which needs to be handled gracefully.

[gemini-code-assist]

Since pyarrow is an optional dependency for pandas-gbq, importing it at the top level of this module will cause import pandas_gbq to fail with an ImportError for any user who does not have pyarrow installed. We should wrap the import in a try-except block and raise a descriptive error only when the function is called. Additionally, when assigning None in the except ImportError block, type: ignore[assignment] is necessary to prevent a mypy error.

try:
    import pyarrow as pa
except ImportError:
    pa = None  # type: ignore[assignment]

References

1. When handling an optional dependency by assigning None in an except ImportError block, type: ignore[assignment] is necessary on the assignment to prevent a mypy error, as using typing.TYPE_CHECKING alone is insufficient.

[shuoweil]

Fixed. Wrapped the optional pyarrow import in a try...except ImportError block with pa = None # type: ignore[assignment] in pandas_gbq/arrow.py, and added a runtime check in from_read_rows_response that raises a descriptive ImportError when pyarrow is missing. Added test coverage in test_arrow.py.