Skip to content

fix(bqjdbc): optimize meetsReadRatio latency to achieve faster page counting#13090

Merged
Neenu1995 merged 4 commits into
mainfrom
perf/optimize-meets-read-ratio
May 12, 2026
Merged

fix(bqjdbc): optimize meetsReadRatio latency to achieve faster page counting#13090
Neenu1995 merged 4 commits into
mainfrom
perf/optimize-meets-read-ratio

Conversation

@Neenu1995
Copy link
Copy Markdown
Contributor

@Neenu1995 Neenu1995 commented May 8, 2026

b/511231568

  • Collection size check inside meetsReadRatio — Speeds up page counting by 300x by replacing a slow $O(N)$ loop over 10,000 rows with a fast $O(1)$ collection size call.
  • All-data-already-fetched short-circuit safeguard (totalRows <= pageSize) — Reduces E2E latency by 30%-40% by preventing redundant gRPC stream setups and double-fetching when results are already fully loaded in memory.
  • O(1) page size approximation fallback — Prevents slow iterations on custom non-Collection iterables by using a fast math fallback (Math.min(totalRows, maxResultPerPage)).
  • Updated default HighThroughputMinTableSize to 10,000 — Ensures small queries under 10,000 rows bypass gRPC connection setup overhead and run on the faster REST execution path.
  • Added 4 comprehensive unit tests — Validates all optimization, fallback, and safeguard paths under various dataset sizes and configurations.

@Neenu1995 Neenu1995 requested review from a team as code owners May 8, 2026 18:58
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the BigQuery JDBC driver by deprecating several legacy methods, optimizing the High Throughput API activation logic, and refining time/timestamp coercion. Key changes include replacing an O(N) iteration for page size calculation with an O(1) approximation in BigQueryStatement, increasing the default minimum table size for High Throughput API usage, and adjusting java.sql.Time and java.sql.Timestamp conversions to ensure consistent timezone handling and precision. Review feedback highlights potential overflow and division-by-zero issues in the new page size approximation logic, as well as an inconsistency in timezone offset adjustments between different time coercion paths.

@Neenu1995 Neenu1995 force-pushed the perf/optimize-meets-read-ratio branch from 5638e32 to 8e18032 Compare May 8, 2026 19:16
}

// Prevent division by zero or negative pageSize due to potential overflows/empty sets:
if (pageSize <= 0) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's an impossible condition. It can't be < 0 and if totalRows is 0 is already checked

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think we don't need this. getMaxResultPerPage() shouldn't be <= 0. We can add this check during connection creation, but we shouldn't check it when we use it

}

// Prevent division by zero or negative pageSize due to potential overflows/empty sets:
if (pageSize <= 0) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think we don't need this. getMaxResultPerPage() shouldn't be <= 0. We can add this check during connection creation, but we shouldn't check it when we use it

@Neenu1995 Neenu1995 enabled auto-merge (squash) May 12, 2026 15:37
@Neenu1995 Neenu1995 merged commit 36af3eb into main May 12, 2026
167 of 168 checks passed
@Neenu1995 Neenu1995 deleted the perf/optimize-meets-read-ratio branch May 12, 2026 16:52
blakeli0 added a commit that referenced this pull request Jun 3, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>1.87.0</summary>

##
[1.87.0](v1.86.0...v1.87.0)
(2026-06-03)


### ⚠ BREAKING CHANGES

* [container] remove image_family from
* [saasservicemgmt] Changed field behavior for an existing field

### Features

* [admanager] added new API dimension:
([39e93fe](39e93fe))
* [aiplatform] Release
([39e93fe](39e93fe))
* [aiplatform] Release
([39e93fe](39e93fe))
* [asset] add authentication scope for Policy Analyzer MCP tool
([9c90001](9c90001))
* [backstory] new module for backstory
([#13334](#13334))
([fd8f666](fd8f666))
* [bigquerymigration] add support for assessment tasks in BigQuery
([9c90001](9c90001))
* [chat] Support force notify and silent notification option for
([39e93fe](39e93fe))
* [cloudsupport] v2 version for `GetComment` and `GetAttachment`
([39e93fe](39e93fe))
* [compute] Update Compute Engine v1 API to revision 20260422
([9c90001](9c90001))
* [container] add confidential instance type and hyperdisk support
([39e93fe](39e93fe))
* [container] add confidential instance type and hyperdisk support
([39e93fe](39e93fe))
* [container] add custom node image configuration
([9c90001](9c90001))
* [databasecenter] add a AggregateQueryStats API
([9c90001](9c90001))
* [datalineage] A new method SearchLineageStreaming is added
([39e93fe](39e93fe))
* [datamanager] add encrypted_user_id.proto and the EncryptedUserId
([9c90001](9c90001))
* [datamanager] add the coupon_codes field to CartData to track
([9c90001](9c90001))
* [dataproc] add support for Cloud Resource Manager tags for
([39e93fe](39e93fe))
* [developerknowledge] new module for developerknowledge
([#13313](#13313))
([502841b](502841b))
* [dlp] Supporting Conversation Content Type
([9c90001](9c90001))
* [health] Introduce Data Subscription Service for managing webhook
([9c90001](9c90001))
* [health] introduce MedicalDeviceInfo message for Software as
([9c90001](9c90001))
* [maps-places] add transit_station data to Places API (New)
([9c90001](9c90001))
* [network-management] add fields and messages for Cloud Run jobs
([9c90001](9c90001))
* [network-security] add networksecurity v1 client library
([9c90001](9c90001))
* [policy-troubleshooter] add authentication scope for
([9c90001](9c90001))
* [recommender] Support utilization insights and cloud hub
([9c90001](9c90001))
* [security-private-ca] add
([9c90001](9c90001))
* [storage] add support for GCS Storage Control Intelligence Finding
([9c90001](9c90001))
* add duration to health metrics
([#13181](#13181))
([ae86384](ae86384))
* add periodic WARNING metrics to assist in debugging
([#12976](#12976))
([c9e2e6d](c9e2e6d))
* add v1 version support for geminidataanalytics
([#13314](#13314))
([946dfb4](946dfb4))
* **bigquery-jdbc:** implement JDBC getObject type coercion support
([#13257](#13257))
([72d4ab6](72d4ab6))
* **bqjdbc:** Add custom logger to handle ResultSet logs
([#13034](#13034))
([887c67f](887c67f))
* **bqjdbc:** implement Per connection logging with Context proxy
([#13001](#13001))
([293f18f](293f18f))
* **dataplex:** Add `RequestDataProductAccess` method to request access
([9c90001](9c90001))
* migrate grpc-gcp to maven
([#237](#237))
([d787372](d787372))
* **spanner:** add getIsolationLevel and getReadLockMode methods to
CommitResponse
([#13004](#13004))
([d5fc504](d5fc504))
* **spanner:** add option for auto-tagging transactions
([#13214](#13214))
([c8234cf](c8234cf))
* **spanner:** add option for inline begin with multi-use read only txn
([#13233](#13233))
([056abb6](056abb6))


### Bug Fixes

* [container] remove image_family from
([9c90001](9c90001))
* [saasservicemgmt] Changed field behavior for an existing field
([9c90001](9c90001))
* **bigquery-jdbc:** Add escape character support for pattern matching
([#13259](#13259))
([af3b7c5](af3b7c5))
* **bigquery-jdbc:** configure logging early to capture URL parsing
warnings and errors
([#13323](#13323))
([98d8e69](98d8e69))
* **bigquery-jdbc:** fallback to standard precision and scale for
numeric, integer, and temporal types
([#13321](#13321))
([51ba902](51ba902))
* **bigquery-jdbc:** fix manual commit mode failure for non-US regions
([#13285](#13285))
([d28436c](d28436c))
* **bigquery-jdbc:** handle EXPORT DATA, EXPORT MODEL, and LOAD DATA
statements
([#13267](#13267))
([5970866](5970866))
* **bigquery-jdbc:** implement JDBC wrapper interface methods
([#13322](#13322))
([46d6a86](46d6a86))
* **bigquery:** prevent NullPointerException in OTel tracing when JobId
is null
([#13301](#13301))
([dcc2a68](dcc2a68))
* **bigtable:** Don't invoke unknown delegate code while holding a lock
([#13202](#13202))
([a7e859e](a7e859e))
* **bigtable:** Ensure that FallbackChannelPool locks doesnt leak to
alien listeners
([#13195](#13195))
([c8e615e](c8e615e))
* **bigtable:** make heartbeat interval volatile
([#13217](#13217))
([279918e](279918e))
* **bigtable:** make ignoreError volatile
([#13218](#13218))
([e5e5f14](e5e5f14))
* **bigtable:** prune older AFEs first
([#13215](#13215))
([114e77b](114e77b))
* **bigtable:** recycle channel on consecutive new stream failures
([#13245](#13245))
([842f64e](842f64e))
* **bigtable:** resolve check-then-act race condition in VRpcImpl
([#13196](#13196))
([9853dd5](9853dd5))
* **bigtable:** resolve deadlock in ClientConfigurationManager when
notifying callbacks
([#13192](#13192))
([c98e53d](c98e53d))
* **bqjdbc:** avoid reusing statement in DatabaseMetaData
([#13224](#13224))
([b1279ab](b1279ab))
* **bqjdbc:** fallback to RestAPI if ReadAPI is not accessible
([#13018](#13018))
([cb9bdc3](cb9bdc3))
* **bqjdbc:** fix BigDecimal usage in mocks
([#13207](#13207))
([65674c2](65674c2))
* **bqjdbc:** fix Long to java.sql.Time coercion
([#13035](#13035))
([8ab9799](8ab9799))
* **bqjdbc:** optimize meetsReadRatio latency to achieve faster page
counting
([#13090](#13090))
([36af3eb](36af3eb))
* **bqjdbc:** Pass connection ID to DataType class loggers
([#13197](#13197))
([ce9c504](ce9c504))
* **bqjdbc:** resolve JDBC metadata compliance gaps for wrappers,
searchability, and type mappings
([#13242](#13242))
([5678fc2](5678fc2))
* **bqjdbc:** update metadata values for GEOGRAPHY/JSON/INTERVAL types
([#13223](#13223))
([d08691a](d08691a))
* **bqjdbc:** update shading to be more targeted
([#13232](#13232))
([cc691a8](cc691a8))
* **bqjdbc:** validate integer connection properties
([#13174](#13174))
([875ff6a](875ff6a))
* **build:** support generating unversioned libraries in OwlBot config
([#13336](#13336))
([90d6661](90d6661))
* **ci:** correct directory typo in shared-dependencies workflow
([#13225](#13225))
([a327bf2](a327bf2))
* **firestore:** increase ITShutdownTest timeout to 50s
([#13199](#13199))
([d432b35](d432b35))
* **gapic-generator:** allow non-versioned proto paths and correct help
text
([#13329](#13329))
([13717a2](13717a2))
* **generator:** align template and post-processor formatting
([#13028](#13028))
([6a2307f](6a2307f))
* **hermetic_build:** add grpc-gcp-java to module allowlist in root pom
generator
([#13211](#13211))
([f7c8407](f7c8407))
* **hermetic_build:** restore Version.java after postprocessing
entrypoint
([#13328](#13328))
([ee3e9cd](ee3e9cd))
* **hermetic:** exclude showcase modules from versions.txt
([#13030](#13030))
([6ce87b7](6ce87b7))
* **iam-admin:** remove redundant IAM project service to prevent state
conflict
([#13016](#13016))
([95cfd5e](95cfd5e))
* remove google-cloud-bigtable-deps-bom from gapic-libraries-bom
([#13209](#13209))
([ce156f2](ce156f2))
* **spanner:** avoid double grpc-gcp wrapping for directpath fallback
([#13155](#13155))
([fe608fe](fe608fe))
* **spanner:** cache auto-tagging options to avoid system property lock
contention
([#13273](#13273))
([dc2feed](dc2feed))
* **spanner:** derive built-in metrics project from database client
([#13262](#13262))
([a577934](a577934))


### Dependencies

* update dependency com.google.cloud:sdk-platform-java-config to v3.62.0
([#2379](#2379))
([e9ec6e0](e9ec6e0))
* update dependency com.google.cloud:sdk-platform-java-config to v3.62.0
([#2729](#2729))
([af4893b](af4893b))
* update googleapis/google-cloud-java action to v1.86.0
([#2377](#2377))
([f2de1c3](f2de1c3))
* update googleapis/google-cloud-java action to v1.86.0
([#2727](#2727))
([d16eda9](d16eda9))
* update shared dependencies
([#2910](#2910))
([8cc8ae0](8cc8ae0))


### Documentation

* add root-level development.md guide for scoped builds
([#13151](#13151))
([bcec186](bcec186))
* **agents:** add API lifecycle and stability guidelines skill
([#13179](#13179))
([9110ad6](9110ad6))
* Update gRPC spelling in READMEs
([#13264](#13264))
([d6a93ee](d6a93ee))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: cloud-java-bot <cloud-java-bot@google.com>
Co-authored-by: blakeli <blakeli@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants