Repository navigation
BigQuery: Cannot list jobs when there has been an invalid job request #3178
Description
I recently ran an invalid job outside of the Java BigQuery client library, and when I try to list jobs when running the BigQuery samples tests with the Java client library, I get an error:
Running com.google.cloud.examples.bigquery.snippets.ITBigQuerySnippets
Tests run: 15, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 162.203 sec <<< FAILURE! - in com.google.cloud.examples.bigquery.snippets.ITBigQuerySnippets
testJob(com.google.cloud.examples.bigquery.snippets.ITBigQuerySnippets) Time elapsed: 1.602 sec <<< ERROR!
java.lang.IllegalArgumentException: value must be set if and only if the type is not ARRAY
at com.google.cloud.examples.bigquery.snippets.ITBigQuerySnippets.testJob(ITBigQuerySnippets.java:273)
This error comes from from the Java BigQuery client library when it attempts to build an invalid job (error defined here)
This issue is similar to one we fixed in Python. The solution we used was to not validate Job objects as they are created from the API, which allowed invalid jobs from the API to be constructed into Job objects.
Activity
alixhami commentedon Apr 20, 2018
cc @tswast
tswast commentedon Apr 20, 2018
Presumably Java will need a slightly different solution from what we did in Python, as it is statically typed and can't really delay when it attempts to parse the various properties from the API response (or maybe I'm wrong about that?).
The Job resource provides a status.errors[] list (exposed in Java as JobStatus.getExecutionErrors()), so maybe we could parse to a default value in the case of errors with the expectation that they can check this property when listing / getting a failed job.
pongad commentedon Apr 20, 2018
@tswast I could be wrong but I'm not sure that would work. I think there's a race between the job finishing (either failing or erroring) and calling get/list.
For example, we could call get() , attempt to construct the QueryJobConfiguration object, and throw this exception before BigQuery realizes that the job fails. We could wait for the job to get to
RUNNINGorDONEstate before trying to parse the configuration but I think this sounds like a breaking change. The user could count on the configuration always existing before.I think another way out is to allow creation of invalid
QueryParameterValuejobs. We can provideisValidmethod and document that we allow creating invalid jobs for the reasons in the issue.tswast commentedon Apr 20, 2018
This doesn't cover the general case that it might be possible to create other kinds of bad job resources (having trouble of thinking what those would be), but it'll at least allow Alix and I to be able to run the system tests again.
bigquery: remove QueryParameterValue validation (#3179)
deps: update dependency com.google.cloud:sdk-platform-java-config to …
deps: update github/codeql-action action to v2.24.6 (#3178)
chore(main): release 2.38.2 (#3191)
deps: update dependency com.google.cloud:sdk-platform-java-config to …