Skip to content

fix: Fixed getting project ID when running on Vertex AI; Fixes #852#943

Merged
sasha-gitg merged 13 commits intogoogleapis:mainfrom
Ark-kun:fix--Fixed-getitng-project-ID-when-running-on-Vertex-AI
Mar 21, 2022
Merged

fix: Fixed getting project ID when running on Vertex AI; Fixes #852#943
sasha-gitg merged 13 commits intogoogleapis:mainfrom
Ark-kun:fix--Fixed-getitng-project-ID-when-running-on-Vertex-AI

Conversation

@Ark-kun
Copy link
Copy Markdown
Contributor

@Ark-kun Ark-kun commented Jan 6, 2022 •
edited
Loading

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #852 🦕

@Ark-kun
fix: Fixed getitng project ID when running on Vertex AI
1e1cbfe 
When project ID is not explicitly specified in `aiplatform.init()` call, the SDK uses `google.auth.default()` to infer the project ID.
However when running under Vertex AI (CustomJob, PipelineJob), the project returned by `google.auth.default()` is not the correct user project.
See googleapis#852
See https://github.com/googleapis/google-auth-library-python/issues/924
This PR fixes the fallback to get the project ID from the `CLOUD_ML_PROJECT_ID` environment variable.
@Ark-kun Ark-kun requested a review from a team January 6, 2022 23:38
@Ark-kun Ark-kun changed the title fix: Fixed getitng project ID when running on Vertex AI; Fixes #852 fix: Fixed getting project ID when running on Vertex AI; Fixes #852 Jan 7, 2022
vinnysenthil
vinnysenthil suggested changes Jan 10, 2022
View reviewed changes
Comment thread setup.py Outdated
@Ark-kun Ark-kun force-pushed the fix--Fixed-getitng-project-ID-when-running-on-Vertex-AI branch from 4ecef80 to 5b63ac3 Compare February 3, 2022 04:21
@Ark-kun Ark-kun requested a review from vinnysenthil February 7, 2022 02:48
sasha-gitg
sasha-gitg requested changes Mar 4, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@sasha-gitg sasha-gitg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a unit test that tests this block. Preferably, also an integration test that executes a custom job with the SDK.

sasha-gitg
sasha-gitg requested changes Mar 4, 2022
View reviewed changes
Comment thread google/cloud/aiplatform/initializer.py Outdated
Comment thread google/cloud/aiplatform/initializer.py Outdated
Comment thread google/cloud/aiplatform/initializer.py Outdated
Comment thread google/cloud/aiplatform/initializer.py Outdated
@morgandu morgandu mentioned this pull request Mar 9, 2022
Merged
4 tasks
@morgandu
Copy link
Copy Markdown
Contributor

morgandu commented Mar 9, 2022

Hi @Ark-kun , PTAL of the #1068 for extracting project ID given a project number.

@Ark-kun Ark-kun force-pushed the fix--Fixed-getitng-project-ID-when-running-on-Vertex-AI branch from f61d533 to 46f6324 Compare March 11, 2022 02:45
@Ark-kun
Copy link
Copy Markdown
Contributor Author

Ark-kun commented Mar 11, 2022

I've switched to the new utility function.

@Ark-kun Ark-kun requested a review from sasha-gitg March 15, 2022 04:54
@Ark-kun Ark-kun force-pushed the fix--Fixed-getitng-project-ID-when-running-on-Vertex-AI branch from 91373a0 to 692ae9e Compare March 17, 2022 01:19
@Ark-kun Ark-kun force-pushed the fix--Fixed-getitng-project-ID-when-running-on-Vertex-AI branch from 235a6d1 to 1f79bce Compare March 17, 2022 03:04
@Ark-kun
Copy link
Copy Markdown
Contributor Author

Ark-kun commented Mar 17, 2022

Please add a unit test that tests this block. Preferably, also an integration test that executes a custom job with the SDK.

I've added both a unit test and an integration test that executes a custom job with the SDK.

@Ark-kun Ark-kun force-pushed the fix--Fixed-getitng-project-ID-when-running-on-Vertex-AI branch from 620043b to 4f20ae0 Compare March 17, 2022 21:42
@kweinmeister kweinmeister requested review from morgandu and sasha-gitg and removed request for sasha-gitg March 19, 2022 16:59
sasha-gitg
sasha-gitg approved these changes Mar 21, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@sasha-gitg sasha-gitg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Alexey!

@sasha-gitg sasha-gitg merged commit 876cb33 into googleapis:main Mar 21, 2022
@release-please release-please Bot mentioned this pull request Mar 21, 2022
Merged
Ark-kun added a commit to Ark-kun/pipeline_components that referenced this pull request Aug 1, 2022
@Ark-kun
Google Cloud - Vertex_AI - Models - Upload_XGBoost_model component - …
8016f51 
…Simplified the component after my Vertex SDK fixes were merged

Some of my Vertex SDK fixes:
googleapis/python-aiplatform#779
googleapis/python-aiplatform#882
googleapis/python-aiplatform#943
googleapis/python-aiplatform#997
Ark-kun added a commit to Ark-kun/pipeline_components that referenced this pull request Aug 1, 2022
@Ark-kun
Google Cloud - Vertex_AI - Models - Upload_Tensorflow_model component…
6af42e1 
… - Simplified the component after my Vertex SDK fixes were merged

Some of my Vertex SDK fixes:
googleapis/python-aiplatform#779
googleapis/python-aiplatform#882
googleapis/python-aiplatform#943
googleapis/python-aiplatform#997
Ark-kun added a commit to Ark-kun/pipeline_components that referenced this pull request Aug 1, 2022
@Ark-kun
Google Cloud - Vertex_AI - Models - Upload_Scikit-learn_pickle_model …
070f286 
…- Simplified the component after my Vertex SDK fixes were merged

Some of my Vertex SDK fixes:
googleapis/python-aiplatform#779
googleapis/python-aiplatform#882
googleapis/python-aiplatform#943
googleapis/python-aiplatform#997
Ark-kun added a commit to Ark-kun/pipeline_components that referenced this pull request Aug 1, 2022
@Ark-kun
Google Cloud - Vertex_AI - Models - Upload_XGBoost_model component - …
e1e8b7c 
…Simplified the component after my Vertex SDK fixes were merged

Some of my Vertex SDK fixes:
googleapis/python-aiplatform#779
googleapis/python-aiplatform#882
googleapis/python-aiplatform#943
googleapis/python-aiplatform#997
Ark-kun added a commit to Ark-kun/pipeline_components that referenced this pull request Aug 1, 2022
@Ark-kun
Google Cloud - Vertex_AI - Models - Upload_Tensorflow_model component…
4677245 
… - Simplified the component after my Vertex SDK fixes were merged

Some of my Vertex SDK fixes:
googleapis/python-aiplatform#779
googleapis/python-aiplatform#882
googleapis/python-aiplatform#943
googleapis/python-aiplatform#997
Ark-kun added a commit to Ark-kun/pipeline_components that referenced this pull request Aug 1, 2022
@Ark-kun
Google Cloud - Vertex_AI - Models - Upload_Scikit-learn_pickle_model …
32a849a 
…- Simplified the component after my Vertex SDK fixes were merged

Some of my Vertex SDK fixes:
googleapis/python-aiplatform#779
googleapis/python-aiplatform#882
googleapis/python-aiplatform#943
googleapis/python-aiplatform#997
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sasha-gitg sasha-gitg sasha-gitg approved these changes

@morgandu morgandu Awaiting requested review from morgandu

+1 more reviewer

@vinnysenthil vinnysenthil vinnysenthil approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

The SDK does not pick up the correct user project when running on Vertex Pipelines or CustomJobs

5 participants

@Ark-kun @morgandu @vinnysenthil @sasha-gitg @kweinmeister