Skip to content

3482 Add schema verification for metadata#3865

Merged
Nic-Ma merged 34 commits intoProject-MONAI:devfrom
Nic-Ma:3482-add-schema-verify
Mar 14, 2022
Merged

3482 Add schema verification for metadata#3865
Nic-Ma merged 34 commits intoProject-MONAI:devfrom
Nic-Ma:3482-add-schema-verify

Conversation

@Nic-Ma
Copy link
Copy Markdown
Contributor

@Nic-Ma Nic-Ma commented Feb 28, 2022 •
edited
Loading

Task step 5 of #3482 .

Description

This PR added the utility to download schema and verify the metadata in a manifest.

Status

Ready

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

@Nic-Ma Nic-Ma mentioned this pull request Feb 28, 2022
Closed
17 tasks
@Nic-Ma
[DLMED] complete temp test schema
09aa7ef 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Feb 28, 2022

Steps that generated the schema:

  1. Generate a draft schema with online tool:
    https://www.liquid-technologies.com/online-json-to-schema-converter
  2. Modify unnecessary or incorrect items in the schema.
  3. Add descriptions to the main items.

Thanks.

@Nic-Ma
[DLMED] add cmd line API
67bd705 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] fix typo
c72088f 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] add more tests
9ed45df 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] add optional import
2cf9d3a 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] test with download link
d3801ee 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma Nic-Ma mentioned this pull request Feb 28, 2022
Merged
7 tasks
@Nic-Ma
[DLMED] enhance doc-string
b6cd1db 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Feb 28, 2022

It depends on PR #3832.

Thanks.

@Nic-Ma
[DLMED] fix mypy
c77888a 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 1, 2022

Hi @ericspod @wyli @rijobro ,

Due to the Blossom CI issue, we can't merge any PRs recently, this PR depends on #3832 (marked the dependency logic with FIXME in the code) but it's almost ready for review. Could you please help review this draft PR first?

Thanks in advance.

@Nic-Ma Nic-Ma requested review from ericspod, rijobro and wyli March 1, 2022 07:59
ericspod
ericspod reviewed Mar 1, 2022
View reviewed changes
Comment thread monai/apps/manifest/utils.py Outdated
ericspod
ericspod reviewed Mar 1, 2022
View reviewed changes
Comment thread monai/apps/manifest/utils.py Outdated
ericspod
ericspod reviewed Mar 1, 2022
View reviewed changes
Comment thread monai/apps/manifest/utils.py Outdated
ericspod
ericspod reviewed Mar 1, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@ericspod ericspod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it looks good to me, the DS_Store files need to be removed, and some pattern modified in .gitignore so that they don't show up again. We should discuss if adding the jsonschema dependency is a problem but I think it's fine as an optional.

@Nic-Ma Nic-Ma mentioned this pull request Mar 2, 2022
Merged
7 tasks
Nic-Ma added 3 commits March 7, 2022 12:27
@Nic-Ma
[DLMED] remove DS_Store
eccc3ab 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Merge branch 'dev' of https://github.com/Project-MONAI/MONAI into 348…
f37ee56 
…2-add-schema-verify

Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] update according to comments
140b8b6 
Signed-off-by: Nic Ma <nma@nvidia.com>
ericspod
ericspod reviewed Mar 9, 2022
View reviewed changes
Comment thread tests/test_bundle_verify_meta.py Outdated
@Nic-Ma Nic-Ma requested a review from wyli March 11, 2022 17:05
@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 11, 2022

/black

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 11, 2022

/build

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 12, 2022

/build

1 similar comment
@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 12, 2022

/build

@Nic-Ma Nic-Ma force-pushed the 3482-add-schema-verify branch from bd35493 to 9be9daa Compare March 13, 2022 02:13
@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 13, 2022

/black

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 13, 2022

/build

@Nic-Ma
[DLMED] skip windows
caf3421 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 13, 2022

/black

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 13, 2022

/build

@Nic-Ma
[DLMED] fix typo
0ce58d9 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 14, 2022

/black

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 14, 2022

/build

wyli
wyli approved these changes Mar 14, 2022
View reviewed changes
Copy link
Copy Markdown
Contributor

@wyli wyli left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, looks good, could you move the schema.json to the testing data repository. Cc @ericspod in case there are more comments

wyli
wyli reviewed Mar 14, 2022
View reviewed changes
Comment thread monai/bundle/scripts.py Outdated
Nic-Ma added 2 commits March 14, 2022 20:10
@Nic-Ma
[DLMED] optimize error log
e7e1d3e 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] update according to comments
7e4c506 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma Nic-Ma force-pushed the 3482-add-schema-verify branch from 0b182fa to 7e4c506 Compare March 14, 2022 13:21
@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 14, 2022

/black

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 14, 2022

/build

1 similar comment
@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 14, 2022

/build

@Nic-Ma
[DLMED] update accoding to comments
f80d9a8 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 14, 2022

/black

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Mar 14, 2022

/build

@Nic-Ma Nic-Ma enabled auto-merge (squash) March 14, 2022 15:30
@Nic-Ma Nic-Ma merged commit f15ff93 into Project-MONAI:dev Mar 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wyli wyli wyli approved these changes

@rijobro rijobro Awaiting requested review from rijobro

@ericspod ericspod Awaiting requested review from ericspod ericspod is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Nic-Ma @wyli @ericspod