Skip to content

Testdef format 2.0 - #653

Open
mwasilew wants to merge 5 commits into
Linaro:masterfrom
mwasilew:testdef-format-2.0
Open

mwasilew wants to merge 5 commits into
Linaro:masterfrom
mwasilew:testdef-format-2.0

Conversation

@mwasilew

Copy link
Copy Markdown
Contributor

Add Test Definition 2.0 format YAML files. The new format changes the list of mandatory metadata fields and updates the test definition validation.

roxell and others added 5 commits November 5, 2026 14:11
LAVA fails the job if a dependency_path entry matches nothing or points
outside the repository. Check it in validate.py too, so CI finds it first.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
The tests need sh-test-lib and send-to-lava.sh. ltp and kselftest also
need skipgen. With dependency_path LAVA copies 2M to 5M to the DUT
instead of the whole repository.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Add a short example, and say when skipgen and .git must be listed.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Add "Lava-Test Test Definition 2.0". It no longer requires the 'os' and
'devices' metadata keys: where a test runs is decided by the test plan
or the LAVA job, not by the test definition. 'os', 'scope' and 'devices'
are optional and still accepted when present.

2.0 requires 'dependency_path', so LAVA copies only the files the test
needs to the DUT instead of the whole repository. The entries are
checked by the existing dependency_path validation. 2.0 files must
contain 'name', 'format', 'description', 'maintainer' and
'dependency_path'.

1.0 test definitions are still accepted with unchanged rules.

validate.py now keeps the mandatory keys for each format in a single
table and checks the format first, so it decides which rules apply.
Unknown formats report the list of supported values.

generate_test_docs.py omits empty OS/Scope/Devices sections, and
testplan2html.py no longer fails when 'os' or 'scope' are missing.

Assisted-By: Claude Opus 5.5
Signed-off-by: Milosz Wasilewski <milosz.wasilewski@oss.qualcomm.com>
Add "Manual Test Definition 2.0". Like "Lava-Test Test Definition 2.0",
'name', 'format', 'description' and 'maintainer' are mandatory, while
'os', 'scope' and 'devices' are optional.

'dependency_path' stays optional for manual tests: they are not run by
LAVA copying the repository to the DUT, so the key has no effect there.

"Manual Test Definition 1.0" files are still accepted with unchanged
rules. test-runner.py detects manual tests by the "Manual Test
Definition" prefix, so it handles 2.0 files without changes.

Assisted-By: Claude Opus 5.5
Signed-off-by: Milosz Wasilewski <milosz.wasilewski@oss.qualcomm.com>
@roxell

roxell commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator

Looks interesting @mwasilew thank you. As we discussed offline, lets bring this up in the lava meeting on Monday.
I have an update to pr-651 as well to add dependency_path to more tests and maybe also drop the os/scope/devices but maybe thatt should go in as its own PR...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants