ci(wip): upload gotestsum JSON, one-acc-test smoke#5068
Draft
ci(wip): upload gotestsum JSON, one-acc-test smoke#5068
Conversation
To figure out where wall-clock time actually goes in Windows test runs, upload gotestsum's `test-output.json` as a CI artifact per matrix entry so we can run `gotestsum tool slowest` (and ad-hoc queries) on the full per-test timing set afterwards. Temporarily restrict the acceptance suite to a single test (`TestAccept/bundle/validate/required`) so the artifact plumbing can be validated end-to-end in ~5 minutes instead of paying the full 20-32 min Windows run. Once the artifact is confirmed to land, the filter will be removed. Co-authored-by: Isaac
Smoke run confirmed the gotestsum JSON artifact pipeline works (linux-direct artifact: 26MB, 91k JSON rows). Now capturing the full-suite timing so we can analyze per-test wall-clock across all matrix entries. Co-authored-by: Isaac
test-unit and test-acc were both writing to `test-output.json` — the acc run overwrote the unit run, so the artifact only contained acceptance timings. Write unit output to `test-output-unit.json` instead (gotestsum --jsonfile overrides on CLI, last-wins) and upload both files. `make slowest` keeps reading `test-output.json` as before. Co-authored-by: Isaac
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Need profiling data to figure out where wall-clock time actually goes in Windows test runs. Previous investigations were guess-and-check; we want the full per-test gotestsum timing set as a CI artifact so we can run
gotestsum tool slowestand ad-hoc queries offline.What
test-output.jsonas a per-matrix-entry artifact in thetestjob.TestAccept/bundle/validate/required) so artifact plumbing is validated in ~5 min per Windows job. Once the artifact is confirmed, the filter will be dropped for a full profiling run.Plan after this run
test-output-windows-terraformartifact downloads cleanly.ACCEPTANCE_TEST_FILTERoverride and run again to get full 32-min profile.This pull request and its description were written by Isaac.