chore(gooddata-eval): use neutral identifiers in fixtures, drop a hardcoded path - #1827
Conversation
…dcoded path Test fixtures carried metric and label names copied out of one particular workspace, and two comments named an external repository rather than saying what they meant. Neither tells a reader of this package anything: the scoring tests exercise attribute-filter ordering and the names are arbitrary, so they now use generic ones. verify_guardrail_refusal_criteria.py loaded .env from an absolute path under one developer's home directory, which made the script runnable on exactly one machine. It now reads GD_EVAL_ENV_FILE, defaulting to .env in the working directory. One test needed care rather than a rename: the two sides of test_the_same_elements_on_a_different_label_still_differ deliberately name different labels, and renaming both to the same thing quietly turned it into a test of nothing. They stay distinct. No behaviour change. 1111 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (4)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📝 WalkthroughWalkthroughThe guardrail script now loads dotenv from a configurable path. Evaluation comments and test fixtures also use updated descriptions, labels, and workspace values. ChangesGuardrail script environment
Evaluation descriptions and fixtures
Priority: ⬇️ Low Estimated code review effort: 2 (Simple) | ~8 minutes Change: Other Suggested reviewers: Merge Risk: ⚪ Minimal · up to The changes make the guardrail script’s environment-file path configurable and refresh evaluation examples without an apparent behavior-breaking mismatch. The PR is mergeable subject to normal checks. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
A rabbit checks the dotenv trail, Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1827 +/- ##
=======================================
Coverage 82.88% 82.88%
=======================================
Files 328 328
Lines 21235 21235
=======================================
Hits 17600 17600
Misses 3635 3635 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…l-open-prs Master has moved on nine gooddata-eval commits, five of them PRs this branch was carrying from their own branches (#1793, #1799, #1806, #1807, #1827) and four the dashboard-skill stack. Resolutions worth naming: - Per-run failure capture vs the new timeline detail. Master replaced `build_latency_breakdown` with `timeline_detail` (latency breakdown plus the tool calls) at the four `detail` sites this branch had refactored into a shared `_run_detail`. Kept the refactor and added `timeline_detail` for the winning run on top, so failing runs keep the compact breakdown and the best run still reports the calls themselves. - `dashboard_skill` arrived from master without per-run failure records, which the structural guard test requires of every multi-run kind. Wired it the same way as its siblings. - Sort normalisation and the scrubbed fixture comments were taken from master wholesale; this branch predates both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Housekeeping in
gooddata-eval, no behaviour change.What
Fixtures use generic names. The scoring tests exercise attribute-filter ordering and ranking-filter equivalence; the metric and label names in them were copied out of one particular workspace and carry no meaning for a reader of this package. They are now ordinary names (
label/region_name,metric/conversion_rate,demo_workspace).verify_guardrail_refusal_criteria.pyis runnable again. It loaded.envfrom an absolute path under one developer's home directory, so nobody else could run it. It now readsGD_EVAL_ENV_FILE, defaulting to.envin the working directory.Two comments say what they mean instead of pointing at a repository the reader cannot open.
One thing worth a second look
test_the_same_elements_on_a_different_label_still_differnames two different labels on purpose — that difference is the whole assertion. Renaming both to the same value turns it green while testing nothing, which is exactly what happened on the first pass here. They stay distinct, and the test still fails if the comparison stops caring about the label.Verification
1111 tests pass,
ruff checkandruff format --checkclean.🤖 Generated with Claude Code
Summary by CodeRabbit
.env, rather than relying on a fixed local path. This makes the tool easier to run across different development environments.