Repository navigation
Conversation
Duel (https://github.com/sferik/duel-ruby) runs the benchmarks in duel/ on the branch a pull request targets and on the pull request, side by side, and fails the Perf workflow when the pull request is slower or allocates more. The 39 benchmarks send requests through an API's Rack interface in memory: routing among 200 routes, nested namespaces, requirements and wildcards, 404 and 405, params coercion and validation (query string, form, JSON body, nested arrays, a custom validator, a 400), declared, JSON formatting, present, content negotiation, error! and rescue_from, callbacks and helpers, path, header and param versioning, mounting, and compiling an API. Comparing master with itself passed three times in a row. An extra dup in Router#process_route failed 17 benchmarks on allocations, and reading the attribute three times in CoerceValidator#validate_param! failed 11 on instructions and CPU time.
dblock
reviewed
Nov 11, 2026
dblock
left a comment
dblock
left a comment
Member
There was a problem hiding this comment.
Nice to see you here again @sferik! Interesting idea. I would want to hear an enthusiastic yes from @ericproulx who has been doing performance improvements left and right in grpe to merge it.
Nit, I'd refactor GHA workflows to be reusable so that they don't have to be copy-pasted all over, like we do here.
Thinking out loud, expanding CI to also check perf is something I'm 50/50 on. CI needs to get faster, not slower, to enable AI agents and benchmarks are as expensive as it gets.
| begin | ||
| require('duel/rake_task') | ||
| Duel::RakeTask.new | ||
| rescue LoadError |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds Duel, a gem I wrote to catch performance regressions in CI. On every pull request it runs the benchmarks in
duel/againstmasterand the branch side by side, in alternating worker processes, and fails the build if the branch is slower or allocates more. When something regresses, it profiles both versions with StackProf and says which method the extra time went to.There are 39 benchmarks, and every one is a request sent through an API's Rack interface in memory, the way a server would hand it over, with no server or sockets involved. I went with whole requests rather than calling internal classes because that's what Grape users actually pay for, and because the router, the middleware stack and the endpoint have been tuned so carefully that most of the interesting work happens between them. The env for each request is built outside the measured time, since Grape and Rack write into the env and read
rack.input, so reusing one would measure cached work. Each benchmark also checks its status and part of its body once before measuring, so none of them can quietly end up timing a 404.grape-entity isn't in the development bundle, so the
presentbenchmarks use a small representer class thatpresentcalls the same way it calls an Entity. Compiling a 200 route API took about 40 ms, which is longer than a benchmark should run, so the compile benchmark uses 20 routes and callschange!before eachcompile!.A full comparison takes about 10 to 12 seconds here when nothing regressed.
The Duel job only reads. The comment on a pull request comes from a second workflow,
perf-comment.yml, which runs none of the pull request's code and shows a report from a fork as plain text. If a change is slower on purpose, aduel: acceptlabel lets it through.