The list of jobsets is very high on hydra.nixos.org and the compare to
dropdown listing goes over multiple full pages in the busy projects.
If we ignore jobsets that we disable this interface becomes more usable
again.
- Add HMAC-SHA256 signature verification for webhooks
- Support multiple secrets for rotation
- Add security logging for authentication events
- Maintain backward compatibility (auth optional during migration)
- Add comprehensive test coverage
Without authentication, anyone could trigger job evaluations by sending
POST requests to webhook endpoints. This could lead to resource exhaustion
through repeated requests or manipulation of build scheduling. While not
a data breach risk, it allows unauthorized control over CI/CD operations.
This prevents a forever-hanging build (don't know why) when < or > are
in the path of hydra-build-products. This is not to prevent any XSS (see
next commits), just to prevent the DOS (if you can even call it that).
- Replace deprecated exec_params/exec_params0 calls with exec()
- Wrap all parameterized queries with pqxx::params{}
- Add .no_rows()/.one_row() to exec calls that don't return results
- Remove bottom margin
- Properly format memory in human format
- Calculate free memory
- Format the load with 2 digits after comma
- Lpad pressure percentages
- Use a macro to render pressure
- Score -> Scheduling Score
- More spacing in the load
- Add IRQ pressure
This is guarded behind a setting and will overwrite everything that was
learned from the machines file. Also drops `sshKeys` since that wasn't
used anyway.
- Add localStore into the stash because it's used in templates
- Hide the Channels button for non-local stores because the link 404s
anyway
- Fix a style issue when having popovers in dark mode
Instead of just going for "whatever is the oldest build we know of",
use the following first:
- Is the step more constrained? If so, schedule it first to avoid
filling up "more desirable" build slots with less constrained builds.
- Does the step have more dependents? If so, schedule it first to try
and maximize open parallelism and breadth of scheduling options.
(cherry picked from commit b8d03adaf4)
New jobs have their "new" status take precedence over them being
"failed" or "queued", which means actions that can act on "failed" or
"queued" jobs weren't shown to the user when they could only act on
"new" jobs.
(cherry picked from commit 9a4a5dd624)
Quickfix for something that annoyed me once too often.
Specifically, I'm talking about `/eval/1#tabs-errors`.
To not fetch long errors on each request, this is only done on-demand.
I.e., when the tab is opened, an iframe is requested with the errors.
This iframe uses a template for both the jobset view and the jobset-eval
view. It is differentiated by checking if `jobset` or `eval` is defined.
However, the jobset-eval view also has a `jobset` variable in its stash
which means that in both cases the `if` path was used. Since
`jobset.fetcherrormsg` isn't defined in the eval case though, you always
got an empty error.
The band-aid fix is relatively simple: swap if and else: the `eval`
variable is not defined in the stash of the jobset view, so now this is
a useful condition to decide which view we're in.
(cherry picked from commit 70c3d75f73)
This is implement in an extremely hacky way due to poor DBIx feature
support. Ideally, what we'd need is a way to tell DBIx to ignore the
errormsg column unless explicitly requested, and to automatically add a
computed 'errormsg IS NULL' column in others. Since it does not support
that, this commit instead hacks some support via method overrides while
taking care to not break anything obvious.
This allows for better builder usage when the queue runner is busy. To
avoid running into uncontrollable imbalances between builder/queue
runner, we only release the machine reservation after the local
throttler has found a slot to start copying the outputs for that build.
As opposed to asserting uniqueness to understand resource utilization,
we just switch to using `std::unique_ptr`.
We don't rely on sequential / monotonic build IDs processing anymore, so
randomizing actually has the advantage of mixing builds for different
systems together, to avoid only one chunk of builds for a single system
getting processed while builders for other systems are starved.
Each output for a given step being ingested is looked up in parallel,
which should basically multiply the speed of builds ingestion by the
average number of outputs per derivation.
Running the query with/without it shows that it makes no difference to
postgres, since there's an index on finished=0 already. This allows a
few simplifications, but also paves the way towards running multiple
parallel monitor threads in the future.
By looking at the ratio of running vs. waiting for the dispatcher and
the queue monitor, we should get better visibility into what hydra is
currently bottlenecked on.
There are other side effects we can try to measure to get to the same
result, but having a simple way doesn't cost us much.
My current theory is that running more parallel xz than available CPU
cores is reducing our overall throughput by requiring more scheduling
overhead and more cache thrashing.
