ahuston-0/hydra
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
4,093 Commits 3 Branches 0 Tags
Commit Graph

231 Commits

Author SHA1 Message Date
John Ericson
9ba4417940 Prepare for CA derivation support with lower impact changes 
This is just C++ changes without any Perl / Frontend / SQL Schema
changes.

The idea is that it should be possible to redeploy Hydra with these
chnages with (a) no schema migration and also (b) no regressions. We
should be able to much more safely deploy these to a staging server and
then production `hydra.nixos.org`.

Extracted from #875

Co-Authored-By: Théophane Hufschmitt <theophane.hufschmitt@tweag.io>
Co-Authored-By: Alexander Sosedkin <monk@unboiled.info>
Co-Authored-By: Andrea Ciceri <andrea.ciceri@autistici.org>
Co-Authored-By: Charlotte 🦝 Delenk Mlotte@chir.rs>
Co-Authored-By: Sandro Jäckel <sandro.jaeckel@gmail.com>
 2023-12-04 16:14:47 -05:00
John Ericson
c922e73c11 Update to Nix 2.19 
Flake lock file updates:

• Updated input 'nix':
    'github:NixOS/nix/f5f4de6a550327b4b1a06123c2e450f1b92c73b6' (2023-10-02)
  → 'github:NixOS/nix/50f8f1c8bc019a4c0fd098b9ac674b94cfc6af0d' (2023-11-27)
 2023-11-30 15:26:46 -05:00
Janne Heß
d135b123cd
Merge pull request #1292 from Ma27/fix-queue-runner-stats 
hydra-queue-runner: fix stats
 2023-07-17 09:56:05 +02:00
Maximilian Bosch
5c35d1be20
hydra-queue-runner: fix stats 2023-06-25 17:28:15 +02:00
Eelco Dolstra
9f69bb5c2c Fix compilation against Nix 2.16 2023-06-23 15:06:55 +02:00
Linus Heckemann
5b35e13898
hydra-queue-runner: use initializer lists for constructing JSON 
And also fix the parts that were broken
 2023-02-04 20:08:27 +01:00
Linus Heckemann
96e36201eb
hydra-queue-runner: adapt to nlohmann::json 2023-02-04 20:08:09 +01:00
Graham Christensen
e1965250b5
Merge pull request #1173 from DeterminateSystems/queue-runner-exporter 
hydra-queue-runner metrics
 2022-04-07 12:27:33 -04:00
Cole Helbling
f8dc48f171
hydra-queue-runner: fixup: remove extraneous newline 2022-04-06 17:53:11 -07:00
Graham Christensen
59ac96a99c Track the number of steps created 2022-04-06 20:23:02 -04:00
Graham Christensen
1c12c5882f hydra queue runner: instrument the process of loading new builds with prom 2022-04-06 20:18:29 -04:00
Graham Christensen
5de08d412e queue metrics: refactor the metrics into a struct 2022-04-06 20:00:30 -04:00
Graham Christensen
46f52b4c4e bring back the working version Cole made 2022-04-06 15:49:38 -04:00
Cole Helbling
5bff730f2c WIP: I love it when they delete the assignment operator :) 2022-04-06 11:41:40 -07:00
Cole Helbling
edf3c348f2 hydra-queue-runner: make entire address configurable 2022-04-06 10:59:45 -07:00
Cole Helbling
33bc60b83c hydra-queue-runner: move exporter back to State::run 
It's (arguably) better than risking pinning the thread at 100% due to
the busy `while` loop.
 2022-04-06 10:49:14 -07:00
Cole Helbling
8c5636fe18
hydra-queue-runner: use port 9198 by default 
Co-authored-by: Graham Christensen <graham@grahamc.com>
 2022-04-02 17:32:14 -07:00
ajs124
089da272c7 fix build against nix 2.7.0 
fix build after such commits as df552ff53e68dff8ca360adbdbea214ece1d08ee
and e862833ec662c1bffbe31b9a229147de391e801a
 2022-03-29 15:38:24 -04:00
ajs124
c64c5f0a7e hydra-queue-runner: rename build-result.hh to hydra-build-result.hh 2022-03-29 15:34:29 -04:00
Cole Helbling
4789eba92c hydra-queue-runer: split metrics functionality into its own function 2022-03-29 10:55:28 -07:00
Cole Helbling
928b3b8268 hydra-queue-runner: fix priority of flag over config file 2022-03-29 10:42:07 -07:00
Cole Helbling
5ddb9a98ca fixup! hydra-queue-runner: log message before and after exporter is started 2022-03-29 08:47:41 -07:00
Cole Helbling
905a7a7beb hydra-queue-runner: read metrics port from queue_runner_metrics_port config 2022-03-29 08:46:43 -07:00
Cole Helbling
9cdc5aceed hydra-queue-runner: log message before and after exporter is started 
This way, if something goes wrong between the two, it's easier to narrow
down where the issue lies.
 2022-03-29 08:41:19 -07:00
Cole Helbling
8503a7917b fixup! hydra-queue-runner: make registry member of State, configurable metrics port 2022-03-22 13:38:13 -07:00
Cole Helbling
c0f826b92d hydra-queue-runner: get the listening port from the exposer itself 
Otherwise, when the port is randomly chosen (e.g. by specifying no port,
or a port of 0), it will just show that the port is 0 and not the port
that is actually serving the metrics.
 2022-03-14 08:41:45 -07:00
Cole Helbling
52a29d43e6 hydra-queue-runner: make registry member of State, configurable metrics port 
Thanks to the updated prometheus-cpp library, specifying a port of 0
will cause it to pick a random (available) port -- ideal for tests.
 2022-03-11 11:58:10 -08:00
Cole Helbling
3bf31bd6a6 hydra-queue-runner: add simple "up" exporter 
There are probably better ways to achieve this (and will likely need to
be refactored a bit to support further metrics).
 2022-03-10 12:36:58 -08:00
Graham Christensen
4acaf9c8b0 hydra-queue-runner: don't dispatch until the machines parser has completed one run 
Periodically, I have seen tests fail because of out of order queue runner behavior:

    checking the queue for builds > 0...
    loading build 1 (tests:basic:empty_dir)
    aborting unsupported build step '...-empty-dir.drv' (type 'x86_64-linux')
    marking build 1 as failed
    adding new machine ‘localhost’

This patch should prevent the dispatcher from running before any machines are
made available.
 2022-02-10 10:54:30 -05:00
regnat
abff212d06 Use system-features from the Nix conf in the default machine file 
Fix #936
 2021-04-28 11:43:04 +02:00
Eelco Dolstra
a7d8ee98da Fix build 2021-02-22 15:10:24 +01:00
Maximilian Bosch
9cc76f6d69
Fix build with latest Nix 
Recently a few internal APIs have changed[1]. The `outputPaths` function
has been removed and a lot of data structures are modeled with
`std::optional` which broke compilation.

This patch updates the code in `hydra-queue-runner` accordingly to make
sure that Hydra compiles again.

[1] https://github.com/NixOS/nix/pull/3883
 2020-09-26 23:37:39 +02:00
Eelco Dolstra
f8e15bc311 Revive putBytes 2020-08-04 18:25:21 +02:00
Eelco Dolstra
4b5813051b
unsigned long long -> uint64_t 2020-08-04 11:38:22 +02:00
Eelco Dolstra
7d3ba616a9
Fix build 2020-08-04 11:33:29 +02:00
Eelco Dolstra
a0e24f446b
Remove unused getMemSize() function 2020-07-27 20:40:57 +02:00
Eelco Dolstra
d4e4be4fd1
Remove SHA-1 hash from BuildProducts 
SHA-1 is deprecated and it will be expensive to compute with the
streaming NAR handler.
 2020-07-27 18:24:10 +02:00
Eelco Dolstra
cbcf6359b4
Remove TokenServer in preparation of making NAR copying O(1) memory 2020-07-27 14:57:22 +02:00
Eelco Dolstra
7985757a1d
Fix build 2020-07-08 12:50:02 +02:00
Eelco Dolstra
bb32aafa4a
Fix build 2020-06-23 13:56:44 +02:00
Graham Christensen
7b705758ec
Merge pull request #732 from Ma27/fix-build 
Fix build against latest Nix
 2020-04-09 09:02:45 -04:00
Bas van Dijk
6e358189ad Separate the build IDs in the build_finished payload with tabs 
hydra-notify splits the payload on tabs so we shouldn't separate the
IDs with spaces.
 2020-04-08 12:05:25 +02:00
Maximilian Bosch
2f9d422172
Fix build against latest Nix 2020-04-07 13:55:38 +02:00
Kevin Quick
a055796ef5
Merge branch 'master' into libpqxx_undeprecate 2020-04-01 11:54:41 -07:00
Eelco Dolstra
6c0bb909df
Merge pull request #704 from basvandijk/sleep-after-catch-exception 
hydra-queue-runner: sleep 5s after handling an exception
 2020-04-01 13:04:31 +02:00
Eelco Dolstra
9727892b61 Don't spam the journal with hydra-queue-runner status dumps 
(cherry picked from commit 15ae932488512ba235ed2f6f841cc5eb56ba9314)
 2020-03-31 22:19:07 +02:00
Eelco Dolstra
ccd046ca3d Keep track of the number of unsupported steps 
(cherry picked from commit 45ffe578b695f9de101b30d44d46f12aa0654f10)
 2020-03-31 22:19:03 +02:00
Eelco Dolstra
4417f9f260 Abort unsupported build steps 
If we don't see machine that supports a build step for
'max_unsupported_time' seconds, the step is aborted. The default is 0,
which is appropriate for Hydra installations that don't provision
missing machines dynamically.

(cherry picked from commit f5cdbfe21d930db43d3812c7d8e87746d6378ef9)
 2020-03-31 22:19:01 +02:00
Eelco Dolstra
e4f5156c41
Build against nix-master 
(cherry picked from commit e7f2139e251cb73195eea6fb84e2a6167b4db968)
 2020-02-20 10:24:04 +01:00
Bas van Dijk
08bfff9d13 hydra-queue-runner: sleep 5s after handling an exception 
instead of immediately calling `readMachinesFiles` again which could
immediately throw another exception again.
 2020-01-14 13:34:35 +01:00