Commit Graph

468 Commits

Author SHA1 Message Date
45ffe578b6 Keep track of the number of unsupported steps 2020-03-26 15:27:13 +01:00
d0688b93e1 Merge remote-tracking branch 'origin/master' into flake 2020-03-26 11:44:11 +01:00
12cc46cdb3 fixup: hydra-init: correct reference to hydra-backill-ids 2020-03-24 11:22:14 -04:00
4b5bb4e760 Merge remote-tracking branch 'origin/master' into flake 2020-03-04 15:28:23 +01:00
994430b94b treewide: allow nix command 2020-03-03 22:52:20 -05:00
2a50daa377 Update aggregate handling
(cherry picked from commit cf961ac893)
2020-02-20 10:13:39 +01:00
15187b059b Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.

(cherry picked from commit 23c9ca3e94)
2020-02-20 09:58:12 +01:00
6f1d68bda4 Revert "hydra-eval-jobs -> nix eval-hydra-jobs"
This reverts commit 345512a6d0.
2020-02-19 20:36:52 +01:00
cf961ac893 Update aggregate handling 2020-02-17 16:33:25 +01:00
345512a6d0 hydra-eval-jobs -> nix eval-hydra-jobs 2020-02-15 15:59:34 +01:00
23c9ca3e94 Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.
2020-02-15 15:59:34 +01:00
881b7449fd Merge remote-tracking branch 'origin/master' into flake 2020-02-11 14:23:16 +01:00
f0f41eaaff LatestSucceededForJob{,set}: Filter with jobset_id 2020-02-11 07:06:20 -05:00
100e09a5b3 Merge remote-tracking branch 'origin/master' into flake
Also update flake.lock
2020-02-10 17:58:10 +01:00
c4cc72f944 hydra-init: Warn about the schema version migration 2020-02-10 11:43:03 -05:00
f69260118b hydra-backfill-ids: create to add jobset_id values to Builds and Jobs
Vacuum every 10 iterations, update 10k at a time.
2020-02-10 11:43:03 -05:00
f3a561aecd Builds: populate Builds.jobset_id in hydra-eval-jobset 2020-02-10 11:43:02 -05:00
624f1d8d2d Jobs: populate Jobs.jobset_id field when writing from hydra-eval-jobset 2020-02-10 11:43:02 -05:00
511c2db8aa Merge remote-tracking branch 'origin/master' into flake 2019-12-28 20:22:54 -05:00
06abfd6b2f hydra-send-stats: Cleanup removed metrics
In 2946899504 these metrics got removed
due to refactoring of how notifications work.
2019-11-13 11:42:58 +01:00
55b0afa08f Merge remote-tracking branch 'origin/master' into flake 2019-11-07 18:42:15 +01:00
840e99f859 hydra-eval-jobset: $firstOutput is not used so can be removed 2019-11-05 13:58:40 +01:00
a816730d8b uri -> url 2019-10-23 20:31:27 +02:00
2de52d8538 Merge remote-tracking branch 'origin/master' into flake 2019-08-15 13:56:00 +02:00
92d8d6baa5 Avoid fetching Projects/Jobsets just to get the name column
In particular, doing a 'select * from Jobsets where ...' must be
avoided, because the 'errormsg' column can be very big.
2019-08-13 18:18:25 +02:00
f49a089fc0 hydra-notify: Don't do an unnecessary fetch of Jobsets 2019-08-13 18:18:24 +02:00
72c36373bb hydra-notify: Fix processing notifications 2019-08-13 18:18:24 +02:00
976d88d675 Send notifications when evaluations start/finish/fail
* 'eval_started' has the format '<tmpId>\t<project>\t<jobset>'.

* 'eval_failed' has the format '<tmpId>'. (The cause of the error can
  be found in the database.)

* 'eval_added' has the format '<tmpId>:<evalId>'.
2019-08-13 18:18:24 +02:00
7114d2aceb Separate payload elements using \t 2019-08-13 18:18:24 +02:00
2946899504 Turn hydra-notify into a daemon
It now receives notifications about started/finished builds/steps via
PostgreSQL. This gets rid of the (substantial) overhead of starting
hydra-notify for every event. It also allows other programs (even on
other machines) to listen to Hydra notifications.
2019-08-13 18:18:21 +02:00
ed00f0b25e Cache flake-based jobset evaluations 2019-05-11 00:41:13 +02:00
842ef9af24 hydra-eval-jobset: Support flakes 2019-05-11 00:11:38 +02:00
04ff9e217b Add hydra-dev-server which uses the classic Catalyst server
This, in turns allows

 - Using --restart for reloading the perl code
 - Printing traces on error
2019-01-22 20:34:21 -05:00
8bffbb7928 evalSettings deprecated, submodule fix 2018-10-03 00:45:42 -04:00
8e17a413f5 hydra-notify step-finished: Don't barf if the step has no log file 2018-08-01 17:17:46 +02:00
1898e31ed8 hydra-eval-jobset: Respect email_notification 2018-08-01 16:56:53 +02:00
298e99449b add metrics for hydra notifications. 2018-06-12 13:00:39 +01:00
99e6c19672 add some more statistics about runnable/running builds per arch/system
features.
2018-06-12 13:00:00 +01:00
8913c682cf Doh 2017-12-25 15:02:17 +01:00
0d79f16fd5 git-export -> source 2017-12-25 14:31:30 +01:00
2660c3cc54 Remove redundant check 2017-12-05 16:00:18 +01:00
364e21919a hydra-eval-{jobs,jobset}: Pass file name as <...> 2017-11-28 16:51:00 +01:00
5986aae704 Add option eval_substituter
When using the "build" or "sysbuild" jobset input types in conjunction
with a binary cache store, the evaluator needs to be able to fetch
store paths from the binary cache. Typical usage:

  store_uri = s3://nix-test-cache?secret-key=...
  eval_substituter = s3://nix-test-cache

Also, the public key of the binary cache must be added to
binary-cache-public-keys in nix.conf, otherwise the local nix-daemon
won't allow the store paths to be copied over.
2017-11-20 16:12:20 +01:00
2cdc84f34f Fix build against Nix master
Also, remove support in hydra-eval-jobs for multiple jobset input
alternatives. The web interface hasn't supported this in a long
time. Thus we can use the regular "--arg" handler.
2017-10-26 13:10:14 +02:00
bc60fccf78 Remove remaining references to store_mode etc. 2017-10-18 12:23:07 +02:00
eb7631fb9d hydra-create-user now has --password-hash option (#504)
When creating a Hydra user with the `hydra-create-user` command, you can now
provide a SHA1 password hash with the `--password-hash` flag. This is useful for
the upcoming work on Fully Declarative Hydra, since the end user should not have
to specify plaintext passwords in their `configuration.nix` file.
2017-10-08 11:55:51 +01:00
dc5e0b120a Fix a race that can cause hydra-queue-runner to ignore newly added builds
As @dtzWill discovered, with the concurrent hydra-evaluator, there can
be multiple active transactions adding builds to the database. As a
result, builds can become visible in a non-monotonically increasing
order, breaking the queue monitor's assumption that build IDs only go
up.

The fix is to have hydra-eval-jobset provide the lowest build ID it
just added in the builds_added notification, and have the queue
monitor check from there.

Fixes #496.
2017-07-21 14:34:48 +02:00
803833aba7 Disable the build-queued hook
This can take an excessive amount of time. For example, on
hydra.nixos.org, a call to hydra-notify takes 0.7s even if there are
no plugins. So for an eval with ~45K new builds, the calls to
hydra-notify add up to about 9 hours.

The proper fix would be to pass a list of build IDs, or an eval ID.
2017-06-22 15:09:04 +02:00
dfb3a52f67 Fix build-queued plugin hook 2017-05-24 11:12:48 -04:00
c40077ece1 Fix tests 2017-05-24 10:00:38 -04:00