417 Commits

Author SHA1 Message Date
Eelco Dolstra
d0688b93e1
Merge remote-tracking branch 'origin/master' into flake 2020-03-26 11:44:11 +01:00
Graham Christensen
12cc46cdb3
fixup: hydra-init: correct reference to hydra-backill-ids 2020-03-24 11:22:14 -04:00
Eelco Dolstra
4b5bb4e760
Merge remote-tracking branch 'origin/master' into flake 2020-03-04 15:28:23 +01:00
Graham Christensen
994430b94b
treewide: allow nix command 2020-03-03 22:52:20 -05:00
Eelco Dolstra
2a50daa377
Update aggregate handling
(cherry picked from commit cf961ac8933b76d9f0a3ac3eb49cc08879f5bcc9)
2020-02-20 10:13:39 +01:00
Eelco Dolstra
15187b059b
Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.

(cherry picked from commit 23c9ca3e94669087d463642baea0cf35a0b8d72f)
2020-02-20 09:58:12 +01:00
Eelco Dolstra
6f1d68bda4
Revert "hydra-eval-jobs -> nix eval-hydra-jobs"
This reverts commit 345512a6d0743eaac9d451833ecc1a71d12809c9.
2020-02-19 20:36:52 +01:00
Eelco Dolstra
cf961ac893
Update aggregate handling 2020-02-17 16:33:25 +01:00
Eelco Dolstra
345512a6d0
hydra-eval-jobs -> nix eval-hydra-jobs 2020-02-15 15:59:34 +01:00
Eelco Dolstra
23c9ca3e94
Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.
2020-02-15 15:59:34 +01:00
Eelco Dolstra
881b7449fd Merge remote-tracking branch 'origin/master' into flake 2020-02-11 14:23:16 +01:00
Graham Christensen
f0f41eaaff
LatestSucceededForJob{,set}: Filter with jobset_id 2020-02-11 07:06:20 -05:00
Eelco Dolstra
100e09a5b3
Merge remote-tracking branch 'origin/master' into flake
Also update flake.lock
2020-02-10 17:58:10 +01:00
Graham Christensen
c4cc72f944
hydra-init: Warn about the schema version migration 2020-02-10 11:43:03 -05:00
Graham Christensen
f69260118b
hydra-backfill-ids: create to add jobset_id values to Builds and Jobs
Vacuum every 10 iterations, update 10k at a time.
2020-02-10 11:43:03 -05:00
Graham Christensen
f3a561aecd
Builds: populate Builds.jobset_id in hydra-eval-jobset 2020-02-10 11:43:02 -05:00
Graham Christensen
624f1d8d2d
Jobs: populate Jobs.jobset_id field when writing from hydra-eval-jobset 2020-02-10 11:43:02 -05:00
Graham Christensen
511c2db8aa
Merge remote-tracking branch 'origin/master' into flake 2019-12-28 20:22:54 -05:00
Nikola Knezevic
06abfd6b2f hydra-send-stats: Cleanup removed metrics
In 29468995040ae21e0e1c14c1bdbb16ccb514caa8 these metrics got removed
due to refactoring of how notifications work.
2019-11-13 11:42:58 +01:00
Eelco Dolstra
55b0afa08f
Merge remote-tracking branch 'origin/master' into flake 2019-11-07 18:42:15 +01:00
Bas van Dijk
840e99f859 hydra-eval-jobset: $firstOutput is not used so can be removed 2019-11-05 13:58:40 +01:00
Eelco Dolstra
a816730d8b uri -> url 2019-10-23 20:31:27 +02:00
Eelco Dolstra
2de52d8538
Merge remote-tracking branch 'origin/master' into flake 2019-08-15 13:56:00 +02:00
Eelco Dolstra
92d8d6baa5
Avoid fetching Projects/Jobsets just to get the name column
In particular, doing a 'select * from Jobsets where ...' must be
avoided, because the 'errormsg' column can be very big.
2019-08-13 18:18:25 +02:00
Eelco Dolstra
f49a089fc0
hydra-notify: Don't do an unnecessary fetch of Jobsets 2019-08-13 18:18:24 +02:00
Eelco Dolstra
72c36373bb
hydra-notify: Fix processing notifications 2019-08-13 18:18:24 +02:00
Eelco Dolstra
976d88d675
Send notifications when evaluations start/finish/fail
* 'eval_started' has the format '<tmpId>\t<project>\t<jobset>'.

* 'eval_failed' has the format '<tmpId>'. (The cause of the error can
  be found in the database.)

* 'eval_added' has the format '<tmpId>:<evalId>'.
2019-08-13 18:18:24 +02:00
Eelco Dolstra
7114d2aceb
Separate payload elements using \t 2019-08-13 18:18:24 +02:00
Eelco Dolstra
2946899504
Turn hydra-notify into a daemon
It now receives notifications about started/finished builds/steps via
PostgreSQL. This gets rid of the (substantial) overhead of starting
hydra-notify for every event. It also allows other programs (even on
other machines) to listen to Hydra notifications.
2019-08-13 18:18:21 +02:00
Eelco Dolstra
ed00f0b25e
Cache flake-based jobset evaluations 2019-05-11 00:41:13 +02:00
Eelco Dolstra
842ef9af24
hydra-eval-jobset: Support flakes 2019-05-11 00:11:38 +02:00
Samuel Dionne-Riel
04ff9e217b Add hydra-dev-server which uses the classic Catalyst server
This, in turns allows

 - Using --restart for reloading the perl code
 - Printing traces on error
2019-01-22 20:34:21 -05:00
Tom Bereknyei
8bffbb7928
evalSettings deprecated, submodule fix 2018-10-03 00:45:42 -04:00
Eelco Dolstra
8e17a413f5
hydra-notify step-finished: Don't barf if the step has no log file 2018-08-01 17:17:46 +02:00
Eelco Dolstra
1898e31ed8
hydra-eval-jobset: Respect email_notification 2018-08-01 16:56:53 +02:00
AmineChikhaoui
298e99449b
add metrics for hydra notifications. 2018-06-12 13:00:39 +01:00
AmineChikhaoui
99e6c19672
add some more statistics about runnable/running builds per arch/system
features.
2018-06-12 13:00:00 +01:00
Eelco Dolstra
8913c682cf
Doh 2017-12-25 15:02:17 +01:00
Eelco Dolstra
0d79f16fd5
git-export -> source 2017-12-25 14:31:30 +01:00
Eelco Dolstra
2660c3cc54
Remove redundant check 2017-12-05 16:00:18 +01:00
Eelco Dolstra
364e21919a
hydra-eval-{jobs,jobset}: Pass file name as <...> 2017-11-28 16:51:00 +01:00
Eelco Dolstra
5986aae704
Add option eval_substituter
When using the "build" or "sysbuild" jobset input types in conjunction
with a binary cache store, the evaluator needs to be able to fetch
store paths from the binary cache. Typical usage:

  store_uri = s3://nix-test-cache?secret-key=...
  eval_substituter = s3://nix-test-cache

Also, the public key of the binary cache must be added to
binary-cache-public-keys in nix.conf, otherwise the local nix-daemon
won't allow the store paths to be copied over.
2017-11-20 16:12:20 +01:00
Eelco Dolstra
2cdc84f34f
Fix build against Nix master
Also, remove support in hydra-eval-jobs for multiple jobset input
alternatives. The web interface hasn't supported this in a long
time. Thus we can use the regular "--arg" handler.
2017-10-26 13:10:14 +02:00
Eelco Dolstra
bc60fccf78
Remove remaining references to store_mode etc. 2017-10-18 12:23:07 +02:00
Remy Goldschmidt
eb7631fb9d hydra-create-user now has --password-hash option (#504)
When creating a Hydra user with the `hydra-create-user` command, you can now
provide a SHA1 password hash with the `--password-hash` flag. This is useful for
the upcoming work on Fully Declarative Hydra, since the end user should not have
to specify plaintext passwords in their `configuration.nix` file.
2017-10-08 11:55:51 +01:00
Eelco Dolstra
dc5e0b120a
Fix a race that can cause hydra-queue-runner to ignore newly added builds
As @dtzWill discovered, with the concurrent hydra-evaluator, there can
be multiple active transactions adding builds to the database. As a
result, builds can become visible in a non-monotonically increasing
order, breaking the queue monitor's assumption that build IDs only go
up.

The fix is to have hydra-eval-jobset provide the lowest build ID it
just added in the builds_added notification, and have the queue
monitor check from there.

Fixes #496.
2017-07-21 14:34:48 +02:00
Eelco Dolstra
803833aba7 Disable the build-queued hook
This can take an excessive amount of time. For example, on
hydra.nixos.org, a call to hydra-notify takes 0.7s even if there are
no plugins. So for an eval with ~45K new builds, the calls to
hydra-notify add up to about 9 hours.

The proper fix would be to pass a list of build IDs, or an eval ID.
2017-06-22 15:09:04 +02:00
Shea Levy
dfb3a52f67 Fix build-queued plugin hook 2017-05-24 11:12:48 -04:00
Shea Levy
c40077ece1 Fix tests 2017-05-24 10:00:38 -04:00
Shea Levy
582c399420 Add buildQueued plugin hook 2017-05-24 09:45:31 -04:00