2435 Commits

Author SHA1 Message Date
Graham Christensen
624f1d8d2d
Jobs: populate Jobs.jobset_id field when writing from hydra-eval-jobset 2020-02-10 11:43:02 -05:00
Graham Christensen
efa1f1d4fb
Jobs: add a nullable jobset_id foreign key to Jobsets.
Also, adds an explicitly named "jobs" accessor to the Jobsets
Schema object, which uses the project/jobset name.
2020-02-10 11:43:02 -05:00
Graham Christensen
e00030563b
Jobsets: add a SERIAL, unique, non-null id column
A postgresql column which is non-null and unique is treated with
the same optimisations as a primary key, so we have no need to
try and recreate the `id` as the primary key.

No read paths are impacted by this change, and the database will
automatically create an ID for each insert. Thus, no code needs to
change.
2020-02-10 11:42:59 -05:00
Graham Christensen
6fe57ab5fa
Copy the flake migration from the flake branch
hydra.nixos.org is already running this rev, and it should be safe to
apply to everyone else. If we make changes to this migration, we'll
need to write another migration anyway.
2020-02-09 15:21:28 -05:00
Graham Christensen
c2f932a7e3
sql: Generate models from postgresql
Lowercasing is due to postgresql not having case-sensitive table names.
It always technically workde before, but those table names never
existed literally.

The switch to generating from postgresql is to handle an upcoming
addition of an auto-incrementign ID to the Jobset table. Sqlite doesn't
seem to be able to handle the table having an auto incrementing ID
field which isn't the primary key, but we can't change the primary
key trivially.

Since hydra doesn't support sqlite and hasn't for many year anyway,
it is easier to just generate from pgsql directly.
2020-02-06 12:23:47 -05:00
Eelco Dolstra
ba8814a245 Fix build 2020-02-03 18:43:45 +01:00
Eelco Dolstra
bb94677526 Merge remote-tracking branch 'origin/master' into flake 2020-02-03 17:49:01 +01:00
Bas van Dijk
5fab5e935b Remove the "log diff" buttons
because they're referencing the removed `logdiff` API.

This API was removed in 4d1816b1529e4877f9527f039facbe9953c187bd.

Fixes #409
2020-01-26 21:45:38 +01:00
Bas van Dijk
bd9b656c54 Fix printing aggregate status
`nrMembers` is undefined and it should have clearly be `nrConstituents`
which is calculated just before.

Fixes #693.
2020-01-26 20:15:18 +01:00
Kevin Quick
cdd9d6e071
Update haserrormsg logic implementation. 2020-01-20 10:40:33 -08:00
Bas van Dijk
08bfff9d13 hydra-queue-runner: sleep 5s after handling an exception
instead of immediately calling `readMachinesFiles` again which could
immediately throw another exception again.
2020-01-14 13:34:35 +01:00
Kevin Quick
ec8b2970e0
Remove added strictness to allow multiple query returns. 2020-01-12 10:18:52 -08:00
Kevin Quick
7bb6b5e206
Update libpqxx usage to move away from deprecated API interactions. 2020-01-11 22:38:40 -08:00
Kevin Quick
de24771a8e
Handle case where jobset has no defined errormsg for api/jobsets 2020-01-11 17:01:44 -08:00
Hamish Mackenzie
c40c887e50
Fixes for macOS
Building on macOS with the latest nixpkgs master and NixOS/nixpkgs#77147
fails.  It seems some `std::experimental` (optional) for instance are
not available as `experimental`, but are in `std`.  Also `toJSON` is
missing for `atomic< unsigned long long >`.
2020-01-07 12:38:06 +13:00
Eelco Dolstra
e7f2139e25 Build against nix-master 2019-12-30 22:49:26 +01:00
Graham Christensen
5cac40a438
Merge remote-tracking branch 'origin/master' into flake 2019-12-29 16:37:25 -05:00
Graham Christensen
d0f1bda0b8
job prometheus endpoint: drop nixname, too variable 2019-12-29 16:37:13 -05:00
Graham Christensen
d24de1b5de
Merge remote-tracking branch 'origin/master' into flake 2019-12-28 20:58:03 -05:00
Graham Christensen
64cdc3413c
job prometheus endpoint: d'oh 2019-12-28 20:57:27 -05:00
Graham Christensen
511c2db8aa
Merge remote-tracking branch 'origin/master' into flake 2019-12-28 20:22:54 -05:00
Graham Christensen
d5445bfc1d
job: create a prometheus endpoint
Export the most recent stop time and exit status in
a prometheus-friendly format.
2019-12-28 16:41:49 -05:00
Nikola Knezevic
06abfd6b2f hydra-send-stats: Cleanup removed metrics
In 29468995040ae21e0e1c14c1bdbb16ccb514caa8 these metrics got removed
due to refactoring of how notifications work.
2019-11-13 11:42:58 +01:00
Eelco Dolstra
55b0afa08f
Merge remote-tracking branch 'origin/master' into flake 2019-11-07 18:42:15 +01:00
Andreas Rammhold
841a47cabe
Add cancel-build role 2019-11-05 22:56:01 +01:00
Andreas Rammhold
ce1e10c116
Add bump-to-front role 2019-11-05 19:32:06 +01:00
Bas van Dijk
840e99f859 hydra-eval-jobset: $firstOutput is not used so can be removed 2019-11-05 13:58:40 +01:00
Janne Heß
6f99d958bc Fix declarative flake builds 2019-10-27 14:57:53 +01:00
Eelco Dolstra
a816730d8b uri -> url 2019-10-23 20:31:27 +02:00
Eelco Dolstra
43d4bc9108 Fix reproduction instructions 2019-10-23 16:29:12 +02:00
Eelco Dolstra
037f6488f6
Merge pull request #680 from grahamc/prometheus
export a /prometheus endpoint
2019-09-25 19:46:42 +02:00
Eelco Dolstra
7c6fd83dda
Merge remote-tracking branch 'origin/master' into flake 2019-09-25 17:28:00 +02:00
Eelco Dolstra
d4b4255dd2
hydra-queue-runner: Support running in a NixOS container
In a NixOS container, cmdBuildDerivation doesn't work because we're
not privileged. But we also don't need it because the store already
has the derivation.

Also, don't copy from/to the store since this gives errors about
missing signatures.
2019-09-25 17:26:03 +02:00
Eelco Dolstra
554bb7d9ce
hydra-queue-runner: Don't pass IN_SYSTEMD to child processes
This caused local builds to have journal priority prefixes
(e.g. '<3>').
2019-09-25 17:25:07 +02:00
Graham Christensen
937e165328
export a /prometheus endpoint
Currently only shows per-machine build times
2019-09-24 16:50:52 -04:00
Eelco Dolstra
0ccf36ca3b
Merge remote-tracking branch 'origin/master' into flake 2019-09-24 19:03:18 +02:00
Antoine Eiche
c8983ca076 Add haserrormsg boolean attribute to jobset API response
This attribute allows to know if an error occurred or not: when an
error occurs, errormsg is not an empty string. Note we can not use the
errormsg attribute because it can be arbitrarily long and is excluded
from the jobset API response.
2019-08-26 16:07:49 +02:00
Rob Vermaas
f10b2c2da8 Update Hydra schema, otherwise hydra-notify will not work. 2019-08-19 17:05:11 +02:00
tobias pflug
919195b04f Extend the jobset API response
This adds the following (pre-existing) attributes to the jobset response:

- nrtotal
- lastcheckedtime
- starttime
- checkinterval
- triggertime
- fetcherrormsg
- errortime
2019-08-16 16:04:04 +02:00
Eelco Dolstra
2de52d8538
Merge remote-tracking branch 'origin/master' into flake 2019-08-15 13:56:00 +02:00
Eelco Dolstra
c8a4030c5f
Fix error in GitlabStatus plugin
May 15 09:20:10 chef hydra-queue-runner[27523]: Hydra::Plugin::GitlabStatus=HASH(0x519a7b8)->buildFinished: Can't call method "value" on an undefined value at /nix/store/858hinflxcl2jd12wv1r3a8j11ybsf6w-hydra-0.1.2629.89fa829/libexec/hydra/lib/Hydra/Plugin/GitlabStatus.pm line 57.

(cherry picked from commit 438ddf52898cddf100cf59ca91f32581368c9b2e)
2019-08-15 13:55:47 +02:00
Eelco Dolstra
92d8d6baa5
Avoid fetching Projects/Jobsets just to get the name column
In particular, doing a 'select * from Jobsets where ...' must be
avoided, because the 'errormsg' column can be very big.
2019-08-13 18:18:25 +02:00
Eelco Dolstra
16811d3e78
Plugins: Add isEnabled method
Plugins are now disabled at startup time unless there is some relevant
configuration in hydra.conf. This avoids hydra-notify having to do a
lot of redundant work (a lot of plugins did a lot of database queries
*before* deciding they were disabled).

Note: BitBucketStatus users will need to add 'enable_bitbucket_status
= 1' to hydra.conf.
2019-08-13 18:18:25 +02:00
Eelco Dolstra
f49a089fc0
hydra-notify: Don't do an unnecessary fetch of Jobsets 2019-08-13 18:18:24 +02:00
Eelco Dolstra
d08cfa48d7
Add a 'step_started' notification 2019-08-13 18:18:24 +02:00
Eelco Dolstra
72c36373bb
hydra-notify: Fix processing notifications 2019-08-13 18:18:24 +02:00
Eelco Dolstra
7c7cc8c059
Fix duplicate step_finished notification 2019-08-13 18:18:24 +02:00
Eelco Dolstra
976d88d675
Send notifications when evaluations start/finish/fail
* 'eval_started' has the format '<tmpId>\t<project>\t<jobset>'.

* 'eval_failed' has the format '<tmpId>'. (The cause of the error can
  be found in the database.)

* 'eval_added' has the format '<tmpId>:<evalId>'.
2019-08-13 18:18:24 +02:00
Eelco Dolstra
7114d2aceb
Separate payload elements using \t 2019-08-13 18:18:24 +02:00
Eelco Dolstra
2946899504
Turn hydra-notify into a daemon
It now receives notifications about started/finished builds/steps via
PostgreSQL. This gets rid of the (substantial) overhead of starting
hydra-notify for every event. It also allows other programs (even on
other machines) to listen to Hydra notifications.
2019-08-13 18:18:21 +02:00