Commit Graph

3074 Commits

Author SHA1 Message Date
7bb6b5e206 Update libpqxx usage to move away from deprecated API interactions. 2020-01-11 22:38:40 -08:00
de24771a8e Handle case where jobset has no defined errormsg for api/jobsets 2020-01-11 17:01:44 -08:00
c40c887e50 Fixes for macOS
Building on macOS with the latest nixpkgs master and 
fails.  It seems some `std::experimental` (optional) for instance are
not available as `experimental`, but are in `std`.  Also `toJSON` is
missing for `atomic< unsigned long long >`.
2020-01-07 12:38:06 +13:00
e7f2139e25 Build against nix-master 2019-12-30 22:49:26 +01:00
5cac40a438 Merge remote-tracking branch 'origin/master' into flake 2019-12-29 16:37:25 -05:00
d0f1bda0b8 job prometheus endpoint: drop nixname, too variable 2019-12-29 16:37:13 -05:00
d24de1b5de Merge remote-tracking branch 'origin/master' into flake 2019-12-28 20:58:03 -05:00
64cdc3413c job prometheus endpoint: d'oh 2019-12-28 20:57:27 -05:00
511c2db8aa Merge remote-tracking branch 'origin/master' into flake 2019-12-28 20:22:54 -05:00
d5445bfc1d job: create a prometheus endpoint
Export the most recent stop time and exit status in
a prometheus-friendly format.
2019-12-28 16:41:49 -05:00
06abfd6b2f hydra-send-stats: Cleanup removed metrics
In 2946899504 these metrics got removed
due to refactoring of how notifications work.
2019-11-13 11:42:58 +01:00
55b0afa08f Merge remote-tracking branch 'origin/master' into flake 2019-11-07 18:42:15 +01:00
841a47cabe Add cancel-build role 2019-11-05 22:56:01 +01:00
ce1e10c116 Add bump-to-front role 2019-11-05 19:32:06 +01:00
840e99f859 hydra-eval-jobset: $firstOutput is not used so can be removed 2019-11-05 13:58:40 +01:00
6f99d958bc Fix declarative flake builds 2019-10-27 14:57:53 +01:00
a816730d8b uri -> url 2019-10-23 20:31:27 +02:00
43d4bc9108 Fix reproduction instructions 2019-10-23 16:29:12 +02:00
037f6488f6 Merge pull request from grahamc/prometheus
export a /prometheus endpoint
2019-09-25 19:46:42 +02:00
7c6fd83dda Merge remote-tracking branch 'origin/master' into flake 2019-09-25 17:28:00 +02:00
d4b4255dd2 hydra-queue-runner: Support running in a NixOS container
In a NixOS container, cmdBuildDerivation doesn't work because we're
not privileged. But we also don't need it because the store already
has the derivation.

Also, don't copy from/to the store since this gives errors about
missing signatures.
2019-09-25 17:26:03 +02:00
554bb7d9ce hydra-queue-runner: Don't pass IN_SYSTEMD to child processes
This caused local builds to have journal priority prefixes
(e.g. '<3>').
2019-09-25 17:25:07 +02:00
937e165328 export a /prometheus endpoint
Currently only shows per-machine build times
2019-09-24 16:50:52 -04:00
0ccf36ca3b Merge remote-tracking branch 'origin/master' into flake 2019-09-24 19:03:18 +02:00
c8983ca076 Add haserrormsg boolean attribute to jobset API response
This attribute allows to know if an error occurred or not: when an
error occurs, errormsg is not an empty string. Note we can not use the
errormsg attribute because it can be arbitrarily long and is excluded
from the jobset API response.
2019-08-26 16:07:49 +02:00
f10b2c2da8 Update Hydra schema, otherwise hydra-notify will not work. 2019-08-19 17:05:11 +02:00
919195b04f Extend the jobset API response
This adds the following (pre-existing) attributes to the jobset response:

- nrtotal
- lastcheckedtime
- starttime
- checkinterval
- triggertime
- fetcherrormsg
- errortime
2019-08-16 16:04:04 +02:00
2de52d8538 Merge remote-tracking branch 'origin/master' into flake 2019-08-15 13:56:00 +02:00
c8a4030c5f Fix error in GitlabStatus plugin
May 15 09:20:10 chef hydra-queue-runner[27523]: Hydra::Plugin::GitlabStatus=HASH(0x519a7b8)->buildFinished: Can't call method "value" on an undefined value at /nix/store/858hinflxcl2jd12wv1r3a8j11ybsf6w-hydra-0.1.2629.89fa829/libexec/hydra/lib/Hydra/Plugin/GitlabStatus.pm line 57.

(cherry picked from commit 438ddf5289)
2019-08-15 13:55:47 +02:00
92d8d6baa5 Avoid fetching Projects/Jobsets just to get the name column
In particular, doing a 'select * from Jobsets where ...' must be
avoided, because the 'errormsg' column can be very big.
2019-08-13 18:18:25 +02:00
16811d3e78 Plugins: Add isEnabled method
Plugins are now disabled at startup time unless there is some relevant
configuration in hydra.conf. This avoids hydra-notify having to do a
lot of redundant work (a lot of plugins did a lot of database queries
*before* deciding they were disabled).

Note: BitBucketStatus users will need to add 'enable_bitbucket_status
= 1' to hydra.conf.
2019-08-13 18:18:25 +02:00
f49a089fc0 hydra-notify: Don't do an unnecessary fetch of Jobsets 2019-08-13 18:18:24 +02:00
d08cfa48d7 Add a 'step_started' notification 2019-08-13 18:18:24 +02:00
72c36373bb hydra-notify: Fix processing notifications 2019-08-13 18:18:24 +02:00
7c7cc8c059 Fix duplicate step_finished notification 2019-08-13 18:18:24 +02:00
976d88d675 Send notifications when evaluations start/finish/fail
* 'eval_started' has the format '<tmpId>\t<project>\t<jobset>'.

* 'eval_failed' has the format '<tmpId>'. (The cause of the error can
  be found in the database.)

* 'eval_added' has the format '<tmpId>:<evalId>'.
2019-08-13 18:18:24 +02:00
7114d2aceb Separate payload elements using \t 2019-08-13 18:18:24 +02:00
2946899504 Turn hydra-notify into a daemon
It now receives notifications about started/finished builds/steps via
PostgreSQL. This gets rid of the (substantial) overhead of starting
hydra-notify for every event. It also allows other programs (even on
other machines) to listen to Hydra notifications.
2019-08-13 18:18:21 +02:00
a74dec6fb1 Merge remote-tracking branch 'origin/master' into flake 2019-08-09 12:46:52 +02:00
06bdc8f85c Added the InfluxDBNotification plugin including a NixOS test
This adds a `InfluxDBNotification` plugin which is configured as:

```
<influxdb>
  url = http://127.0.0.1:8086
  db = hydra
</influxdb>
```

which will write a notification for every finished job to the
configured database in InfluxDB looking like:

```
hydra_build_status,cached=false,job=job,jobset=default,project=sample,repo=default,result=success,status=success,system=x86_64-linux build_id="1",build_status=0i,closure_size=584i,duration=0i,main_build_id="1",queued=0i,size=168i 1564156212
```
2019-07-26 17:47:03 +02:00
8a0a5ec3a3 Create extension pg_trgm in the NixOS module
The creation of the `pg_trgm` extension needs superuser power. So,
this patch makes the extension creation in the Hydra NixOS module when
a local database is used.

If it is not possible to create this extension (remote database for
instance with nosuperuser), the creation of the `pg_trgm` index is
skipped (this index speedup queries on builds.drvpath) and warnings
are emitted:

    initialising the Hydra database schema...
    WARNING:  Can not create extension pg_trgm: permission denied to create extension "pg_trgm"
    WARNING:  HINT: Temporary provide superuser role to your Hydra Postgresql user and run the script src/sql/upgrade-57.sql
    WARNING:  The pg_trgm index on builds.drvpath has been skipped (slower complex queries on builds.drvpath)

This allows to keep smooth migrations: the migration process doesn't
require a manual step (but this manual step is recommended on big
remote databases).
2019-07-25 16:19:33 +02:00
42784a9053 sql: refactor some sql statements to lowercase 2019-07-25 16:19:33 +02:00
d1e590af1f hydra-server: add limit parameter to the search path
This allows a client to set a limit to the search results it wants to
get:

    http://hydra.nixos.org/search?query=emacs&limit=1

This returns only 1 results (while the default is 500).
2019-06-19 14:30:32 +02:00
cb1fce21ba hydra-server: set a limit on builds and buildoutputs search
This patch adds a limit statement for Postgresql queries on `builds`
and `buildsoutputs` tables.
2019-06-19 14:28:55 +02:00
7935cffd6b Create a pg_trgm index on builds.drvpath
The search query uses the LIKE operator which requires a sequential
scan (it can't use the already existing B-tree index). This new
index (trigram) avoids a sequential scan of the builds table when the
LIKE operator is used.

Here is the analyze of a request on the builds table with this index:

     explain analyze select * from builds where drvpath like '%k3r71gz0gv16ld8rhcp2bb8gb5w1xc4b%';
                                                                 QUERY PLAN
     -----------------------------------------------------------------------------------------------------------------------------------
      Bitmap Heap Scan on builds  (cost=128.00..132.01 rows=1 width=492) (actual time=0.070..0.077 rows=1 loops=1)
        Recheck Cond: (drvpath ~~ '%k3r71gz0gv16ld8rhcp2bb8gb5w1xc4b%'::text)
        ->  Bitmap Index Scan on indextrgmbuildsondrvpath  (cost=0.00..128.00 rows=1 width=0) (actual time=0.047..0.047 rows=3 loops=1)
              Index Cond: (drvpath ~~ '%k3r71gz0gv16ld8rhcp2bb8gb5w1xc4b%'::text)
      Total runtime: 0.206 ms
     (5 rows)
2019-06-06 14:28:25 +02:00
778fc03570 Allow to search builds by hash
Currently, a full store path has to be provided to search in
builds. This patch permits to search jobs with a output path or
derivation hash.

Usecase: we are building Docker images with Hydra. The tag of the
Docker image is the hash of the image output path. This patch would
allow us to find back the build job from the tag of a running
container image.
2019-06-05 11:56:42 +02:00
7936a29364 hydra-eval-jobs: Use 'checks' output if there is no 'hydraJobs' output 2019-06-03 13:55:32 +02:00
9a8880fd28 Doh 2019-06-03 13:53:22 +02:00
0042183717 Fix build 2019-06-03 12:23:49 +02:00
b7be2bf9ad Fix build against the latest flakes branch 2019-05-22 14:51:55 +02:00