649 Commits

Author SHA1 Message Date
Graham Christensen
5fae9d96a2
hydra-evaluator: add a 'ONE_AT_A_TIME' evaluator style
In the past, jobsets which are automatically evaluated are evaluated
regularly, on a schedule. This schedule means a new evaluation is
created every checkInterval seconds (assuming something changed.)

This model works well for architectures where our build farm can
easily keep up with demand.

This commit adds a new type of evaluation, called ONE_AT_A_TIME, which
only schedules a new evaluation if the previous evaluation of the
jobset has no unfinished builds.

This model of evaluation lets us have 'low-tier' architectures.

For example, we could now have a jobset for ARMv7l builds, where
the buildfarm only has a single, underpowered ARMv7l builder.
Configuring that jobset as ONE_AT_A_TIME will create an evaluation
and then won't schedule another evaluation until every job of
the existing evaluation is complete.

This way, the cache will have a complete collection of pre-built
software for some commits, but the underpowered architecture will
never become backlogged in ancient revisions.
2020-03-03 19:28:44 -05:00
Eelco Dolstra
15187b059b
Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.

(cherry picked from commit 23c9ca3e94669087d463642baea0cf35a0b8d72f)
2020-02-20 09:58:12 +01:00
Eelco Dolstra
881b7449fd Merge remote-tracking branch 'origin/master' into flake 2020-02-11 14:23:16 +01:00
Graham Christensen
f0f41eaaff
LatestSucceededForJob{,set}: Filter with jobset_id 2020-02-11 07:06:20 -05:00
Kevin Quick
cdd9d6e071
Update haserrormsg logic implementation. 2020-01-20 10:40:33 -08:00
Kevin Quick
de24771a8e
Handle case where jobset has no defined errormsg for api/jobsets 2020-01-11 17:01:44 -08:00
Graham Christensen
5cac40a438
Merge remote-tracking branch 'origin/master' into flake 2019-12-29 16:37:25 -05:00
Graham Christensen
d0f1bda0b8
job prometheus endpoint: drop nixname, too variable 2019-12-29 16:37:13 -05:00
Graham Christensen
d24de1b5de
Merge remote-tracking branch 'origin/master' into flake 2019-12-28 20:58:03 -05:00
Graham Christensen
64cdc3413c
job prometheus endpoint: d'oh 2019-12-28 20:57:27 -05:00
Graham Christensen
511c2db8aa
Merge remote-tracking branch 'origin/master' into flake 2019-12-28 20:22:54 -05:00
Graham Christensen
d5445bfc1d
job: create a prometheus endpoint
Export the most recent stop time and exit status in
a prometheus-friendly format.
2019-12-28 16:41:49 -05:00
Eelco Dolstra
55b0afa08f
Merge remote-tracking branch 'origin/master' into flake 2019-11-07 18:42:15 +01:00
Andreas Rammhold
841a47cabe
Add cancel-build role 2019-11-05 22:56:01 +01:00
Andreas Rammhold
ce1e10c116
Add bump-to-front role 2019-11-05 19:32:06 +01:00
Graham Christensen
937e165328
export a /prometheus endpoint
Currently only shows per-machine build times
2019-09-24 16:50:52 -04:00
Eelco Dolstra
0ccf36ca3b
Merge remote-tracking branch 'origin/master' into flake 2019-09-24 19:03:18 +02:00
Antoine Eiche
c8983ca076 Add haserrormsg boolean attribute to jobset API response
This attribute allows to know if an error occurred or not: when an
error occurs, errormsg is not an empty string. Note we can not use the
errormsg attribute because it can be arbitrarily long and is excluded
from the jobset API response.
2019-08-26 16:07:49 +02:00
tobias pflug
919195b04f Extend the jobset API response
This adds the following (pre-existing) attributes to the jobset response:

- nrtotal
- lastcheckedtime
- starttime
- checkinterval
- triggertime
- fetcherrormsg
- errortime
2019-08-16 16:04:04 +02:00
Eelco Dolstra
2de52d8538
Merge remote-tracking branch 'origin/master' into flake 2019-08-15 13:56:00 +02:00
Eelco Dolstra
92d8d6baa5
Avoid fetching Projects/Jobsets just to get the name column
In particular, doing a 'select * from Jobsets where ...' must be
avoided, because the 'errormsg' column can be very big.
2019-08-13 18:18:25 +02:00
Eelco Dolstra
a74dec6fb1
Merge remote-tracking branch 'origin/master' into flake 2019-08-09 12:46:52 +02:00
Antoine Eiche
d1e590af1f hydra-server: add limit parameter to the search path
This allows a client to set a limit to the search results it wants to
get:

    http://hydra.nixos.org/search?query=emacs&limit=1

This returns only 1 results (while the default is 500).
2019-06-19 14:30:32 +02:00
Antoine Eiche
cb1fce21ba hydra-server: set a limit on builds and buildoutputs search
This patch adds a limit statement for Postgresql queries on `builds`
and `buildsoutputs` tables.
2019-06-19 14:28:55 +02:00
Antoine Eiche
778fc03570 Allow to search builds by hash
Currently, a full store path has to be provided to search in
builds. This patch permits to search jobs with a output path or
derivation hash.

Usecase: we are building Docker images with Hydra. The tag of the
Docker image is the hash of the image output path. This patch would
allow us to find back the build job from the tag of a running
container image.
2019-06-05 11:56:42 +02:00
Eelco Dolstra
f9f595cd21
Add flake configuration to the web interface 2019-05-11 00:11:38 +02:00
Graham Christensen
215ca5da9c
Merge pull request #607 from nlewo/json-search
Add JSON search API endpoint
2019-03-18 15:08:32 -05:00
Graham Christensen
88a92256e1
Merge pull request #636 from Ma27/serve-json-for-evals-and-machines
Serialize data as JSON with `Accept: application/json`
2019-03-17 17:17:17 -04:00
Maximilian Bosch
1cbbc6c52c
Serialize data as JSON with Accept: application/json
Similar to #607. According to the Catalyst[1] docs it's possible to
specify a data structure that is supposed to be serialized when
requesting i.e. a JSON response.

[1] https://metacpan.org/pod/Catalyst::Controller::REST#status_ok
2019-02-14 01:22:48 +01:00
Samuel Dionne-Riel
ed85daf2ac User: jobs tab returns its error as a lazy error. 2018-12-01 13:40:41 -05:00
Samuel Dionne-Riel
9986053e73 Controllers: allows lazy tabs to return custom errors. 2018-12-01 13:40:41 -05:00
Eelco Dolstra
cd234f6a14
Merge pull request #529 from bennofs/feat-all-builds
feat: add /eval/<id>/builds endpoint
2018-11-19 17:36:19 +01:00
Antoine Eiche
0d2a2d8923 Add json output for the search API endpoint
This commit also add a test of this feature.

Note the search JSON output doesn't contain any jobs because they can
not be exported to JSON yet.

The JSON output on a search query matching a build looks like:

```
{
  "builds": [
    {
      "buildoutputs": {
        "out": {
          "path": "/nix/store/wdag3pznrvqk01byk989irg7rq3q2a2c-job"
        }
      },
      "finished": 0,
      "releasename": null,
      "starttime": null,
      "project": "sample",
      "buildproducts": {},
      "timestamp": 1541007629,
      "buildstatus": null,
      "nixname": "job",
      "drvpath": "/nix/store/n9zqndn7j7nyr6gg3bmxvw26cfmdwv2n-job.drv",
      "job": "job",
      "id": 1,
      "stoptime": null,
      "priority": 100,
      "system": "x86_64-linux",
      "jobsetevals": [
        1
      ],
      "jobset": "default",
      "buildmetrics": {}
    }
  ],
  "projects": [],
  "jobsets": [],
  "buildsdrv": []
}
```
2018-11-01 09:31:15 +01:00
Andreas Rammhold
63a294d4ca
allow users with 'restart-jobs' role to restart individual builds 2018-10-04 21:59:42 +02:00
Rob Vermaas
879333fd3c
Make /api/push-github public for private Hydra instances. This makes it possible to use webhooks to trigger evaluations for private Hydra instances. 2018-06-14 17:23:01 +02:00
Eelco Dolstra
85524b2d0f
Disable jobset deletion
This is a good way to make Hydra hang. (E.g. we had a deletion of
nixos:gcc-7 running for > 12 hours and blocking UPDATE statements from
hydra-queue-runner.) Generally it's better to just disable/hide an old
jobset anyway.
2018-03-13 14:05:55 +01:00
Eelco Dolstra
02888105a8
Add a hydra.conf option to enable email notification
Note that it's disabled by default.
2018-03-12 14:04:23 +01:00
Eelco Dolstra
fd463ff403
Remove nix-install-package support
'nix-install-package' no longer exists in Nix 1.12.

Closes #268.
2018-01-15 14:27:58 +01:00
Benno Fünfstück
3ab5d299c1 feat: add /eval/<id>/builds endpoint
This endpoint allows efficient retrieval of all the builds in an
evaluation, without making a request for each single build.
2018-01-14 01:29:37 +01:00
Graham Christensen
83a48d2d4f
Add a restart-jobs role
Frequently users want Hydra access just to restart jobs. However,
prior to this commit the only way to grant that access was by giving
them full Admin access which isn't necessarily what we want to do.

By having a restart-jobs role, we can grant this privilege to users
who are known to the community and want to help, but aren't long-time
members.

I haven't tested this commit, but it looks good to me...
2017-12-30 08:28:23 -05:00
Eelco Dolstra
7ccec0900b Fix links to steps in the machines page 2017-12-07 16:20:23 +01:00
Eelco Dolstra
e9670641ec
Distinguish build step states
The web interface now shows whether a build step is connecting,
copying inputs/outputs, building, etc.
2017-12-07 15:35:31 +01:00
Eelco Dolstra
7857f83251
Serve directories and symlinks correctly 2017-11-14 17:17:07 +01:00
Eelco Dolstra
62c6cca937
Make one-click installs work for binary cache stores
This requires setting binary_cache_public_uri in hydra.conf.
2017-10-18 14:09:28 +02:00
Eelco Dolstra
44ef743d84
Hide some things that require a local store 2017-10-18 13:44:41 +02:00
Eelco Dolstra
30943dd217
Detect MIME type 2017-10-18 13:38:34 +02:00
Eelco Dolstra
42fbde0383
Use "nix cat-store" to serve files from the Nix store
This makes downloading/viewing build results work with binary cache
stores. For good performance, this should be used in conjunction with
ca580bec35,
i.e. you should set store_uri to something like

  s3://my-cache?local-nar-cache=/tmp/nar-cache

to cache NARs between requests.
2017-10-18 12:48:31 +02:00
Eelco Dolstra
bc60fccf78
Remove remaining references to store_mode etc. 2017-10-18 12:23:07 +02:00
Rob Vermaas
b645b03536
Fix string comparison 2017-09-25 11:54:55 +00:00
Rob Vermaas
2318baaade
Add 'readonly_ips' option, to allow readonly access for certain IPs when 'private' option is enabled. 2017-09-22 11:36:49 +00:00