Commit Graph

701 Commits

Author SHA1 Message Date
Eelco Dolstra
3cc1deb125 Merge pull request #721 from grahamc/one-by-one
hydra-evaluator: add a 'ONE_AT_A_TIME' evaluator style
2020-03-04 08:44:43 +01:00
Graham Christensen
994430b94b treewide: allow nix command 2020-03-03 22:52:20 -05:00
Graham Christensen
5fae9d96a2 hydra-evaluator: add a 'ONE_AT_A_TIME' evaluator style
In the past, jobsets which are automatically evaluated are evaluated
regularly, on a schedule. This schedule means a new evaluation is
created every checkInterval seconds (assuming something changed.)

This model works well for architectures where our build farm can
easily keep up with demand.

This commit adds a new type of evaluation, called ONE_AT_A_TIME, which
only schedules a new evaluation if the previous evaluation of the
jobset has no unfinished builds.

This model of evaluation lets us have 'low-tier' architectures.

For example, we could now have a jobset for ARMv7l builds, where
the buildfarm only has a single, underpowered ARMv7l builder.
Configuring that jobset as ONE_AT_A_TIME will create an evaluation
and then won't schedule another evaluation until every job of
the existing evaluation is complete.

This way, the cache will have a complete collection of pre-built
software for some commits, but the underpowered architecture will
never become backlogged in ancient revisions.
2020-03-03 19:28:44 -05:00
Eelco Dolstra
15187b059b Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.

(cherry picked from commit 23c9ca3e94)
2020-02-20 09:58:12 +01:00
Eelco Dolstra
881b7449fd Merge remote-tracking branch 'origin/master' into flake 2020-02-11 14:23:16 +01:00
Graham Christensen
f0f41eaaff LatestSucceededForJob{,set}: Filter with jobset_id 2020-02-11 07:06:20 -05:00
Kevin Quick
cdd9d6e071 Update haserrormsg logic implementation. 2020-01-20 10:40:33 -08:00
Kevin Quick
de24771a8e Handle case where jobset has no defined errormsg for api/jobsets 2020-01-11 17:01:44 -08:00
Graham Christensen
5cac40a438 Merge remote-tracking branch 'origin/master' into flake 2019-12-29 16:37:25 -05:00
Graham Christensen
d0f1bda0b8 job prometheus endpoint: drop nixname, too variable 2019-12-29 16:37:13 -05:00
Graham Christensen
d24de1b5de Merge remote-tracking branch 'origin/master' into flake 2019-12-28 20:58:03 -05:00
Graham Christensen
64cdc3413c job prometheus endpoint: d'oh 2019-12-28 20:57:27 -05:00
Graham Christensen
511c2db8aa Merge remote-tracking branch 'origin/master' into flake 2019-12-28 20:22:54 -05:00
Graham Christensen
d5445bfc1d job: create a prometheus endpoint
Export the most recent stop time and exit status in
a prometheus-friendly format.
2019-12-28 16:41:49 -05:00
Eelco Dolstra
55b0afa08f Merge remote-tracking branch 'origin/master' into flake 2019-11-07 18:42:15 +01:00
Andreas Rammhold
841a47cabe Add cancel-build role 2019-11-05 22:56:01 +01:00
Andreas Rammhold
ce1e10c116 Add bump-to-front role 2019-11-05 19:32:06 +01:00
Graham Christensen
937e165328 export a /prometheus endpoint
Currently only shows per-machine build times
2019-09-24 16:50:52 -04:00
Eelco Dolstra
0ccf36ca3b Merge remote-tracking branch 'origin/master' into flake 2019-09-24 19:03:18 +02:00
Antoine Eiche
c8983ca076 Add haserrormsg boolean attribute to jobset API response
This attribute allows to know if an error occurred or not: when an
error occurs, errormsg is not an empty string. Note we can not use the
errormsg attribute because it can be arbitrarily long and is excluded
from the jobset API response.
2019-08-26 16:07:49 +02:00
tobias pflug
919195b04f Extend the jobset API response
This adds the following (pre-existing) attributes to the jobset response:

- nrtotal
- lastcheckedtime
- starttime
- checkinterval
- triggertime
- fetcherrormsg
- errortime
2019-08-16 16:04:04 +02:00
Eelco Dolstra
2de52d8538 Merge remote-tracking branch 'origin/master' into flake 2019-08-15 13:56:00 +02:00
Eelco Dolstra
92d8d6baa5 Avoid fetching Projects/Jobsets just to get the name column
In particular, doing a 'select * from Jobsets where ...' must be
avoided, because the 'errormsg' column can be very big.
2019-08-13 18:18:25 +02:00
Eelco Dolstra
a74dec6fb1 Merge remote-tracking branch 'origin/master' into flake 2019-08-09 12:46:52 +02:00
Antoine Eiche
d1e590af1f hydra-server: add limit parameter to the search path
This allows a client to set a limit to the search results it wants to
get:

    http://hydra.nixos.org/search?query=emacs&limit=1

This returns only 1 results (while the default is 500).
2019-06-19 14:30:32 +02:00
Antoine Eiche
cb1fce21ba hydra-server: set a limit on builds and buildoutputs search
This patch adds a limit statement for Postgresql queries on `builds`
and `buildsoutputs` tables.
2019-06-19 14:28:55 +02:00
Antoine Eiche
778fc03570 Allow to search builds by hash
Currently, a full store path has to be provided to search in
builds. This patch permits to search jobs with a output path or
derivation hash.

Usecase: we are building Docker images with Hydra. The tag of the
Docker image is the hash of the image output path. This patch would
allow us to find back the build job from the tag of a running
container image.
2019-06-05 11:56:42 +02:00
Eelco Dolstra
f9f595cd21 Add flake configuration to the web interface 2019-05-11 00:11:38 +02:00
Graham Christensen
215ca5da9c Merge pull request #607 from nlewo/json-search
Add JSON search API endpoint
2019-03-18 15:08:32 -05:00
Graham Christensen
88a92256e1 Merge pull request #636 from Ma27/serve-json-for-evals-and-machines
Serialize data as JSON with `Accept: application/json`
2019-03-17 17:17:17 -04:00
Maximilian Bosch
1cbbc6c52c Serialize data as JSON with Accept: application/json
Similar to #607. According to the Catalyst[1] docs it's possible to
specify a data structure that is supposed to be serialized when
requesting i.e. a JSON response.

[1] https://metacpan.org/pod/Catalyst::Controller::REST#status_ok
2019-02-14 01:22:48 +01:00
Samuel Dionne-Riel
ed85daf2ac User: jobs tab returns its error as a lazy error. 2018-12-01 13:40:41 -05:00
Samuel Dionne-Riel
9986053e73 Controllers: allows lazy tabs to return custom errors. 2018-12-01 13:40:41 -05:00
Eelco Dolstra
cd234f6a14 Merge pull request #529 from bennofs/feat-all-builds
feat: add /eval/<id>/builds endpoint
2018-11-19 17:36:19 +01:00
Antoine Eiche
0d2a2d8923 Add json output for the search API endpoint
This commit also add a test of this feature.

Note the search JSON output doesn't contain any jobs because they can
not be exported to JSON yet.

The JSON output on a search query matching a build looks like:

```
{
  "builds": [
    {
      "buildoutputs": {
        "out": {
          "path": "/nix/store/wdag3pznrvqk01byk989irg7rq3q2a2c-job"
        }
      },
      "finished": 0,
      "releasename": null,
      "starttime": null,
      "project": "sample",
      "buildproducts": {},
      "timestamp": 1541007629,
      "buildstatus": null,
      "nixname": "job",
      "drvpath": "/nix/store/n9zqndn7j7nyr6gg3bmxvw26cfmdwv2n-job.drv",
      "job": "job",
      "id": 1,
      "stoptime": null,
      "priority": 100,
      "system": "x86_64-linux",
      "jobsetevals": [
        1
      ],
      "jobset": "default",
      "buildmetrics": {}
    }
  ],
  "projects": [],
  "jobsets": [],
  "buildsdrv": []
}
```
2018-11-01 09:31:15 +01:00
Andreas Rammhold
63a294d4ca allow users with 'restart-jobs' role to restart individual builds 2018-10-04 21:59:42 +02:00
Rob Vermaas
879333fd3c Make /api/push-github public for private Hydra instances. This makes it possible to use webhooks to trigger evaluations for private Hydra instances. 2018-06-14 17:23:01 +02:00
Eelco Dolstra
85524b2d0f Disable jobset deletion
This is a good way to make Hydra hang. (E.g. we had a deletion of
nixos:gcc-7 running for > 12 hours and blocking UPDATE statements from
hydra-queue-runner.) Generally it's better to just disable/hide an old
jobset anyway.
2018-03-13 14:05:55 +01:00
Eelco Dolstra
02888105a8 Add a hydra.conf option to enable email notification
Note that it's disabled by default.
2018-03-12 14:04:23 +01:00
Eelco Dolstra
fd463ff403 Remove nix-install-package support
'nix-install-package' no longer exists in Nix 1.12.

Closes #268.
2018-01-15 14:27:58 +01:00
Benno Fünfstück
3ab5d299c1 feat: add /eval/<id>/builds endpoint
This endpoint allows efficient retrieval of all the builds in an
evaluation, without making a request for each single build.
2018-01-14 01:29:37 +01:00
Graham Christensen
83a48d2d4f Add a restart-jobs role
Frequently users want Hydra access just to restart jobs. However,
prior to this commit the only way to grant that access was by giving
them full Admin access which isn't necessarily what we want to do.

By having a restart-jobs role, we can grant this privilege to users
who are known to the community and want to help, but aren't long-time
members.

I haven't tested this commit, but it looks good to me...
2017-12-30 08:28:23 -05:00
Eelco Dolstra
7ccec0900b Fix links to steps in the machines page 2017-12-07 16:20:23 +01:00
Eelco Dolstra
e9670641ec Distinguish build step states
The web interface now shows whether a build step is connecting,
copying inputs/outputs, building, etc.
2017-12-07 15:35:31 +01:00
Eelco Dolstra
7857f83251 Serve directories and symlinks correctly 2017-11-14 17:17:07 +01:00
Eelco Dolstra
62c6cca937 Make one-click installs work for binary cache stores
This requires setting binary_cache_public_uri in hydra.conf.
2017-10-18 14:09:28 +02:00
Eelco Dolstra
44ef743d84 Hide some things that require a local store 2017-10-18 13:44:41 +02:00
Eelco Dolstra
30943dd217 Detect MIME type 2017-10-18 13:38:34 +02:00
Eelco Dolstra
42fbde0383 Use "nix cat-store" to serve files from the Nix store
This makes downloading/viewing build results work with binary cache
stores. For good performance, this should be used in conjunction with
ca580bec35,
i.e. you should set store_uri to something like

  s3://my-cache?local-nar-cache=/tmp/nar-cache

to cache NARs between requests.
2017-10-18 12:48:31 +02:00
Eelco Dolstra
bc60fccf78 Remove remaining references to store_mode etc. 2017-10-18 12:23:07 +02:00