ahuston-0/hydra
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
2,671 Commits 6 Branches 0 Tags
de24771a8ea84a7ffe2b47cba8a613d432830604
Commit Graph

1004 Commits

Author SHA1 Message Date
Kevin Quick
de24771a8e Handle case where jobset has no defined errormsg for api/jobsets 2020-01-11 17:01:44 -08:00
Graham Christensen
d0f1bda0b8 job prometheus endpoint: drop nixname, too variable 2019-12-29 16:37:13 -05:00
Graham Christensen
64cdc3413c job prometheus endpoint: d'oh 2019-12-28 20:57:27 -05:00
Graham Christensen
d5445bfc1d job: create a prometheus endpoint 
Export the most recent stop time and exit status in
a prometheus-friendly format.
 2019-12-28 16:41:49 -05:00
Andreas Rammhold
841a47cabe Add cancel-build role 2019-11-05 22:56:01 +01:00
Andreas Rammhold
ce1e10c116 Add bump-to-front role 2019-11-05 19:32:06 +01:00
Graham Christensen
937e165328 export a /prometheus endpoint 
Currently only shows per-machine build times
 2019-09-24 16:50:52 -04:00
Antoine Eiche
c8983ca076 Add haserrormsg boolean attribute to jobset API response 
This attribute allows to know if an error occurred or not: when an
error occurs, errormsg is not an empty string. Note we can not use the
errormsg attribute because it can be arbitrarily long and is excluded
from the jobset API response.
 2019-08-26 16:07:49 +02:00
Rob Vermaas
f10b2c2da8 Update Hydra schema, otherwise hydra-notify will not work. 2019-08-19 17:05:11 +02:00
tobias pflug
919195b04f Extend the jobset API response 
This adds the following (pre-existing) attributes to the jobset response:

- nrtotal
- lastcheckedtime
- starttime
- checkinterval
- triggertime
- fetcherrormsg
- errortime
 2019-08-16 16:04:04 +02:00
Eelco Dolstra
c8a4030c5f Fix error in GitlabStatus plugin 
May 15 09:20:10 chef hydra-queue-runner[27523]: Hydra::Plugin::GitlabStatus=HASH(0x519a7b8)->buildFinished: Can't call method "value" on an undefined value at /nix/store/858hinflxcl2jd12wv1r3a8j11ybsf6w-hydra-0.1.2629.89fa829/libexec/hydra/lib/Hydra/Plugin/GitlabStatus.pm line 57.

(cherry picked from commit 438ddf5289)
 2019-08-15 13:55:47 +02:00
Eelco Dolstra
92d8d6baa5 Avoid fetching Projects/Jobsets just to get the name column 
In particular, doing a 'select * from Jobsets where ...' must be
avoided, because the 'errormsg' column can be very big.
 2019-08-13 18:18:25 +02:00
Eelco Dolstra
16811d3e78 Plugins: Add isEnabled method 
Plugins are now disabled at startup time unless there is some relevant
configuration in hydra.conf. This avoids hydra-notify having to do a
lot of redundant work (a lot of plugins did a lot of database queries
*before* deciding they were disabled).

Note: BitBucketStatus users will need to add 'enable_bitbucket_status
= 1' to hydra.conf.
 2019-08-13 18:18:25 +02:00
Nikola Knezevic
06bdc8f85c Added the InfluxDBNotification plugin including a NixOS test 
This adds a `InfluxDBNotification` plugin which is configured as:

```
<influxdb>
  url = http://127.0.0.1:8086
  db = hydra
</influxdb>
```

which will write a notification for every finished job to the
configured database in InfluxDB looking like:

```
hydra_build_status,cached=false,job=job,jobset=default,project=sample,repo=default,result=success,status=success,system=x86_64-linux build_id="1",build_status=0i,closure_size=584i,duration=0i,main_build_id="1",queued=0i,size=168i 1564156212
```
 2019-07-26 17:47:03 +02:00
Antoine Eiche
d1e590af1f hydra-server: add limit parameter to the search path 
This allows a client to set a limit to the search results it wants to
get:

    http://hydra.nixos.org/search?query=emacs&limit=1

This returns only 1 results (while the default is 500).
 2019-06-19 14:30:32 +02:00
Antoine Eiche
cb1fce21ba hydra-server: set a limit on builds and buildoutputs search 
This patch adds a limit statement for Postgresql queries on `builds`
and `buildsoutputs` tables.
 2019-06-19 14:28:55 +02:00
Antoine Eiche
778fc03570 Allow to search builds by hash 
Currently, a full store path has to be provided to search in
builds. This patch permits to search jobs with a output path or
derivation hash.

Usecase: we are building Docker images with Hydra. The tag of the
Docker image is the hash of the image output path. This patch would
allow us to find back the build job from the tag of a running
container image.
 2019-06-05 11:56:42 +02:00
Robin Gloster
f4e7c104ff Create a gitlab status plugin 
This plugin expects as inputs to a jobset the following:
 - gitlab_status_repo => Name of the repository input pointing to that
   status updates should be POST'ed, i.e. the jobset has a git input
   "nixexprs": "https://gitlab.example.com/project/nixexprs", in which
   case "gitlab_status_repo" would be "nixexprs".
 - gitlab_project_id => ID of the project in Gitlab, i.e. in the above
   case the ID in gitlab of "nixexprs"
 2019-04-11 16:40:44 +02:00
Graham Christensen
0e337e6f9c Merge pull request #644 from input-output-hk/declarative-jobset-error-message 
improve the error messages when invalid declarative jobsets are defined
 2019-03-21 14:25:23 -04:00
Michael Bishop
c741576563 improve the error messages when invalid declarative jobsets are defined 
(cherry picked from commit 7568b89a1a9da3a58a0cdddc7b5bcea7bb6209d8)
 2019-03-20 15:27:37 -04:00
Michael Bishop
3ad091faf3 allow using a shorter context and increase hydra-notify debug 
(cherry picked from commit 1c76ad393669af2f728fd519a050f417319412a6)
 2019-03-20 15:22:24 -04:00
Graham Christensen
8a41ea5f60 Merge pull request #571 from kquick/moreinfo 
Additional helpful information in error messages.
 2019-03-18 15:18:14 -05:00
Graham Christensen
215ca5da9c Merge pull request #607 from nlewo/json-search 
Add JSON search API endpoint
 2019-03-18 15:08:32 -05:00
Kevin Quick
2e225ba7c8 Do not attempt to report dir for grab command failure if not specified. 2019-03-17 23:15:24 -07:00
Graham Christensen
88a92256e1 Merge pull request #636 from Ma27/serve-json-for-evals-and-machines 
Serialize data as JSON with `Accept: application/json`
 2019-03-17 17:17:17 -04:00
Maximilian Bosch
1cbbc6c52c Serialize data as JSON with Accept: application/json 
Similar to #607. According to the Catalyst[1] docs it's possible to
specify a data structure that is supposed to be serialized when
requesting i.e. a JSON response.

[1] https://metacpan.org/pod/Catalyst::Controller::REST#status_ok
 2019-02-14 01:22:48 +01:00
Samuel Dionne-Riel
04ff9e217b Add hydra-dev-server which uses the classic Catalyst server 
This, in turns allows

 - Using --restart for reloading the perl code
 - Printing traces on error
 2019-01-22 20:34:21 -05:00
Eelco Dolstra
e0d8dcfe2d Merge pull request #619 from samueldr/feature/lazy_errors 
Adds error messages to lazy tabs
 2019-01-20 23:22:17 +01:00
Shea Levy
b298ba4bbe Merge branch 'pr-gitlabpulls' of git://github.com/nlewo/hydra 2019-01-11 10:08:23 -05:00
Samuel Dionne-Riel
ed85daf2ac User: jobs tab returns its error as a lazy error. 2018-12-01 13:40:41 -05:00
Samuel Dionne-Riel
9986053e73 Controllers: allows lazy tabs to return custom errors. 2018-12-01 13:40:41 -05:00
Antoine Eiche
d9253543e4 plugin/GitLabPulls: support for using a personal access token (PAT) 
In order to access protected or private repositories. Using the target
repository URL along with the merge-request ref instead of the source
repository url and branch is necessary to avoid running into issues if
the source repository is not actually accessible to the user Hydra is
authenticating as.

Thanks Alexei Robyn for this patch.
 2018-11-20 16:27:40 +01:00
Rob Vermaas
14d5577bf8 Add duration to Slack notification. 2018-11-20 14:57:50 +01:00
Eelco Dolstra
cd234f6a14 Merge pull request #529 from bennofs/feat-all-builds 
feat: add /eval/<id>/builds endpoint
 2018-11-19 17:36:19 +01:00
Michael Bishop
7916c6b185 [DEVOPS-1126] throttle github status calls to remain under api ratelimits 2018-11-09 12:06:37 -04:00
Eelco Dolstra
1c44de1779 Merge pull request #567 from phile314/bitbucket_pulls 
Add BitBucket pull request support
 2018-11-06 10:48:37 +01:00
Antoine Eiche
0d2a2d8923 Add json output for the search API endpoint 
This commit also add a test of this feature.

Note the search JSON output doesn't contain any jobs because they can
not be exported to JSON yet.

The JSON output on a search query matching a build looks like:

```
{
  "builds": [
    {
      "buildoutputs": {
        "out": {
          "path": "/nix/store/wdag3pznrvqk01byk989irg7rq3q2a2c-job"
        }
      },
      "finished": 0,
      "releasename": null,
      "starttime": null,
      "project": "sample",
      "buildproducts": {},
      "timestamp": 1541007629,
      "buildstatus": null,
      "nixname": "job",
      "drvpath": "/nix/store/n9zqndn7j7nyr6gg3bmxvw26cfmdwv2n-job.drv",
      "job": "job",
      "id": 1,
      "stoptime": null,
      "priority": 100,
      "system": "x86_64-linux",
      "jobsetevals": [
        1
      ],
      "jobset": "default",
      "buildmetrics": {}
    }
  ],
  "projects": [],
  "jobsets": [],
  "buildsdrv": []
}
```
 2018-11-01 09:31:15 +01:00
Antoine Eiche
68aad22d19 Add GitlabPulls input plugin 2018-10-15 14:45:26 +02:00
Andreas Rammhold
63a294d4ca allow users with 'restart-jobs' role to restart individual builds 2018-10-04 21:59:42 +02:00
Kevin Quick
35bcab74ed Fix darcs input to use darcs-specific SCM cache dir. 
Currently re-using the git cache dir which could cause overlap problems.
 2018-09-09 22:04:32 -07:00
Nathan van Doorn
a77954be4d Allow for precisely one instance of RunCommand plugin 2018-08-21 15:52:41 +01:00
Eelco Dolstra
e122d3bef3 RunCommand: Return metrics as a float 
Apparently, DBIx::Class doesn't handle columns with type 'double
precision' properly.
 2018-08-02 12:31:28 +02:00
Eelco Dolstra
6db2cbf094 Add a plugin to execute arbitrary commands when a build finishes 
The plugin can be configured using one or more <runcommand> sections
in hydra.conf, e.g.

<runcommand>
  command = echo Build finished
</runcommand>

Optionally, the command can be executed for specific
projects/jobsets/jobs:

  job = patchelf:master:tarball

or

  job = patchelf:*:*

The default is *:*:*.

The command is executed with the environment variable $HYDRA_JSON
pointing to a JSON file containing info about the build, e.g.

  {
    "build": 3772978,
    "buildStatus": 0,
    "drvPath": "/nix/store/9y4h1fyx9pl3ic08i2f09239b90x1lww-patchelf-tarball-0.8pre894_ed92f9f.drv",
    "event": "buildFinished",
    "finished": 1,
    "job": "tarball",
    "jobset": "master",
    "metrics": [
      {
        "name": "random1",
        "unit": null,
        "value": "20282"
      },
      {
        "name": "random2",
        "unit": "KiB",
        "value": "6664"
      }
    ],
    "outputs": [
      {
        "name": "out",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f"
      }
    ],
    "products": [
      {
        "defaultPath": "",
        "fileSize": 148216,
        "name": "patchelf-0.8pre894_ed92f9f.tar.gz",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f/tarballs/patchelf-0.8pre894_ed92f9f.tar.gz",
        "productNr": 4,
        "sha1hash": "9f27d18382436a7f743f6c2f6ad66e1b536ab4c8",
        "sha256hash": "b04faef2916c411f10711b58ea26965df7cb860ca33a87f1e868051b874c44b3",
        "subtype": "source-dist",
        "type": "file"
      },
      {
        "defaultPath": "",
        "fileSize": 121279,
        "name": "patchelf-0.8pre894_ed92f9f.tar.bz2",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f/tarballs/patchelf-0.8pre894_ed92f9f.tar.bz2",
        "productNr": 3,
        "sha1hash": "7a664841fb779dec19023be6a6121e0398067b7c",
        "sha256hash": "c81e36099893f541a11480f869fcdebd2fad3309900519065c8745f614dd024a",
        "subtype": "source-dist",
        "type": "file"
      },
      {
        "defaultPath": "README",
        "fileSize": null,
        "name": "",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f",
        "productNr": 2,
        "sha1hash": null,
        "sha256hash": null,
        "subtype": "readme",
        "type": "doc"
      },
      {
        "defaultPath": "",
        "fileSize": 6230,
        "name": "README",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f/README",
        "productNr": 1,
        "sha1hash": "dc6bb09093183ab52d7e6a35b72d179869bd6fbf",
        "sha256hash": "5371aee9de0216b3ea2d5ea869da9d5ee441b99156a99055e7e11e7a705f7920",
        "subtype": "readme",
        "type": "doc"
      }
    ],
    "project": "patchelf",
    "startTime": 1533137091,
    "stopTime": 1533137094,
    "timestamp": 1533136076
  }

So for example, the following command:

  command = echo Build $(jq -r .build $HYDRA_JSON) \($(jq -r .project $HYDRA_JSON):$(jq -r .jobset $HYDRA_JSON):$(jq -r .job $HYDRA_JSON)\) finished, metrics: $(jq -r '.metrics[].value' $HYDRA_JSON).

will print

  Build 3772978 (patchelf:master:tarball) finished, metrics: 20282 6664.
 2018-08-01 19:43:50 +02:00
Kevin Quick
e523e8a643 Additional helpful information in error messages. 2018-06-29 17:41:23 -07:00
Philipp Hausmann
f738f6feb3 Add BitBucket pull request support 2018-06-22 11:22:54 +02:00
Rob Vermaas
879333fd3c Make /api/push-github public for private Hydra instances. This makes it possible to use webhooks to trigger evaluations for private Hydra instances. 2018-06-14 17:23:01 +02:00
Shea Levy
70eb67fe5b Allow channels to be usefully imported by other nix expressions. 2018-05-25 11:22:20 -04:00
Michael Bishop
4015bca225 fix a stray newline in a path 2018-03-16 13:04:48 -03:00
Eelco Dolstra
85524b2d0f Disable jobset deletion 
This is a good way to make Hydra hang. (E.g. we had a deletion of
nixos:gcc-7 running for > 12 hours and blocking UPDATE statements from
hydra-queue-runner.) Generally it's better to just disable/hide an old
jobset anyway.
 2018-03-13 14:05:55 +01:00
Eelco Dolstra
02888105a8 Add a hydra.conf option to enable email notification 
Note that it's disabled by default.
 2018-03-12 14:04:23 +01:00