Commit Graph

2214 Commits

Author SHA1 Message Date
d1e590af1f hydra-server: add limit parameter to the search path
This allows a client to set a limit to the search results it wants to
get:

    http://hydra.nixos.org/search?query=emacs&limit=1

This returns only 1 results (while the default is 500).
2019-06-19 14:30:32 +02:00
cb1fce21ba hydra-server: set a limit on builds and buildoutputs search
This patch adds a limit statement for Postgresql queries on `builds`
and `buildsoutputs` tables.
2019-06-19 14:28:55 +02:00
7935cffd6b Create a pg_trgm index on builds.drvpath
The search query uses the LIKE operator which requires a sequential
scan (it can't use the already existing B-tree index). This new
index (trigram) avoids a sequential scan of the builds table when the
LIKE operator is used.

Here is the analyze of a request on the builds table with this index:

     explain analyze select * from builds where drvpath like '%k3r71gz0gv16ld8rhcp2bb8gb5w1xc4b%';
                                                                 QUERY PLAN
     -----------------------------------------------------------------------------------------------------------------------------------
      Bitmap Heap Scan on builds  (cost=128.00..132.01 rows=1 width=492) (actual time=0.070..0.077 rows=1 loops=1)
        Recheck Cond: (drvpath ~~ '%k3r71gz0gv16ld8rhcp2bb8gb5w1xc4b%'::text)
        ->  Bitmap Index Scan on indextrgmbuildsondrvpath  (cost=0.00..128.00 rows=1 width=0) (actual time=0.047..0.047 rows=3 loops=1)
              Index Cond: (drvpath ~~ '%k3r71gz0gv16ld8rhcp2bb8gb5w1xc4b%'::text)
      Total runtime: 0.206 ms
     (5 rows)
2019-06-06 14:28:25 +02:00
778fc03570 Allow to search builds by hash
Currently, a full store path has to be provided to search in
builds. This patch permits to search jobs with a output path or
derivation hash.

Usecase: we are building Docker images with Hydra. The tag of the
Docker image is the hash of the image output path. This patch would
allow us to find back the build job from the tag of a running
container image.
2019-06-05 11:56:42 +02:00
ff64583d07 Add evaluation error status to evaluation time on project jobset tab. 2019-05-06 14:08:32 +02:00
f4e7c104ff Create a gitlab status plugin
This plugin expects as inputs to a jobset the following:
 - gitlab_status_repo => Name of the repository input pointing to that
   status updates should be POST'ed, i.e. the jobset has a git input
   "nixexprs": "https://gitlab.example.com/project/nixexprs", in which
   case "gitlab_status_repo" would be "nixexprs".
 - gitlab_project_id => ID of the project in Gitlab, i.e. in the above
   case the ID in gitlab of "nixexprs"
2019-04-11 16:40:44 +02:00
0e337e6f9c Merge pull request #644 from input-output-hk/declarative-jobset-error-message
improve the error messages when invalid declarative jobsets are defined
2019-03-21 14:25:23 -04:00
c741576563 improve the error messages when invalid declarative jobsets are defined
(cherry picked from commit 7568b89a1a9da3a58a0cdddc7b5bcea7bb6209d8)
2019-03-20 15:27:37 -04:00
3ad091faf3 allow using a shorter context and increase hydra-notify debug
(cherry picked from commit 1c76ad393669af2f728fd519a050f417319412a6)
2019-03-20 15:22:24 -04:00
8a41ea5f60 Merge pull request #571 from kquick/moreinfo
Additional helpful information in error messages.
2019-03-18 15:18:14 -05:00
215ca5da9c Merge pull request #607 from nlewo/json-search
Add JSON search API endpoint
2019-03-18 15:08:32 -05:00
2e225ba7c8 Do not attempt to report dir for grab command failure if not specified. 2019-03-17 23:15:24 -07:00
0721f6623f Merge pull request #627 from samueldr/hydra-eval-jobs/warn-max-heap-size
hydra-eval-jobs: Warn and resets maxHeapSize on bad values
2019-03-17 19:36:53 -04:00
baf828a560 Merge pull request #624 from kquick/job_headers
Add job status headers (web useability for red/green colorblindness).
2019-03-17 17:24:38 -04:00
88a92256e1 Merge pull request #636 from Ma27/serve-json-for-evals-and-machines
Serialize data as JSON with `Accept: application/json`
2019-03-17 17:17:17 -04:00
21a4433fec Merge branch 'master' into job_headers 2019-03-17 17:16:37 -04:00
c8fa7ffc12 Merge pull request #629 from Ma27/fix-maintainer-notifications
hydra-eval-jobs: fix maintainer resolution
2019-03-17 17:12:55 -04:00
014e5aa10d Merge pull request #628 from dasJ/patch-1
Support $X-Request-Base for the Logo
2019-03-17 16:22:53 -04:00
1cbbc6c52c Serialize data as JSON with Accept: application/json
Similar to #607. According to the Catalyst[1] docs it's possible to
specify a data structure that is supposed to be serialized when
requesting i.e. a JSON response.

[1] https://metacpan.org/pod/Catalyst::Controller::REST#status_ok
2019-02-14 01:22:48 +01:00
5dd4736aed Merge pull request #632 from nlewo/localhost-error-message
hydra-queue-runner: better error message if nix-store can not be started
2019-01-23 10:46:30 +01:00
9a73ec6455 hydra-queue-runner: better error message if nix-store can not be started
The hydra-queue-runner opens a connection to the builder. If the
builder is 'localhost' it starts `nix-store`, otherwise it starts
'ssh'.

Currently, if the hydra-queue-runner can not start `nix-store` (not in
the PATH for instance), the error message is:

  cannot connect to ‘localhost’: error: cannot start ssh: No such file
  or directory

This is not useful since ssh is actually not started:/

With this patch the error message is now:

  cannot connect to ‘localhost’: error: cannot start nix-store: No such file
  or directory
2019-01-23 10:42:47 +01:00
f5686e0958 Merge pull request #633 from samueldr/feature/dev-server
Add hydra-dev-server which uses the classic Catalyst server
2019-01-23 10:39:47 +01:00
04ff9e217b Add hydra-dev-server which uses the classic Catalyst server
This, in turns allows

 - Using --restart for reloading the perl code
 - Printing traces on error
2019-01-22 20:34:21 -05:00
a16d0c7d6b Makes user-facing dates local when possible
Also ensures it makes available the hydra timezone in the title of the
dates.
2019-01-22 17:53:46 -05:00
2430b12ec9 Adds moment.js for client-side date manipulations 2019-01-21 19:57:22 -05:00
e0d8dcfe2d Merge pull request #619 from samueldr/feature/lazy_errors
Adds error messages to lazy tabs
2019-01-20 23:22:17 +01:00
b298ba4bbe Merge branch 'pr-gitlabpulls' of git://github.com/nlewo/hydra 2019-01-11 10:08:23 -05:00
aa87e7a52e hydra-eval-jobs: fix maintainer resolution
Some time ago the data structure for maintainer descriptions in
`nixpkgs` changed from a simple attr set with maintainer emails as
values to an attribute set where the maintainer' nick is associated to
an attribute set with email, GitHub handle and full name.

Hydra can either parse a Nix list or fetches `shortName` from the
associated attribute set (which is used for `meta.licenses` as each
value in it contains a `shortName`). This behavior needs to be
replicated for maintainers to retrieve the emails for `hydra-notify`.

This change is backwards-compatible since `queryMetaStrings` is still
able to understand lists, so old versions of `nixpkgs` or packages using
the old maintainer data structure remain usable.
2018-12-29 14:47:03 +01:00
3a70688985 Support $X-Request-Base for the Logo 2018-12-27 17:08:20 +01:00
dd188d3035 hydra-eval-jobs: Warn and resets maxHeapSize on bad values
This is because setting only the initial heap size to more than
the default value (or the configured value) will cause all initial evals
until maxHeapSize expands to the given value to abort.

The 1.1 multiplier comes from the the configured defaults on NixOS' hydra,
and from the previous multiplier used before
7876cf677c.
2018-12-21 22:57:07 -05:00
423c0440ea Typo 2018-12-20 12:07:02 +01:00
8315e20ab5 Lighten delta mark to blend better with other column headers. 2018-12-16 22:34:22 -08:00
dfce2ad69a Use rowspans to align Jobs reference in evaluations rendering 2018-12-16 22:26:44 -08:00
0e7593a4a1 Remove blank lines, consistent attribute quoting, use svg for delta header, column sizes. 2018-12-15 22:25:02 -08:00
5716e69767 Add job status headers (web useability for red/green colorblindness). 2018-12-15 11:58:26 -08:00
ed85daf2ac User: jobs tab returns its error as a lazy error. 2018-12-01 13:40:41 -05:00
9986053e73 Controllers: allows lazy tabs to return custom errors. 2018-12-01 13:40:41 -05:00
d9253543e4 plugin/GitLabPulls: support for using a personal access token (PAT)
In order to access protected or private repositories. Using the target
repository URL along with the merge-request ref instead of the source
repository url and branch is necessary to avoid running into issues if
the source repository is not actually accessible to the user Hydra is
authenticating as.

Thanks Alexei Robyn for this patch.
2018-11-20 16:27:40 +01:00
14d5577bf8 Add duration to Slack notification. 2018-11-20 14:57:50 +01:00
cd234f6a14 Merge pull request #529 from bennofs/feat-all-builds
feat: add /eval/<id>/builds endpoint
2018-11-19 17:36:19 +01:00
7916c6b185 [DEVOPS-1126] throttle github status calls to remain under api ratelimits 2018-11-09 12:06:37 -04:00
1c44de1779 Merge pull request #567 from phile314/bitbucket_pulls
Add BitBucket pull request support
2018-11-06 10:48:37 +01:00
0d2a2d8923 Add json output for the search API endpoint
This commit also add a test of this feature.

Note the search JSON output doesn't contain any jobs because they can
not be exported to JSON yet.

The JSON output on a search query matching a build looks like:

```
{
  "builds": [
    {
      "buildoutputs": {
        "out": {
          "path": "/nix/store/wdag3pznrvqk01byk989irg7rq3q2a2c-job"
        }
      },
      "finished": 0,
      "releasename": null,
      "starttime": null,
      "project": "sample",
      "buildproducts": {},
      "timestamp": 1541007629,
      "buildstatus": null,
      "nixname": "job",
      "drvpath": "/nix/store/n9zqndn7j7nyr6gg3bmxvw26cfmdwv2n-job.drv",
      "job": "job",
      "id": 1,
      "stoptime": null,
      "priority": 100,
      "system": "x86_64-linux",
      "jobsetevals": [
        1
      ],
      "jobset": "default",
      "buildmetrics": {}
    }
  ],
  "projects": [],
  "jobsets": [],
  "buildsdrv": []
}
```
2018-11-01 09:31:15 +01:00
adf59a3959 Merge pull request #604 from andir/single-build-restart
allow users with 'restart-jobs' role to restart individual builds
2018-10-31 12:26:35 +01:00
8d26144121 Fix building against nix master 2018-10-30 14:41:21 +01:00
68aad22d19 Add GitlabPulls input plugin 2018-10-15 14:45:26 +02:00
63a294d4ca allow users with 'restart-jobs' role to restart individual builds 2018-10-04 21:59:42 +02:00
8bffbb7928 evalSettings deprecated, submodule fix 2018-10-03 00:45:42 -04:00
35bcab74ed Fix darcs input to use darcs-specific SCM cache dir.
Currently re-using the git cache dir which could cause overlap problems.
2018-09-09 22:04:32 -07:00
b7c42a38a3 Merge pull request #575 from Ma27/fix-tab-jobset-view
jobset view: jump back to the proper tab when displaying all failed jobsets
2018-09-06 21:11:59 +02:00