2659 Commits

Author SHA1 Message Date
Eelco Dolstra
96a514c169
Remove the "releases" feature
We haven't used this in many years (it was really only used for nix
and patchelf releases).
2020-05-06 12:39:21 +02:00
Ben Wolsieffer
f020f7efef
hydra-queue-runner: don't try to distribute builds on localhost 2020-05-03 00:05:52 -04:00
Emery Hemingway
e93c36aab1 SoTest: read credentials from file 2020-04-26 12:12:04 +05:30
Nikola Knezevic
f03e7ef800 Add GithubRefs plugin
This plugin is a counterpart to GithubPulls plugin. Instead of fetching pull
requests, it will fetch all references (branches and tags) that start with a
particular prefix.

The plugin is a copy of GithubPulls plugin with appropriate changes to call the
right API and parse the config matching the need.
2020-04-23 10:45:37 +02:00
Emery Hemingway
a63e349476 Add SoTest plugin
https://opensource.sotest.io/
https://docs.sotest.io/
2020-04-21 15:25:44 +05:30
Maximilian Bosch
721c764951
Remove Hydra::Helper::nix::txn_do from the Perl code
To quote the function's comment:

  Awful hack to handle timeouts in SQLite: just retry the transaction.
  DBD::SQLite *has* a 30 second retry window, but apparently it
  doesn't work.

Since SQLite is now dropped entirely, this wrapper can be removed
completely.
2020-04-16 00:42:40 +02:00
Maximilian Bosch
efcbc08686
Get rid of dependency to SQLite
SQLite isn't properly supported by Hydra for a few years now[1], but
Hydra still depends on it. Apart from a slightly bigger closure this can
cause confusion by users since Hydra picks up SQLite rather than
PostgreSQL by default if HYDRA_DBI isn't configured properly[2]

[1] 78974abb6943533c3d94c7bd3282bf738b7be61c
[2] https://logs.nix.samueldr.com/nixos-dev/2020-04-10#3297342;
2020-04-16 00:42:40 +02:00
Graham Christensen
7b705758ec
Merge pull request #732 from Ma27/fix-build
Fix build against latest Nix
2020-04-09 09:02:45 -04:00
Bas van Dijk
6e358189ad Separate the build IDs in the build_finished payload with tabs
hydra-notify splits the payload on tabs so we shouldn't separate the
IDs with spaces.
2020-04-08 12:05:25 +02:00
Maximilian Bosch
2f9d422172
Fix build against latest Nix 2020-04-07 13:55:38 +02:00
Eelco Dolstra
4cabb37ebd
Merge pull request #730 from NixOS/flake
Flake support
2020-04-07 11:18:38 +02:00
Kevin Quick
a055796ef5
Merge branch 'master' into libpqxx_undeprecate 2020-04-01 11:54:41 -07:00
Eelco Dolstra
2d092a6fbc
Merge pull request #702 from kquick/fix_api_push
Handle case where jobset has no defined errormsg for api/jobsets
2020-04-01 13:09:05 +02:00
Eelco Dolstra
8564089186
Merge pull request #707 from basvandijk/fix-aggregate-status
Fix printing aggregate status
2020-04-01 13:04:53 +02:00
Eelco Dolstra
6c0bb909df
Merge pull request #704 from basvandijk/sleep-after-catch-exception
hydra-queue-runner: sleep 5s after handling an exception
2020-04-01 13:04:31 +02:00
Eelco Dolstra
9668bc01da
Merge pull request #726 from knl/document-slack-notifications-plugin
Document SlackNotifications plugin
2020-04-01 13:04:00 +02:00
Eelco Dolstra
9727892b61 Don't spam the journal with hydra-queue-runner status dumps
(cherry picked from commit 15ae932488512ba235ed2f6f841cc5eb56ba9314)
2020-03-31 22:19:07 +02:00
Eelco Dolstra
ccd046ca3d Keep track of the number of unsupported steps
(cherry picked from commit 45ffe578b695f9de101b30d44d46f12aa0654f10)
2020-03-31 22:19:03 +02:00
Eelco Dolstra
4417f9f260 Abort unsupported build steps
If we don't see machine that supports a build step for
'max_unsupported_time' seconds, the step is aborted. The default is 0,
which is appropriate for Hydra installations that don't provision
missing machines dynamically.

(cherry picked from commit f5cdbfe21d930db43d3812c7d8e87746d6378ef9)
2020-03-31 22:19:01 +02:00
Eelco Dolstra
68a59f34a0 Remove tabs
(cherry picked from commit 5b731004da89a9b94f4ea7945ea6538837a64e41)
2020-03-31 22:18:58 +02:00
Samuel Dionne-Riel
7889aef792
hydra-eval-jobs: Mirror eval errors in STDERR
Otherwise, errors will not be shown to end-users, which makes debugging
long evals pretty much impossible.

(cherry picked from commit 76299b9174d7cba06427220166dc4707b43d407d)
2020-03-31 12:48:33 +02:00
Samuel Dionne-Riel
76299b9174 hydra-eval-jobs: Mirror eval errors in STDERR
Otherwise, errors will not be shown to end-users, which makes debugging
long evals pretty much impossible.
2020-03-30 16:11:44 -04:00
Eelco Dolstra
15ae932488
Don't spam the journal with hydra-queue-runner status dumps 2020-03-26 15:30:37 +01:00
Eelco Dolstra
45ffe578b6
Keep track of the number of unsupported steps 2020-03-26 15:27:13 +01:00
Eelco Dolstra
f5cdbfe21d
Abort unsupported build steps
If we don't see machine that supports a build step for
'max_unsupported_time' seconds, the step is aborted. The default is 0,
which is appropriate for Hydra installations that don't provision
missing machines dynamically.
2020-03-26 15:00:04 +01:00
Eelco Dolstra
5b731004da
Remove tabs 2020-03-26 14:59:52 +01:00
Nikola Knezevic
1bee6e3d8a Add debug logging
This will help us track potential problems with the plugin.
2020-03-26 13:27:44 +01:00
Nikola Knezevic
986fde8888 Refactor code
Extract the conditions before the loop, as they do not change due to channel
definition.
2020-03-26 13:27:44 +01:00
Nikola Knezevic
956f009672 Add documentation for SlackNotification plugin 2020-03-26 13:27:44 +01:00
Eelco Dolstra
d0688b93e1
Merge remote-tracking branch 'origin/master' into flake 2020-03-26 11:44:11 +01:00
Graham Christensen
12cc46cdb3
fixup: hydra-init: correct reference to hydra-backill-ids 2020-03-24 11:22:14 -04:00
Maximilian Bosch
73694087a0
Use the stopsign as icon for timeouts and exceeded log-limits
When I browse failed builds in a jobset-eval on Hydra, I regularly
mistake actual build-failures with temporary issues like timeouts (that
probably disappear at the next eval).

To prevent this kind of issue, I figured that using the stopsign-svg for
builds with timeouts or exceeded log-limits is a reasonable choice for
the following reasons:

* A user can now distinguish between actual build-errors (like
  compilation-failures or oversized outputs) and (usually) temporary issues
  (like a bloated log or a timeout).

* The stopsign is also used for aborted jobs that are shown in a
  different tab and can't be confused with timeouts for that reason.
2020-03-21 21:08:33 +01:00
Nikola Knezevic
01ae944c80 Add host tag to InfluxDB metrics
This should help us discern machines in environments with multiple Hydra deployments.
2020-03-12 14:23:07 +01:00
Eelco Dolstra
4b5bb4e760
Merge remote-tracking branch 'origin/master' into flake 2020-03-04 15:28:23 +01:00
Eelco Dolstra
123bee1db5
Restore job type checking 2020-03-04 15:16:26 +01:00
Eelco Dolstra
69a6f3448a
Fix calling job functions
Fixes #718.
2020-03-04 15:16:04 +01:00
Eelco Dolstra
3cc1deb125
Merge pull request #721 from grahamc/one-by-one
hydra-evaluator: add a 'ONE_AT_A_TIME' evaluator style
2020-03-04 08:44:43 +01:00
Graham Christensen
994430b94b
treewide: allow nix command 2020-03-03 22:52:20 -05:00
Graham Christensen
117b9ecef1
Nix.pm: readNixFile: pass «--experimental-features nix-command»
Declarative jobsets were broken by the Nix update, causing
nix cat-file to break silently.

This commit restores declarative jobsets, based on top of a commit
making it easier to see what broke.
2020-03-03 22:36:21 -05:00
Graham Christensen
113a312f67
handleDeclarativeJobsetBuild: handle errors from readNixFile 2020-03-03 22:32:13 -05:00
Graham Christensen
5fae9d96a2
hydra-evaluator: add a 'ONE_AT_A_TIME' evaluator style
In the past, jobsets which are automatically evaluated are evaluated
regularly, on a schedule. This schedule means a new evaluation is
created every checkInterval seconds (assuming something changed.)

This model works well for architectures where our build farm can
easily keep up with demand.

This commit adds a new type of evaluation, called ONE_AT_A_TIME, which
only schedules a new evaluation if the previous evaluation of the
jobset has no unfinished builds.

This model of evaluation lets us have 'low-tier' architectures.

For example, we could now have a jobset for ARMv7l builds, where
the buildfarm only has a single, underpowered ARMv7l builder.
Configuring that jobset as ONE_AT_A_TIME will create an evaluation
and then won't schedule another evaluation until every job of
the existing evaluation is complete.

This way, the cache will have a complete collection of pre-built
software for some commits, but the underpowered architecture will
never become backlogged in ancient revisions.
2020-03-03 19:28:44 -05:00
Graham Christensen
eaa65f51f4
hydra-evaluator: make the logic of the scheduler easier to read 2020-03-03 18:17:21 -05:00
Eelco Dolstra
eb5873ae53
Fix build 2020-02-20 11:19:45 +01:00
Eelco Dolstra
c642f787ee
hydra-eval-jobs: Parallelize
(cherry picked from commit be8eb9d00d6a3fcac520bc7dfd5740c0b3ade746)
2020-02-20 10:28:58 +01:00
Eelco Dolstra
adf61e5cf8
Fix build
(cherry picked from commit 639c660abfd5de62ecfcd8d3cbc2eb6924c7ec75)
2020-02-20 10:26:45 +01:00
Eelco Dolstra
e4f5156c41
Build against nix-master
(cherry picked from commit e7f2139e251cb73195eea6fb84e2a6167b4db968)
2020-02-20 10:24:04 +01:00
Eelco Dolstra
b790a00729
Disable deprecation warnings
(cherry picked from commit 950e8bef6cd90befcf14e96826053d1d154e39fe)
2020-02-20 10:22:21 +01:00
Eelco Dolstra
2a50daa377
Update aggregate handling
(cherry picked from commit cf961ac8933b76d9f0a3ac3eb49cc08879f5bcc9)
2020-02-20 10:13:39 +01:00
Eelco Dolstra
15187b059b
Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.

(cherry picked from commit 23c9ca3e94669087d463642baea0cf35a0b8d72f)
2020-02-20 09:58:12 +01:00
Eelco Dolstra
be8eb9d00d
hydra-eval-jobs: Parallelize 2020-02-19 21:10:22 +01:00