Commit Graph

3042 Commits

Author SHA1 Message Date
ccd046ca3d Keep track of the number of unsupported steps
(cherry picked from commit 45ffe578b6)
2020-03-31 22:19:03 +02:00
4417f9f260 Abort unsupported build steps
If we don't see machine that supports a build step for
'max_unsupported_time' seconds, the step is aborted. The default is 0,
which is appropriate for Hydra installations that don't provision
missing machines dynamically.

(cherry picked from commit f5cdbfe21d)
2020-03-31 22:19:01 +02:00
68a59f34a0 Remove tabs
(cherry picked from commit 5b731004da)
2020-03-31 22:18:58 +02:00
7889aef792 hydra-eval-jobs: Mirror eval errors in STDERR
Otherwise, errors will not be shown to end-users, which makes debugging
long evals pretty much impossible.

(cherry picked from commit 76299b9174)
2020-03-31 12:48:33 +02:00
76299b9174 hydra-eval-jobs: Mirror eval errors in STDERR
Otherwise, errors will not be shown to end-users, which makes debugging
long evals pretty much impossible.
2020-03-30 16:11:44 -04:00
15ae932488 Don't spam the journal with hydra-queue-runner status dumps 2020-03-26 15:30:37 +01:00
45ffe578b6 Keep track of the number of unsupported steps 2020-03-26 15:27:13 +01:00
f5cdbfe21d Abort unsupported build steps
If we don't see machine that supports a build step for
'max_unsupported_time' seconds, the step is aborted. The default is 0,
which is appropriate for Hydra installations that don't provision
missing machines dynamically.
2020-03-26 15:00:04 +01:00
5b731004da Remove tabs 2020-03-26 14:59:52 +01:00
1bee6e3d8a Add debug logging
This will help us track potential problems with the plugin.
2020-03-26 13:27:44 +01:00
986fde8888 Refactor code
Extract the conditions before the loop, as they do not change due to channel
definition.
2020-03-26 13:27:44 +01:00
956f009672 Add documentation for SlackNotification plugin 2020-03-26 13:27:44 +01:00
d0688b93e1 Merge remote-tracking branch 'origin/master' into flake 2020-03-26 11:44:11 +01:00
12cc46cdb3 fixup: hydra-init: correct reference to hydra-backill-ids 2020-03-24 11:22:14 -04:00
73694087a0 Use the stopsign as icon for timeouts and exceeded log-limits
When I browse failed builds in a jobset-eval on Hydra, I regularly
mistake actual build-failures with temporary issues like timeouts (that
probably disappear at the next eval).

To prevent this kind of issue, I figured that using the stopsign-svg for
builds with timeouts or exceeded log-limits is a reasonable choice for
the following reasons:

* A user can now distinguish between actual build-errors (like
  compilation-failures or oversized outputs) and (usually) temporary issues
  (like a bloated log or a timeout).

* The stopsign is also used for aborted jobs that are shown in a
  different tab and can't be confused with timeouts for that reason.
2020-03-21 21:08:33 +01:00
01ae944c80 Add host tag to InfluxDB metrics
This should help us discern machines in environments with multiple Hydra deployments.
2020-03-12 14:23:07 +01:00
4b5bb4e760 Merge remote-tracking branch 'origin/master' into flake 2020-03-04 15:28:23 +01:00
123bee1db5 Restore job type checking 2020-03-04 15:16:26 +01:00
69a6f3448a Fix calling job functions
Fixes #718.
2020-03-04 15:16:04 +01:00
3cc1deb125 Merge pull request #721 from grahamc/one-by-one
hydra-evaluator: add a 'ONE_AT_A_TIME' evaluator style
2020-03-04 08:44:43 +01:00
994430b94b treewide: allow nix command 2020-03-03 22:52:20 -05:00
117b9ecef1 Nix.pm: readNixFile: pass «--experimental-features nix-command»
Declarative jobsets were broken by the Nix update, causing
nix cat-file to break silently.

This commit restores declarative jobsets, based on top of a commit
making it easier to see what broke.
2020-03-03 22:36:21 -05:00
113a312f67 handleDeclarativeJobsetBuild: handle errors from readNixFile 2020-03-03 22:32:13 -05:00
5fae9d96a2 hydra-evaluator: add a 'ONE_AT_A_TIME' evaluator style
In the past, jobsets which are automatically evaluated are evaluated
regularly, on a schedule. This schedule means a new evaluation is
created every checkInterval seconds (assuming something changed.)

This model works well for architectures where our build farm can
easily keep up with demand.

This commit adds a new type of evaluation, called ONE_AT_A_TIME, which
only schedules a new evaluation if the previous evaluation of the
jobset has no unfinished builds.

This model of evaluation lets us have 'low-tier' architectures.

For example, we could now have a jobset for ARMv7l builds, where
the buildfarm only has a single, underpowered ARMv7l builder.
Configuring that jobset as ONE_AT_A_TIME will create an evaluation
and then won't schedule another evaluation until every job of
the existing evaluation is complete.

This way, the cache will have a complete collection of pre-built
software for some commits, but the underpowered architecture will
never become backlogged in ancient revisions.
2020-03-03 19:28:44 -05:00
eaa65f51f4 hydra-evaluator: make the logic of the scheduler easier to read 2020-03-03 18:17:21 -05:00
eb5873ae53 Fix build 2020-02-20 11:19:45 +01:00
c642f787ee hydra-eval-jobs: Parallelize
(cherry picked from commit be8eb9d00d)
2020-02-20 10:28:58 +01:00
adf61e5cf8 Fix build
(cherry picked from commit 639c660abf)
2020-02-20 10:26:45 +01:00
e4f5156c41 Build against nix-master
(cherry picked from commit e7f2139e25)
2020-02-20 10:24:04 +01:00
b790a00729 Disable deprecation warnings
(cherry picked from commit 950e8bef6c)
2020-02-20 10:22:21 +01:00
2a50daa377 Update aggregate handling
(cherry picked from commit cf961ac893)
2020-02-20 10:13:39 +01:00
15187b059b Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.

(cherry picked from commit 23c9ca3e94)
2020-02-20 09:58:12 +01:00
be8eb9d00d hydra-eval-jobs: Parallelize 2020-02-19 21:10:22 +01:00
6f1d68bda4 Revert "hydra-eval-jobs -> nix eval-hydra-jobs"
This reverts commit 345512a6d0.
2020-02-19 20:36:52 +01:00
cf961ac893 Update aggregate handling 2020-02-17 16:33:25 +01:00
68bdf834ee Fix hydra-evaluator --unlock 2020-02-15 16:40:16 +01:00
345512a6d0 hydra-eval-jobs -> nix eval-hydra-jobs 2020-02-15 15:59:34 +01:00
23c9ca3e94 Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.
2020-02-15 15:59:34 +01:00
639c660abf Fix build 2020-02-15 14:54:35 +01:00
ae1b9face3 Merge remote-tracking branch 'origin/master' into flake 2020-02-11 12:52:46 -05:00
027668f0db hydra.sql: add an index for slow queries in production
These queries used to use (jobset, project) based indexes,
and the addition of jobset_id makes most of those indexes
unusable now.
2020-02-11 12:52:28 -05:00
53896ff19b Merge remote-tracking branch 'origin/master' into flake 2020-02-11 08:36:40 -05:00
834793468f fixup: d'oh, make the migrations from #710 part-2 sequential 2020-02-11 08:36:14 -05:00
881b7449fd Merge remote-tracking branch 'origin/master' into flake 2020-02-11 14:23:16 +01:00
2637a7ad76 Builds: index literally what latest-finished queries 2020-02-11 07:06:21 -05:00
f0f41eaaff LatestSucceededForJob{,set}: Filter with jobset_id 2020-02-11 07:06:20 -05:00
66fbbd9692 Jobs.builds: Fetch via Jobs.jobset_id 2020-02-11 07:06:20 -05:00
7c71f9df28 Jobsets.builds: Fetch via Jobsets.id 2020-02-11 07:06:20 -05:00
3c392b8cd8 Jobsets.jobs: Fetch via Jobsets.id 2020-02-11 07:06:20 -05:00
8ef08f1385 Builds.jobset_id: make not-null 2020-02-11 07:06:20 -05:00