Commit Graph

3444 Commits

Author SHA1 Message Date
99bfc37764 Don't abort steps that have an unsupported system type
This is necessary because the required system type can become
available later (e.g. by being provisioned by the
auto-scaler). However, in the future, we may want to fail steps if
they have been unsupported for more than a certain amount of time.
2015-08-17 15:10:41 +02:00
ea1eb2e3fb Keep track of requiredSystemFeatures in the machine stats
For example, steps that require the "kvm" feature may require a
different kind of machine to be provisioned. This can also be used to
require performance-sensitive tests to run on a particular kind of
machine, e.g., by setting requiredSystemFeatures to something like
"ec2-i2.8xlarge".
2015-08-17 14:37:57 +02:00
a6e3cb53b9 Use /usr/bin/env to find perl
This is nicer in nix-shell.
2015-08-17 14:18:20 +02:00
d571e44b86 Keep stats for the Hydra auto scaler
"hydra-queue-runner --status" now prints how many runnable and running
build steps exist for each machine type. This allows additional
machines to be provisioned based on the Hydra load.
2015-08-17 13:50:41 +02:00
69e9f73cf6 Use nixUnstable for the tarball
Not needed for building, only for dev-shell.
2015-08-17 11:17:54 +02:00
b0c8eecd37 Merge branch 'build-ng' 2015-08-12 20:32:48 +02:00
48d6f0de2a Fix the API test 2015-08-12 16:07:19 +02:00
62cd9d18b6 Check for a valid number of shares 2015-08-12 15:36:55 +02:00
006d105f93 Fix syntax error 2015-08-12 14:53:34 +02:00
d4759c1da2 hydra-queue-runner: Detect changes to the scheduling shares 2015-08-12 13:17:56 +02:00
2e3899ed27 Don't set Expires header for logs of unfinished builds/steps 2015-08-12 12:22:14 +02:00
576dc0c120 For completeness, re-implement meta.schedulingPriority 2015-08-12 12:05:43 +02:00
b7965df928 Load the queue in order of global priority 2015-08-11 02:14:34 +02:00
97f11baa8d Revive jobset scheduling
(I.e. taking the jobset scheduling share into account.)
2015-08-11 01:31:56 +02:00
cf9516baa4 Merge pull request #231 from shlevy/prev-build-drv-info
Pass along drvPath and outputName for inputs that are previous builds.
2015-08-10 20:23:29 +02:00
1705ca41e7 Remove unneeded camelcase 2015-08-10 13:59:22 -04:00
163e696813 Copy-paste error 2015-08-10 13:52:40 -04:00
882b6b3377 Pass a build's drv path as a store path 2015-08-10 13:48:09 -04:00
ce5ffa9fba Only pass the drv path if it is still valid 2015-08-10 13:47:39 -04:00
08739a2a5a Allow bumping an entire evaluation to the front of the queue
This is mostly useful if we need to prioritize (say) a nixos-stable
eval that has some critical security bug fix.
2015-08-10 18:57:59 +02:00
eb13007fe6 Allow build to be bumped to the front of the queue via the web interface
Builds now have a "Bump up" action. This will cause the queue runner
to prioritise the steps of the build above all other steps.
2015-08-10 16:19:47 +02:00
27182c7c1d Start steps in order of ascending build ID 2015-08-10 16:19:47 +02:00
2a240e458e Pass along drvPath and outputName for inputs that are previous builds.
This allows importing the .drv and getting the same store paths as if the
input had been passed in as nix expressions defining a proper derivation.
2015-08-10 08:02:10 -04:00
52ec971154 Don't render machine name if not applicable to step 2015-08-10 12:54:56 +02:00
593850b956 Fix potential race in dispatcher wakeup 2015-08-10 12:54:55 +02:00
6a1c950e94 Unindent 2015-08-10 11:33:22 +02:00
f21b88e388 Remove superfluous check 2015-08-07 04:20:34 +02:00
f1fbf8c605 Fix race in finishing builds that have been cancelled 2015-08-07 04:18:48 +02:00
90c462a222 Merge remote-tracking branch 'origin/master' into build-ng
Conflicts:
	hydra-module.nix
2015-08-04 14:30:22 +02:00
9e33f07cb8 Merge pull request #229 from shlevy/pass-all-inputs
hydra-eval-jobs: Pass all inputs as 'inputs' arg.
2015-08-04 14:19:23 +02:00
7b7f189b3e Whoops, misread signature 2015-08-04 08:12:24 -04:00
a5b986cf33 Only pass the first value of each input in the inputs argument
Multiple alts are on their way out
2015-08-04 08:10:47 -04:00
4d967dd17a hydra-eval-jobs: Pass all inputs as 'inputs' arg.
If there is no input named 'inputs', hydra-eval-jobs now passes in a set
of lists, where each attribute corresponds to an input defined in the
jobset specification and each list element is a different input alt, as
an argument named 'inputs'.

Among other things, this allows for generic hydra expressions to be
shared amongst projects with similar structures but different sets of
specific inputs.
2015-08-04 07:54:24 -04:00
deee99b518 Merge pull request #226 from shlevy/pass-input-uri-and-type
buildInputToString: Pass along the input's type and urr
2015-08-04 13:01:26 +02:00
d450d08929 buildInputToString: Use inputType attribute instead of type attribute 2015-08-04 06:50:04 -04:00
6f04299411 Allow downloading file by filename
For instance, instead of

  http://.../build/3770911/download/4/source.tar.gz

you can get

  http://.../build/3770911/download/source.tar.gz

This is useful when file names are constant across builds but product
numbers are not.
2015-07-31 17:50:22 +02:00
07634e8862 buildInputToString: Pass along the input's type and urr 2015-07-31 09:47:44 -04:00
ff3f5eb4d8 Fix remote building on Nix 1.10 2015-07-31 03:41:55 +02:00
5b9a288123 Workaround for RemoteStore not supporting cmdBuildDerivation yet 2015-07-31 03:39:20 +02:00
706746d4bf Fix build 2015-07-31 01:45:16 +02:00
4d26546d3c Add support for tracking custom metrics
Builds can now emit metrics that Hydra will store in its database and
render as time series via flot charts. Typical applications are to
keep track of performance indicators, coverage percentages, artifact
sizes, and so on.

For example, a coverage build can emit the coverage percentage as
follows:

  echo "lineCoverage $pct %" > $out/nix-support/hydra-metrics

Graphs of all metrics for a job can be seen at

  http://.../job/<project>/<jobset>/<job>#tabs-charts

Specific metrics are also visible at

  http://.../job/<project>/<jobset>/<job>/metric/<metric>

The latter URL also allows getting the data in JSON format (e.g. via
"curl -H 'Accept: application/json'").
2015-07-31 00:57:30 +02:00
8092149a9f Use nixUnstable 2015-07-30 16:55:11 +02:00
9afaf322b3 Use latest DBIx::Class::Schema::Loader 2015-07-30 16:55:11 +02:00
fed1cc88c0 Merge pull request #225 from aszlig/fix-scmdiff-url
common.tt: Properly generate URL for /api/scmdiff.
2015-07-22 13:00:21 +02:00
36cbdd1bd8 common.tt: Properly generate URL for /api/scmdiff.
If Hydra isn't hosted on https://example.com/ but something like
https://example.com/hydra/, the URL for /api/scmdiff would have ended up
on /api/scmdiff rather than /hydra/api/scmdiff.

This is because we didn't use the URI resolver from the controller,
hence we're using it now to build up the whole URL including the query
string.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-07-22 12:00:15 +02:00
c18fb0ad74 Temporarily disable machines after a connection failure 2015-07-21 15:58:47 +02:00
7e026d35f7 Split hydra-queue-runner.cc more 2015-07-21 15:14:17 +02:00
6ddcd37df1 hydra-queue-runner.service: Set IN_SYSTEMD 2015-07-21 01:54:24 +02:00
5370be9f52 hydra-queue-runner: Use cmdBuildDerivation
See 1511aa9f48 and eda2f36c2a.
2015-07-21 01:54:24 +02:00
62b1b095ad Handle IP addresses as machine names 2015-07-21 01:54:24 +02:00