ahuston-0/hydra
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
2,537 Commits 3 Branches 0 Tags
Commit Graph

333 Commits

Author SHA1 Message Date
Eelco Dolstra
e003665146 Split timeSpent query into 2 separate queries, as postgresql isn't able to figure out a decent query plan. With 120k jobs in queue, this makes some queries go from 100s to 1-2s. 2015-05-01 12:28:29 +02:00
Eelco Dolstra
01cd6397cb Better error message 2015-04-14 15:16:24 +02:00
Eelco Dolstra
63306aaf5a hydra-evaluator: Add some debug code 2015-04-09 17:35:04 +02:00
Eelco Dolstra
9e664cf8b0 Fix not-null constraint violation inserting build step 
DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR:  null value in column "machine" violates not-null constraint
 2015-04-07 13:34:00 +02:00
Eelco Dolstra
ad2b7646ac Don't show missing paths in logs 2015-03-04 15:44:04 +01:00
Eelco Dolstra
adc72d2409 Record which build a failed build step was propagated from 2015-02-25 16:42:32 +01:00
Eelco Dolstra
48af914e28 When propagating failure, propagate the duration and machine 
Previously the duration would just show as "0" and you would have to
search for the original build to get the duration.
 2015-02-25 16:21:54 +01:00
Eelco Dolstra
a12135fc51 Don't use Perl's -w flag 2014-12-12 17:39:52 +01:00
Eelco Dolstra
6a0b9a3476 hydra-build: Handle new trace messages 
Now build step duration no longer includes network overhead (i.e.
time to copy closures to the build machine).
 2014-12-12 17:20:50 +01:00
Eelco Dolstra
c0ca5489e1 Don't use given/when 
These give warnings in Perl >= 5.18:

  given is experimental at /home/hydra/src/hydra/src/lib/Hydra/Helper/CatalystUtils.pm line 241.
  when is experimental at /home/hydra/src/hydra/src/lib/Hydra/Helper/CatalystUtils.pm line 242.
  ...
 2014-12-12 11:27:17 +01:00
Eelco Dolstra
8523130ebb Use Email::MIME instead of Email::Simple 
Email::Simple cannot handle non-ASCII characters.

Fixes #191.
 2014-11-19 14:45:46 +01:00
Eelco Dolstra
5a7efc0469 Fix aggregate handling 2014-10-01 15:34:05 +02:00
Eelco Dolstra
9b38b5f134 Remove the longDescription field 
It's not useful and takes up a lot of space.
 2014-09-30 15:44:09 +02:00
Eelco Dolstra
5b4de2dee6 hydra-evaluator: Reduce verbosity 2014-09-30 15:44:08 +02:00
Eelco Dolstra
09a96c642a hydra-eval-jobs: Use JSON instead of XML 
XML::Simple is pretty slow - reading the output for the Nixpkgs jobset
takes half a minute or so. JSON is pretty much instantaneous.
 2014-09-30 15:44:08 +02:00
Eelco Dolstra
1c20cfdf24 Drop the errorMsg column in the Jobs table 
We're not using it anywhere.
 2014-09-29 19:46:11 +02:00
aszlig
9c7f303255 Use mktemp for tempdir creation in prefetchers. 
This incorporates the following two commits from <nixpkgs>:

NixOS/nixpkgs@f83af95f8a
NixOS/nixpkgs@5e7a1cf955

Hydra was the original reason why I was fixing tempdir creation in the
first place. Seeing that Hydra ships its own versions of these scripts,
we need to patch them here as well.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
 2014-08-29 12:40:11 +02:00
Eelco Dolstra
a80bfceaca Remove timeout detection hack 2014-08-17 19:26:03 +02:00
Eelco Dolstra
69e3aa0438 Write Hydra roots as regular files instead of symlinks 
Note that this requires at least NixOS/Nix@1c208f2b7e.
 2014-08-01 17:24:55 +02:00
Eelco Dolstra
365de86ead Fix hydra-update-gc-roots 2014-07-16 23:20:58 +02:00
Eelco Dolstra
fb5f01097b Fix race between hydra-eval-jobs and hydra-update-gc-roots 
If hydra-eval-jobs creates a new root, and hydra-update-gc-roots runs
before hydra-evaluator has had a chance to add the corresponding build
to the database, then hydra-update-gc-roots will remove the root. If
subsequently the Nix garbage collector kicks in, it may remove the
build's .drv file before the build is performed. Since evaluation of
the Nixpkgs and NixOS jobsets nowadays takes a lot of time (e.g. an
hour), the probability of this happening is fairly high.

The quick fix is not to delete roots that are less than a day old. So
long as evaluation doesn't take longer than a day, this should be fine
;-)

Fixes #166.
 2014-07-14 13:18:07 +02:00
Ludovic Courtès
4471cae07e hydra-eval-guile-jobs: Add "This file is part of Hydra". 2014-04-08 18:22:21 +02:00
Ludovic Courtès
cc46456598 hydra-eval-guile-jobs: Register derivations as GC roots. 
* src/script/hydra-eval-guile-jobs.in (register-gc-root): New
  procedure.
  (job-evaluations->sxml): Add #:gc-roots-dir parameter.  Call
  'register-gc-root'.
  (job-evaluations->xml): Add #:gc-roots-dir parameter; pass it to
  'job-evaluations->sxml'.
  (eval-guile-jobs): Warn when --gc-roots-dir isn't passed.  Pass
  GC-ROOTS-DIR to 'job-evaluations->xml'.
 2014-04-08 18:22:21 +02:00
Ludovic Courtès
e3e8c1bc74 hydra-eval-guile-jobs: Honor $NIX_STORE_DIR. 
* src/script/hydra-eval-guile-jobs.in (strip-store-path): Honor
  $NIX_STORE_DIR.
 2014-04-08 18:22:21 +02:00
Eelco Dolstra
4cf0d7c4b3 hydra-update-gc-roots: Keep derivations of failed builds 
By keeping the derivations of failed builds in the most recent
evaluations, we ensure that failed builds can be restarted.
 2014-04-08 17:54:11 +02:00
Ludovic Courtès
61448ca2bd guile: Raise default absolute timeout to from 2h to 20h. 2014-02-27 22:19:52 +01:00
Shea Levy
a92a57f3b0 hydra-evaluator: When evaluating a single jobset, exit with a failure code if evaluation fails 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-11-27 14:29:05 -05:00
Eelco Dolstra
55f9d23933 Add a command `hydra-create-user' for managing user accounts 2013-11-06 13:36:29 +01:00
Eelco Dolstra
3315d1ea51 Remove obsolete hydra-control script 2013-11-06 11:42:49 +00:00
Eelco Dolstra
7a18e5f0c1 Remove unused file 2013-11-06 11:42:04 +00:00
Eelco Dolstra
ecadcef642 Prevent a division by zero in hydra-queue-runner 
Fixes #131.
 2013-11-06 12:15:11 +01:00
Eelco Dolstra
647d6bc3f1 Force creation of a new uncached eval if jobs have been removed 
Previously we only checked if jobs had been added.  We should probably
rename the "hasNewBuilds" field in the database.
 2013-11-01 19:24:52 +01:00
Shea Levy
0db950931a Merge remote-tracking branch 'upstream/who-broke-builds' into upstream-master 
Include information about who changed the build status in notification
emails, and enable optional per-input notification of said committers.

Conflicts due to two branches modifying the database schema.

Signed-off-by: Shea Levy <shea@shealevy.com>

Conflicts:
	src/lib/Hydra/Schema/Jobsets.pm
	src/sql/upgrade-23.sql
 2013-10-15 09:49:20 -04:00
Eelco Dolstra
c4e39d4769 Add one-shot jobsets 
There are jobsets that are evaluated only once, that is, after they've
been evaluated, they're disabled automatically.  This is primarily
useful for doing releases: for instance, doing an evaluation with
"officialRelease" set to "true" should be done only once.
 2013-10-11 12:01:52 +02:00
Eelco Dolstra
f592ce0026 Fix extreme slowness in hydra-queue-runner 
If there are builds in the queue that depend on another scheduled
build, then hydra-queue-runner will start the dependency first and
block the dependent builds.  This is implemented in
findBuildDependencyInQueue.  However, if there are tens of thousands
of such dependent builds, since each call to
findBuildDependencyInQueue may take a second or so, hydra-queue-runner
will spend hours just deciding which builds *not* to do.  Thus very
little progress is made.

So now, when a build is started, we immediately check which builds are
"blocked" by it (i.e. depend on it), and remove such builds from
consideration.
 2013-10-11 10:54:02 +02:00
Shea Levy
26470f1656 Check all inputs for blame but only email selected inputs 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-08 14:47:24 -04:00
Shea Levy
3e4a4e3761 Propagate checkresponsible from JobsetInput to BuildInput 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-08 13:24:49 -04:00
Eelco Dolstra
720c3892a3 Use delete instead of delete_all 
DBIC's delete_all method fetches all rows separately, which is slow.
 2013-10-03 19:42:44 +02:00
Eelco Dolstra
b1f7096935 Restore old findBuildDependencyInQueue behaviour 2013-10-03 13:08:32 +02:00
Eelco Dolstra
b1a26e6caa Revert "Add a dependency_lookup configuration option to enable (slow) dependency lookup in queue. This behaviour was disabled temporarily in accefbb79 due to slowness in very large queues, but some people might be dependent on it, so it is configurable until the previous behaviour is implemented more efficiently." 
This reverts commit 24f5a6b15ff148ac339918a998aa1291ad2c933f.
 2013-10-03 13:07:32 +02:00
Rob Vermaas
24f5a6b15f Add a dependency_lookup configuration option to enable (slow) dependency lookup in queue. This behaviour was disabled temporarily in accefbb79 due to slowness in very large queues, but some people might be dependent on it, so it is configurable until the previous behaviour is implemented more efficiently. 2013-10-03 09:09:18 +00:00
Eelco Dolstra
4dd1197d89 Fix uninitialized value warning 2013-09-30 10:01:09 +00:00
Eelco Dolstra
af2b0c8bad Remove dead code 2013-09-30 11:57:38 +02:00
Eelco Dolstra
d46ebeea99 Distinguish between permanent evaluation errors and transient input errors 
Fixes #112.
 2013-09-25 16:21:16 +02:00
Eelco Dolstra
e1c9e28589 Handle UTF-8 characters in eval error messages 2013-09-25 15:51:03 +02:00
Eelco Dolstra
a8db329839 Warn against multiple jobs with the same name 2013-09-25 15:30:59 +02:00
Eelco Dolstra
a2491f76a4 Use the same start/stop time for the build steps as for the build 2013-09-25 01:00:20 +02:00
Eelco Dolstra
f037a318e3 *headdesk* 
DBIC::Class helpfully doesn't warn you when you're matching against
unselected columns.  So this query actually returned all builds...
 2013-09-25 01:00:20 +02:00
Rob Vermaas
b1e29e50a7 Only send email notification of evaluation error when the evaluation error has changed. Fixes #121. 2013-09-24 12:01:57 -04:00
Shea Levy
6d5a3d0580 Derivations with multiple outputs break the 'link name is store path' assumption 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-09-22 21:26:59 -04:00