ahuston-0/hydra
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
2,718 Commits 6 Branches 0 Tags
511c2db8aa6ec7f59fc3a775374020ff327c9657
Commit Graph

350 Commits

Author SHA1 Message Date
Eelco Dolstra
2b9bf7526d hydra-send-stats: Keep track of the number of machines 2015-08-27 12:05:06 +02:00
Eelco Dolstra
a6e3cb53b9 Use /usr/bin/env to find perl 
This is nicer in nix-shell.
 2015-08-17 14:18:20 +02:00
Eelco Dolstra
3ded87329d Keep track of how many threads are waiting 2015-07-10 19:10:14 +02:00
Eelco Dolstra
3e7bbec40b hydra-evaluator: Send statistics to statsd 2015-07-10 16:40:50 +02:00
Eelco Dolstra
bbee81efae Use triggers for all notifications on Builds table changes 2015-07-08 12:05:32 +02:00
Eelco Dolstra
e35b704d80 Drop the 5 minute minimum interval between triggered evals 2015-07-01 14:45:39 +02:00
Eelco Dolstra
f5e5a1b96e Don't wake up the queue runner for cached evals 2015-06-26 20:59:14 +02:00
Eelco Dolstra
401f5bdce2 Add a unit for hydra-send-stats 2015-06-26 15:24:12 +02:00
Eelco Dolstra
c6fcce3b3b Moar stats 2015-06-25 16:47:39 +02:00
Eelco Dolstra
1a0e1eb5a0 More stats 2015-06-24 13:19:27 +02:00
Eelco Dolstra
62219adaf3 Send queue runner stats to statsd 
This is currently done by a separate program that periodically
calls "hydra-queue-runner --status". Eventually, I'll do this
in the queue runner directly.

Fixes #220.
 2015-06-23 14:56:43 +02:00
Eelco Dolstra
a317d24b29 hydra-queue-runner: Send build notifications 
Since our notification plugins are written in Perl, sending
notification from C++ requires a small Perl helper named
‘hydra-notify’.
 2015-06-23 00:14:49 +02:00
Eelco Dolstra
e32ee3d5b9 Remove hydra-build and the old hydra-queue-runner 2015-06-22 15:43:15 +02:00
Eelco Dolstra
a984c0badc Merge branch 'master' into build-ng 2015-06-15 18:21:07 +02:00
Eelco Dolstra
c08883966c Use PostgreSQL notifications for queue events 
Hydra-queue-runner now no longer polls the queue periodically, but
instead sleeps until it receives a notification from PostgreSQL about
a change to the queue (build added, build cancelled or build
restarted).

Also, for the "build added" case, we now only check for builds with an
ID greater than the previous greatest ID. This is much more efficient
if the queue is large.
 2015-06-11 17:41:59 +02:00
Eelco Dolstra
672bbb1c67 hydra-update-gc-roots: Get Builds and BuildOutputs in the same query 
This greatly reduces the number of roundtrips to the database.
 2015-06-11 14:10:21 +02:00
Eelco Dolstra
9989a90e99 Keep the most recent successful build of current jobs 
Fixes #140.
 2015-06-11 14:10:21 +02:00
Eelco Dolstra
e003665146 Split timeSpent query into 2 separate queries, as postgresql isn't able to figure out a decent query plan. With 120k jobs in queue, this makes some queries go from 100s to 1-2s. 2015-05-01 12:28:29 +02:00
Eelco Dolstra
01cd6397cb Better error message 2015-04-14 15:16:24 +02:00
Eelco Dolstra
63306aaf5a hydra-evaluator: Add some debug code 2015-04-09 17:35:04 +02:00
Eelco Dolstra
9e664cf8b0 Fix not-null constraint violation inserting build step 
DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR:  null value in column "machine" violates not-null constraint
 2015-04-07 13:34:00 +02:00
Eelco Dolstra
ad2b7646ac Don't show missing paths in logs 2015-03-04 15:44:04 +01:00
Eelco Dolstra
adc72d2409 Record which build a failed build step was propagated from 2015-02-25 16:42:32 +01:00
Eelco Dolstra
48af914e28 When propagating failure, propagate the duration and machine 
Previously the duration would just show as "0" and you would have to
search for the original build to get the duration.
 2015-02-25 16:21:54 +01:00
Eelco Dolstra
a12135fc51 Don't use Perl's -w flag 2014-12-12 17:39:52 +01:00
Eelco Dolstra
6a0b9a3476 hydra-build: Handle new trace messages 
Now build step duration no longer includes network overhead (i.e.
time to copy closures to the build machine).
 2014-12-12 17:20:50 +01:00
Eelco Dolstra
c0ca5489e1 Don't use given/when 
These give warnings in Perl >= 5.18:

  given is experimental at /home/hydra/src/hydra/src/lib/Hydra/Helper/CatalystUtils.pm line 241.
  when is experimental at /home/hydra/src/hydra/src/lib/Hydra/Helper/CatalystUtils.pm line 242.
  ...
 2014-12-12 11:27:17 +01:00
Eelco Dolstra
8523130ebb Use Email::MIME instead of Email::Simple 
Email::Simple cannot handle non-ASCII characters.

Fixes #191.
 2014-11-19 14:45:46 +01:00
Eelco Dolstra
5a7efc0469 Fix aggregate handling 2014-10-01 15:34:05 +02:00
Eelco Dolstra
9b38b5f134 Remove the longDescription field 
It's not useful and takes up a lot of space.
 2014-09-30 15:44:09 +02:00
Eelco Dolstra
5b4de2dee6 hydra-evaluator: Reduce verbosity 2014-09-30 15:44:08 +02:00
Eelco Dolstra
09a96c642a hydra-eval-jobs: Use JSON instead of XML 
XML::Simple is pretty slow - reading the output for the Nixpkgs jobset
takes half a minute or so. JSON is pretty much instantaneous.
 2014-09-30 15:44:08 +02:00
Eelco Dolstra
1c20cfdf24 Drop the errorMsg column in the Jobs table 
We're not using it anywhere.
 2014-09-29 19:46:11 +02:00
aszlig
9c7f303255 Use mktemp for tempdir creation in prefetchers. 
This incorporates the following two commits from <nixpkgs>:

NixOS/nixpkgs@f83af95f8a
NixOS/nixpkgs@5e7a1cf955

Hydra was the original reason why I was fixing tempdir creation in the
first place. Seeing that Hydra ships its own versions of these scripts,
we need to patch them here as well.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
 2014-08-29 12:40:11 +02:00
Eelco Dolstra
a80bfceaca Remove timeout detection hack 2014-08-17 19:26:03 +02:00
Eelco Dolstra
69e3aa0438 Write Hydra roots as regular files instead of symlinks 
Note that this requires at least NixOS/Nix@1c208f2b7e.
 2014-08-01 17:24:55 +02:00
Eelco Dolstra
365de86ead Fix hydra-update-gc-roots 2014-07-16 23:20:58 +02:00
Eelco Dolstra
fb5f01097b Fix race between hydra-eval-jobs and hydra-update-gc-roots 
If hydra-eval-jobs creates a new root, and hydra-update-gc-roots runs
before hydra-evaluator has had a chance to add the corresponding build
to the database, then hydra-update-gc-roots will remove the root. If
subsequently the Nix garbage collector kicks in, it may remove the
build's .drv file before the build is performed. Since evaluation of
the Nixpkgs and NixOS jobsets nowadays takes a lot of time (e.g. an
hour), the probability of this happening is fairly high.

The quick fix is not to delete roots that are less than a day old. So
long as evaluation doesn't take longer than a day, this should be fine
;-)

Fixes #166.
 2014-07-14 13:18:07 +02:00
Ludovic Courtès
4471cae07e hydra-eval-guile-jobs: Add "This file is part of Hydra". 2014-04-08 18:22:21 +02:00
Ludovic Courtès
cc46456598 hydra-eval-guile-jobs: Register derivations as GC roots. 
* src/script/hydra-eval-guile-jobs.in (register-gc-root): New
  procedure.
  (job-evaluations->sxml): Add #:gc-roots-dir parameter.  Call
  'register-gc-root'.
  (job-evaluations->xml): Add #:gc-roots-dir parameter; pass it to
  'job-evaluations->sxml'.
  (eval-guile-jobs): Warn when --gc-roots-dir isn't passed.  Pass
  GC-ROOTS-DIR to 'job-evaluations->xml'.
 2014-04-08 18:22:21 +02:00
Ludovic Courtès
e3e8c1bc74 hydra-eval-guile-jobs: Honor $NIX_STORE_DIR. 
* src/script/hydra-eval-guile-jobs.in (strip-store-path): Honor
  $NIX_STORE_DIR.
 2014-04-08 18:22:21 +02:00
Eelco Dolstra
4cf0d7c4b3 hydra-update-gc-roots: Keep derivations of failed builds 
By keeping the derivations of failed builds in the most recent
evaluations, we ensure that failed builds can be restarted.
 2014-04-08 17:54:11 +02:00
Ludovic Courtès
61448ca2bd guile: Raise default absolute timeout to from 2h to 20h. 2014-02-27 22:19:52 +01:00
Shea Levy
a92a57f3b0 hydra-evaluator: When evaluating a single jobset, exit with a failure code if evaluation fails 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-11-27 14:29:05 -05:00
Eelco Dolstra
55f9d23933 Add a command `hydra-create-user' for managing user accounts 2013-11-06 13:36:29 +01:00
Eelco Dolstra
3315d1ea51 Remove obsolete hydra-control script 2013-11-06 11:42:49 +00:00
Eelco Dolstra
7a18e5f0c1 Remove unused file 2013-11-06 11:42:04 +00:00
Eelco Dolstra
ecadcef642 Prevent a division by zero in hydra-queue-runner 
Fixes #131.
 2013-11-06 12:15:11 +01:00
Eelco Dolstra
647d6bc3f1 Force creation of a new uncached eval if jobs have been removed 
Previously we only checked if jobs had been added.  We should probably
rename the "hasNewBuilds" field in the database.
 2013-11-01 19:24:52 +01:00
Shea Levy
0db950931a Merge remote-tracking branch 'upstream/who-broke-builds' into upstream-master 
Include information about who changed the build status in notification
emails, and enable optional per-input notification of said committers.

Conflicts due to two branches modifying the database schema.

Signed-off-by: Shea Levy <shea@shealevy.com>

Conflicts:
	src/lib/Hydra/Schema/Jobsets.pm
	src/sql/upgrade-23.sql
 2013-10-15 09:49:20 -04:00