ahuston-0/hydra
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity

Concurrent hydra-evaluator

Browse Source 
This rewrites the top-level loop of hydra-evaluator in C++. The Perl
stuff is moved into hydra-eval-jobset. (Rewriting the entire evaluator
would be nice but is a bit too much work.) The new version has some
advantages:

* It can run multiple jobset evaluations in parallel.

* It uses PostgreSQL notifications so it doesn't have to poll the
  database. So if a jobset is triggered via the web interface or from
  a GitHub / Bitbucket webhook, evaluation of the jobset will start
  almost instantaneously (assuming the evaluator is not at its
  concurrency limit).

* It imposes a timeout on evaluations. So if e.g. hydra-eval-jobset
  hangs connecting to a Mercurial server, it will eventually be
  killed.
This commit is contained in:
Eelco Dolstra
2016-10-13 15:53:05 +02:00
parent 16feddd5d4
commit e0b2921ff2
15 changed files with 325 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
src/script/Makefile.am
View File

@ -4,7 +4,7 @@ EXTRA_DIST = \
distributable_scripts = \
hydra-init \
hydra-evaluator \
hydra-eval-jobset \
hydra-server \
hydra-update-gc-roots \
hydra-s3-backup-collect-garbage \

49
src/script/hydra-evaluator → src/script/hydra-eval-jobset
View File

@ -345,47 +345,10 @@ sub checkJobset {
}
sub checkSomeJobset {
# If any jobset has been triggered by a push, check it.
my ($jobset) = $db->resultset('Jobsets')->search(
{ 'triggertime' => { '!=', undef } },
{ join => 'project', order_by => [ 'triggertime' ], rows => 1 });
die "syntax: $0 <PROJECT> <JOBSET>\n" unless @ARGV == 2;
# Otherwise, check the jobset that hasn't been checked for the
# longest time (but don't check more often than the jobset's
# minimal check interval).
($jobset) = $db->resultset('Jobsets')->search(
{ 'project.enabled' => 1, 'me.enabled' => { '!=' => 0 },
, 'checkinterval' => { '!=', 0 }
, -or => [ 'lastcheckedtime' => undef, 'lastcheckedtime' => { '<', \ (time() . " - me.checkinterval") } ] },
{ join => 'project', order_by => [ 'lastcheckedtime nulls first' ], rows => 1 })
unless defined $jobset;
return 0 unless defined $jobset;
return system($0, $jobset->project->name, $jobset->name) == 0;
}
if (scalar @ARGV == 2) {
my $projectName = $ARGV[0];
my $jobsetName = $ARGV[1];
my $jobset = $db->resultset('Jobsets')->find($projectName, $jobsetName) or
die "$0: specified jobset \"$projectName:$jobsetName\" does not exist\n";
exit checkJobset($jobset);
}
while (1) {
eval {
if (checkSomeJobset) {
# Just so we don't go completely crazy if lastcheckedtime
# isn't updated properly.
sleep 1;
} else {
# print STDERR "sleeping...\n";
sleep 30;
}
};
if ($@) { print STDERR "$@"; }
}
my $projectName = $ARGV[0];
my $jobsetName = $ARGV[1];
my $jobset = $db->resultset('Jobsets')->find($projectName, $jobsetName) or
die "$0: specified jobset \"$projectName:$jobsetName\" does not exist\n";
exit checkJobset($jobset);