423 Commits

Author SHA1 Message Date
Nikola Knezevic
f79810bac1 Improve handling of Perl's block eval errors
Taken from `Perl::Critic`:

A common idiom in perl for dealing with possible errors is to use `eval`
followed by a check of `$@`/`$EVAL_ERROR`:

    eval {
        ...
    };
    if ($EVAL_ERROR) {
        ...
    }

There's a problem with this: the value of `$EVAL_ERROR` (`$@`) can change
between the end of the `eval` and the `if` statement. The issue are object
destructors:

    package Foo;

    ...

    sub DESTROY {
        ...
        eval { ... };
        ...
    }

    package main;

    eval {
        my $foo = Foo->new();
        ...
    };
    if ($EVAL_ERROR) {
        ...
    }

Assuming there are no other references to `$foo` created, when the
`eval` block in `main` is exited, `Foo::DESTROY()` will be invoked,
regardless of whether the `eval` finished normally or not. If the `eval`
in `main` fails, but the `eval` in `Foo::DESTROY()` succeeds, then
`$EVAL_ERROR` will be empty by the time that the `if` is executed.
Additional issues arise if you depend upon the exact contents of
`$EVAL_ERROR` and both `eval`s fail, because the messages from both will
be concatenated.

Even if there isn't an `eval` directly in the `DESTROY()` method code,
it may invoke code that does use `eval` or otherwise affects
`$EVAL_ERROR`.

The solution is to ensure that, upon normal exit, an `eval` returns a
true value and to test that value:

    # Constructors are no problem.
    my $object = eval { Class->new() };

    # To cover the possiblity that an operation may correctly return a
    # false value, end the block with "1":
    if ( eval { something(); 1 } ) {
        ...
    }

    eval {
        ...
        1;
    }
        or do {
            # Error handling here
        };

Unfortunately, you can't use the `defined` function to test the result;
`eval` returns an empty string on failure.

Various modules have been written to take some of the pain out of
properly localizing and checking `$@`/`$EVAL_ERROR`. For example:

    use Try::Tiny;
    try {
        ...
    } catch {
        # Error handling here;
        # The exception is in $_/$ARG, not $@/$EVAL_ERROR.
    };  # Note semicolon.

"But we don't use DESTROY() anywhere in our code!" you say. That may be
the case, but do any of the third-party modules you use have them? What
about any you may use in the future or updated versions of the ones you
already use?
2020-05-26 11:19:43 +02:00
Eelco Dolstra
e379628db0
hydra-eval-jobset: Pass --no-allow-import-from-derivation
https://github.com/NixOS/nixpkgs/issues/87592
2020-05-12 15:17:50 +02:00
Eelco Dolstra
96a514c169
Remove the "releases" feature
We haven't used this in many years (it was really only used for nix
and patchelf releases).
2020-05-06 12:39:21 +02:00
Maximilian Bosch
721c764951
Remove Hydra::Helper::nix::txn_do from the Perl code
To quote the function's comment:

  Awful hack to handle timeouts in SQLite: just retry the transaction.
  DBD::SQLite *has* a 30 second retry window, but apparently it
  doesn't work.

Since SQLite is now dropped entirely, this wrapper can be removed
completely.
2020-04-16 00:42:40 +02:00
Maximilian Bosch
efcbc08686
Get rid of dependency to SQLite
SQLite isn't properly supported by Hydra for a few years now[1], but
Hydra still depends on it. Apart from a slightly bigger closure this can
cause confusion by users since Hydra picks up SQLite rather than
PostgreSQL by default if HYDRA_DBI isn't configured properly[2]

[1] 78974abb6943533c3d94c7bd3282bf738b7be61c
[2] https://logs.nix.samueldr.com/nixos-dev/2020-04-10#3297342;
2020-04-16 00:42:40 +02:00
Eelco Dolstra
45ffe578b6
Keep track of the number of unsupported steps 2020-03-26 15:27:13 +01:00
Eelco Dolstra
d0688b93e1
Merge remote-tracking branch 'origin/master' into flake 2020-03-26 11:44:11 +01:00
Graham Christensen
12cc46cdb3
fixup: hydra-init: correct reference to hydra-backill-ids 2020-03-24 11:22:14 -04:00
Eelco Dolstra
4b5bb4e760
Merge remote-tracking branch 'origin/master' into flake 2020-03-04 15:28:23 +01:00
Graham Christensen
994430b94b
treewide: allow nix command 2020-03-03 22:52:20 -05:00
Eelco Dolstra
2a50daa377
Update aggregate handling
(cherry picked from commit cf961ac8933b76d9f0a3ac3eb49cc08879f5bcc9)
2020-02-20 10:13:39 +01:00
Eelco Dolstra
15187b059b
Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.

(cherry picked from commit 23c9ca3e94669087d463642baea0cf35a0b8d72f)
2020-02-20 09:58:12 +01:00
Eelco Dolstra
6f1d68bda4
Revert "hydra-eval-jobs -> nix eval-hydra-jobs"
This reverts commit 345512a6d0743eaac9d451833ecc1a71d12809c9.
2020-02-19 20:36:52 +01:00
Eelco Dolstra
cf961ac893
Update aggregate handling 2020-02-17 16:33:25 +01:00
Eelco Dolstra
345512a6d0
hydra-eval-jobs -> nix eval-hydra-jobs 2020-02-15 15:59:34 +01:00
Eelco Dolstra
23c9ca3e94
Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.
2020-02-15 15:59:34 +01:00
Eelco Dolstra
881b7449fd Merge remote-tracking branch 'origin/master' into flake 2020-02-11 14:23:16 +01:00
Graham Christensen
f0f41eaaff
LatestSucceededForJob{,set}: Filter with jobset_id 2020-02-11 07:06:20 -05:00
Eelco Dolstra
100e09a5b3
Merge remote-tracking branch 'origin/master' into flake
Also update flake.lock
2020-02-10 17:58:10 +01:00
Graham Christensen
c4cc72f944
hydra-init: Warn about the schema version migration 2020-02-10 11:43:03 -05:00
Graham Christensen
f69260118b
hydra-backfill-ids: create to add jobset_id values to Builds and Jobs
Vacuum every 10 iterations, update 10k at a time.
2020-02-10 11:43:03 -05:00
Graham Christensen
f3a561aecd
Builds: populate Builds.jobset_id in hydra-eval-jobset 2020-02-10 11:43:02 -05:00
Graham Christensen
624f1d8d2d
Jobs: populate Jobs.jobset_id field when writing from hydra-eval-jobset 2020-02-10 11:43:02 -05:00
Graham Christensen
511c2db8aa
Merge remote-tracking branch 'origin/master' into flake 2019-12-28 20:22:54 -05:00
Nikola Knezevic
06abfd6b2f hydra-send-stats: Cleanup removed metrics
In 29468995040ae21e0e1c14c1bdbb16ccb514caa8 these metrics got removed
due to refactoring of how notifications work.
2019-11-13 11:42:58 +01:00
Eelco Dolstra
55b0afa08f
Merge remote-tracking branch 'origin/master' into flake 2019-11-07 18:42:15 +01:00
Bas van Dijk
840e99f859 hydra-eval-jobset: $firstOutput is not used so can be removed 2019-11-05 13:58:40 +01:00
Eelco Dolstra
a816730d8b uri -> url 2019-10-23 20:31:27 +02:00
Eelco Dolstra
2de52d8538
Merge remote-tracking branch 'origin/master' into flake 2019-08-15 13:56:00 +02:00
Eelco Dolstra
92d8d6baa5
Avoid fetching Projects/Jobsets just to get the name column
In particular, doing a 'select * from Jobsets where ...' must be
avoided, because the 'errormsg' column can be very big.
2019-08-13 18:18:25 +02:00
Eelco Dolstra
f49a089fc0
hydra-notify: Don't do an unnecessary fetch of Jobsets 2019-08-13 18:18:24 +02:00
Eelco Dolstra
72c36373bb
hydra-notify: Fix processing notifications 2019-08-13 18:18:24 +02:00
Eelco Dolstra
976d88d675
Send notifications when evaluations start/finish/fail
* 'eval_started' has the format '<tmpId>\t<project>\t<jobset>'.

* 'eval_failed' has the format '<tmpId>'. (The cause of the error can
  be found in the database.)

* 'eval_added' has the format '<tmpId>:<evalId>'.
2019-08-13 18:18:24 +02:00
Eelco Dolstra
7114d2aceb
Separate payload elements using \t 2019-08-13 18:18:24 +02:00
Eelco Dolstra
2946899504
Turn hydra-notify into a daemon
It now receives notifications about started/finished builds/steps via
PostgreSQL. This gets rid of the (substantial) overhead of starting
hydra-notify for every event. It also allows other programs (even on
other machines) to listen to Hydra notifications.
2019-08-13 18:18:21 +02:00
Eelco Dolstra
ed00f0b25e
Cache flake-based jobset evaluations 2019-05-11 00:41:13 +02:00
Eelco Dolstra
842ef9af24
hydra-eval-jobset: Support flakes 2019-05-11 00:11:38 +02:00
Samuel Dionne-Riel
04ff9e217b Add hydra-dev-server which uses the classic Catalyst server
This, in turns allows

 - Using --restart for reloading the perl code
 - Printing traces on error
2019-01-22 20:34:21 -05:00
Tom Bereknyei
8bffbb7928
evalSettings deprecated, submodule fix 2018-10-03 00:45:42 -04:00
Eelco Dolstra
8e17a413f5
hydra-notify step-finished: Don't barf if the step has no log file 2018-08-01 17:17:46 +02:00
Eelco Dolstra
1898e31ed8
hydra-eval-jobset: Respect email_notification 2018-08-01 16:56:53 +02:00
AmineChikhaoui
298e99449b
add metrics for hydra notifications. 2018-06-12 13:00:39 +01:00
AmineChikhaoui
99e6c19672
add some more statistics about runnable/running builds per arch/system
features.
2018-06-12 13:00:00 +01:00
Eelco Dolstra
8913c682cf
Doh 2017-12-25 15:02:17 +01:00
Eelco Dolstra
0d79f16fd5
git-export -> source 2017-12-25 14:31:30 +01:00
Eelco Dolstra
2660c3cc54
Remove redundant check 2017-12-05 16:00:18 +01:00
Eelco Dolstra
364e21919a
hydra-eval-{jobs,jobset}: Pass file name as <...> 2017-11-28 16:51:00 +01:00
Eelco Dolstra
5986aae704
Add option eval_substituter
When using the "build" or "sysbuild" jobset input types in conjunction
with a binary cache store, the evaluator needs to be able to fetch
store paths from the binary cache. Typical usage:

  store_uri = s3://nix-test-cache?secret-key=...
  eval_substituter = s3://nix-test-cache

Also, the public key of the binary cache must be added to
binary-cache-public-keys in nix.conf, otherwise the local nix-daemon
won't allow the store paths to be copied over.
2017-11-20 16:12:20 +01:00
Eelco Dolstra
2cdc84f34f
Fix build against Nix master
Also, remove support in hydra-eval-jobs for multiple jobset input
alternatives. The web interface hasn't supported this in a long
time. Thus we can use the regular "--arg" handler.
2017-10-26 13:10:14 +02:00
Eelco Dolstra
bc60fccf78
Remove remaining references to store_mode etc. 2017-10-18 12:23:07 +02:00