Commit Graph

423 Commits

Author SHA1 Message Date
f79810bac1 Improve handling of Perl's block eval errors
Taken from `Perl::Critic`:

A common idiom in perl for dealing with possible errors is to use `eval`
followed by a check of `$@`/`$EVAL_ERROR`:

    eval {
        ...
    };
    if ($EVAL_ERROR) {
        ...
    }

There's a problem with this: the value of `$EVAL_ERROR` (`$@`) can change
between the end of the `eval` and the `if` statement. The issue are object
destructors:

    package Foo;

    ...

    sub DESTROY {
        ...
        eval { ... };
        ...
    }

    package main;

    eval {
        my $foo = Foo->new();
        ...
    };
    if ($EVAL_ERROR) {
        ...
    }

Assuming there are no other references to `$foo` created, when the
`eval` block in `main` is exited, `Foo::DESTROY()` will be invoked,
regardless of whether the `eval` finished normally or not. If the `eval`
in `main` fails, but the `eval` in `Foo::DESTROY()` succeeds, then
`$EVAL_ERROR` will be empty by the time that the `if` is executed.
Additional issues arise if you depend upon the exact contents of
`$EVAL_ERROR` and both `eval`s fail, because the messages from both will
be concatenated.

Even if there isn't an `eval` directly in the `DESTROY()` method code,
it may invoke code that does use `eval` or otherwise affects
`$EVAL_ERROR`.

The solution is to ensure that, upon normal exit, an `eval` returns a
true value and to test that value:

    # Constructors are no problem.
    my $object = eval { Class->new() };

    # To cover the possiblity that an operation may correctly return a
    # false value, end the block with "1":
    if ( eval { something(); 1 } ) {
        ...
    }

    eval {
        ...
        1;
    }
        or do {
            # Error handling here
        };

Unfortunately, you can't use the `defined` function to test the result;
`eval` returns an empty string on failure.

Various modules have been written to take some of the pain out of
properly localizing and checking `$@`/`$EVAL_ERROR`. For example:

    use Try::Tiny;
    try {
        ...
    } catch {
        # Error handling here;
        # The exception is in $_/$ARG, not $@/$EVAL_ERROR.
    };  # Note semicolon.

"But we don't use DESTROY() anywhere in our code!" you say. That may be
the case, but do any of the third-party modules you use have them? What
about any you may use in the future or updated versions of the ones you
already use?
2020-05-26 11:19:43 +02:00
e379628db0 hydra-eval-jobset: Pass --no-allow-import-from-derivation
https://github.com/NixOS/nixpkgs/issues/87592
2020-05-12 15:17:50 +02:00
96a514c169 Remove the "releases" feature
We haven't used this in many years (it was really only used for nix
and patchelf releases).
2020-05-06 12:39:21 +02:00
721c764951 Remove Hydra::Helper::nix::txn_do from the Perl code
To quote the function's comment:

  Awful hack to handle timeouts in SQLite: just retry the transaction.
  DBD::SQLite *has* a 30 second retry window, but apparently it
  doesn't work.

Since SQLite is now dropped entirely, this wrapper can be removed
completely.
2020-04-16 00:42:40 +02:00
efcbc08686 Get rid of dependency to SQLite
SQLite isn't properly supported by Hydra for a few years now[1], but
Hydra still depends on it. Apart from a slightly bigger closure this can
cause confusion by users since Hydra picks up SQLite rather than
PostgreSQL by default if HYDRA_DBI isn't configured properly[2]

[1] 78974abb69
[2] https://logs.nix.samueldr.com/nixos-dev/2020-04-10#3297342;
2020-04-16 00:42:40 +02:00
45ffe578b6 Keep track of the number of unsupported steps 2020-03-26 15:27:13 +01:00
d0688b93e1 Merge remote-tracking branch 'origin/master' into flake 2020-03-26 11:44:11 +01:00
12cc46cdb3 fixup: hydra-init: correct reference to hydra-backill-ids 2020-03-24 11:22:14 -04:00
4b5bb4e760 Merge remote-tracking branch 'origin/master' into flake 2020-03-04 15:28:23 +01:00
994430b94b treewide: allow nix command 2020-03-03 22:52:20 -05:00
2a50daa377 Update aggregate handling
(cherry picked from commit cf961ac893)
2020-02-20 10:13:39 +01:00
15187b059b Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.

(cherry picked from commit 23c9ca3e94)
2020-02-20 09:58:12 +01:00
6f1d68bda4 Revert "hydra-eval-jobs -> nix eval-hydra-jobs"
This reverts commit 345512a6d0.
2020-02-19 20:36:52 +01:00
cf961ac893 Update aggregate handling 2020-02-17 16:33:25 +01:00
345512a6d0 hydra-eval-jobs -> nix eval-hydra-jobs 2020-02-15 15:59:34 +01:00
23c9ca3e94 Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.
2020-02-15 15:59:34 +01:00
881b7449fd Merge remote-tracking branch 'origin/master' into flake 2020-02-11 14:23:16 +01:00
f0f41eaaff LatestSucceededForJob{,set}: Filter with jobset_id 2020-02-11 07:06:20 -05:00
100e09a5b3 Merge remote-tracking branch 'origin/master' into flake
Also update flake.lock
2020-02-10 17:58:10 +01:00
c4cc72f944 hydra-init: Warn about the schema version migration 2020-02-10 11:43:03 -05:00
f69260118b hydra-backfill-ids: create to add jobset_id values to Builds and Jobs
Vacuum every 10 iterations, update 10k at a time.
2020-02-10 11:43:03 -05:00
f3a561aecd Builds: populate Builds.jobset_id in hydra-eval-jobset 2020-02-10 11:43:02 -05:00
624f1d8d2d Jobs: populate Jobs.jobset_id field when writing from hydra-eval-jobset 2020-02-10 11:43:02 -05:00
511c2db8aa Merge remote-tracking branch 'origin/master' into flake 2019-12-28 20:22:54 -05:00
06abfd6b2f hydra-send-stats: Cleanup removed metrics
In 2946899504 these metrics got removed
due to refactoring of how notifications work.
2019-11-13 11:42:58 +01:00
55b0afa08f Merge remote-tracking branch 'origin/master' into flake 2019-11-07 18:42:15 +01:00
840e99f859 hydra-eval-jobset: $firstOutput is not used so can be removed 2019-11-05 13:58:40 +01:00
a816730d8b uri -> url 2019-10-23 20:31:27 +02:00
2de52d8538 Merge remote-tracking branch 'origin/master' into flake 2019-08-15 13:56:00 +02:00
92d8d6baa5 Avoid fetching Projects/Jobsets just to get the name column
In particular, doing a 'select * from Jobsets where ...' must be
avoided, because the 'errormsg' column can be very big.
2019-08-13 18:18:25 +02:00
f49a089fc0 hydra-notify: Don't do an unnecessary fetch of Jobsets 2019-08-13 18:18:24 +02:00
72c36373bb hydra-notify: Fix processing notifications 2019-08-13 18:18:24 +02:00
976d88d675 Send notifications when evaluations start/finish/fail
* 'eval_started' has the format '<tmpId>\t<project>\t<jobset>'.

* 'eval_failed' has the format '<tmpId>'. (The cause of the error can
  be found in the database.)

* 'eval_added' has the format '<tmpId>:<evalId>'.
2019-08-13 18:18:24 +02:00
7114d2aceb Separate payload elements using \t 2019-08-13 18:18:24 +02:00
2946899504 Turn hydra-notify into a daemon
It now receives notifications about started/finished builds/steps via
PostgreSQL. This gets rid of the (substantial) overhead of starting
hydra-notify for every event. It also allows other programs (even on
other machines) to listen to Hydra notifications.
2019-08-13 18:18:21 +02:00
ed00f0b25e Cache flake-based jobset evaluations 2019-05-11 00:41:13 +02:00
842ef9af24 hydra-eval-jobset: Support flakes 2019-05-11 00:11:38 +02:00
04ff9e217b Add hydra-dev-server which uses the classic Catalyst server
This, in turns allows

 - Using --restart for reloading the perl code
 - Printing traces on error
2019-01-22 20:34:21 -05:00
8bffbb7928 evalSettings deprecated, submodule fix 2018-10-03 00:45:42 -04:00
8e17a413f5 hydra-notify step-finished: Don't barf if the step has no log file 2018-08-01 17:17:46 +02:00
1898e31ed8 hydra-eval-jobset: Respect email_notification 2018-08-01 16:56:53 +02:00
298e99449b add metrics for hydra notifications. 2018-06-12 13:00:39 +01:00
99e6c19672 add some more statistics about runnable/running builds per arch/system
features.
2018-06-12 13:00:00 +01:00
8913c682cf Doh 2017-12-25 15:02:17 +01:00
0d79f16fd5 git-export -> source 2017-12-25 14:31:30 +01:00
2660c3cc54 Remove redundant check 2017-12-05 16:00:18 +01:00
364e21919a hydra-eval-{jobs,jobset}: Pass file name as <...> 2017-11-28 16:51:00 +01:00
5986aae704 Add option eval_substituter
When using the "build" or "sysbuild" jobset input types in conjunction
with a binary cache store, the evaluator needs to be able to fetch
store paths from the binary cache. Typical usage:

  store_uri = s3://nix-test-cache?secret-key=...
  eval_substituter = s3://nix-test-cache

Also, the public key of the binary cache must be added to
binary-cache-public-keys in nix.conf, otherwise the local nix-daemon
won't allow the store paths to be copied over.
2017-11-20 16:12:20 +01:00
2cdc84f34f Fix build against Nix master
Also, remove support in hydra-eval-jobs for multiple jobset input
alternatives. The web interface hasn't supported this in a long
time. Thus we can use the regular "--arg" handler.
2017-10-26 13:10:14 +02:00
bc60fccf78 Remove remaining references to store_mode etc. 2017-10-18 12:23:07 +02:00