Commit Graph

391 Commits

Author SHA1 Message Date
f1e75c8bff Move evaluation errors from evaluations to EvaluationErrors, a new table
DBIx likes to eagerly select all columns without a way to really tell
it so. Therefore, this splits this one large column in to its own
table.

I'd also like to make "jobsets" use this table too, but that is on hold
to stop the bleeding caused by the extreme amount of traffic this is
causing.
2021-02-01 21:33:14 -05:00
091d58c128 hydra-dev-server: autoflush stderr/stdout 2021-01-26 13:51:39 -05:00
54b8cb188e perl: jobsetevals -> jobset via by jobset_id
Frankly, this was suspiciously little work.
2021-01-26 13:51:39 -05:00
9516b256f1 Normalize nixexpr{input,path} from builds to jobsetevals.
Duplicating this data on every record of the builds table cost
approximately 4G of duplication.

Note that the database migration included took about 4h45m on an
untuned server which uses very slow rotational disks in a RAID5 setup,
with not a lot of RAM. I imagine in production it might take an hour
or two, but not 4. If this should become a chunked migration, I can do
that.

Note: Because of the question about chunked migrations, I have NOT
YET tested this migration thoroughly enough for merge.
2021-01-22 09:10:18 -05:00
086eed5147 hydra-eval-jobs: write evaluation errorMsg to the jobseteval table 2021-01-21 13:10:41 -05:00
bbd4891133 Implement GitHub logins
Requires the following configuration options
enable_github_login = 1
github_client_id
github_client_secret
Or github_client_secret_file which points to a file with the secret
2020-12-28 14:37:03 +01:00
be709d450b Fix sysbuild
596f4cf4b9
2020-10-22 13:27:52 +02:00
24acb9d6bb Fix non-static declarative jobsets
With the current implementation, if ANY hash was found inside the decl
spec, the spec would be treated as static. This is problematic since
`inputs` is a hash and hence any configuration would be handled as a
static one.
This fixes the code to match the documentation and only switch to static
processing when ALL values are hashes.
2020-09-13 18:21:38 +02:00
e707990e2d Merge pull request from grahamc/fully-static-jobsets
declarative projects: support fully static, declarative configuration
2020-09-02 21:09:01 +02:00
7f16c0d243 declarative projects: support fully static, declarative configuration 2020-09-02 12:35:41 -04:00
648eb980dd hydra-notify: autoflush stdout too 2020-09-02 12:35:18 -04:00
3acdd21569 Remove references to hydra-postgresql.sql
As of https://github.com/NixOS/hydra/pull/737 (removal of sqlite
dependency), the only supported database is Postgresql.

This change removes all references to hydra-postgresql.sql file. This
file is generated using a cpp on hydra.sql, but doesn't differ from
hydra.sql at all.
2020-06-05 13:42:56 +02:00
d4822a5f4b Fix syntax error in hydra-send-stats 2020-06-02 15:30:42 +02:00
750e2e618a Merge pull request from NixOS/remove-jobs
Remove the Jobs table
2020-06-01 10:25:41 +02:00
d5844897da Merge pull request from knl/fix-unprocessed-notificationpendingsince
Fix: Set notificationpendingsince for dependent builds
2020-05-28 11:15:24 +02:00
7d52946982 Set notificationpendingsince for dependent builds
`build_finished` Postgres event will never be fired for the dependent builds.

For example, on our Hydra, the following query always returns increasing
numbers, even though all notifications have been delivered:

```
hydra=> select count(1) from builds where notificationpendingsince is not null;
 count
-------
  4583
(1 row)
```

Thus, we have to iterate over all dependent builds and mark their
`notificationpendingsince` as `null`, otherwise they will pile up until
the next restart of hydra-notify, when they will get delivered.
2020-05-28 10:45:41 +02:00
7148923d30 Make --no-allow-import-from-derivation configurable in hydra-eval-jobset
When deploying Hydra different than hydra.nixos.org one may encounter a problem
as building any job that uses IFD fails with:

May 22 19:41:07 hydra hydra-evaluator[6960]: error: "attempted to realize '/nix/store/1jm02mfiv58rpy8zrx95cpqxzsp64ssh-source.drv' during evaluation but 'allow-import-from-derivation' is false"
May 22 19:41:07 hydra hydra-evaluator[6960]: error: "attempted to realize '/nix/store/av3jr8ix4qcadq2wm3y3hplvxwzlhl4y-source.drv' during evaluation but 'allow-import-from-derivation' is false"
May 22 19:41:07 hydra hydra-evaluator[6960]: error: "attempted to realize
'/nix/store/2jm02mfiv58rpy8zrx95cpqxzsp64ssh-source.drv' during evaluation but
'allow-import-from-derivation' is false"

The recent change enforced passing `--no-allow-import-from-derivation`
to `hydra-eval-job` unconditionally. This change makes it configurable and
defaults to **NOT PASSING IT** -- most of the deployments allow IFDs.

The configuration option is called `allow_import_from_derivation` and
defaults to `true`. It is interpreted as a boolean, with only true option being
`true`.
2020-05-28 10:15:46 +02:00
8adb433e3b Remove the Jobs table
This table has been superfluous for a long time.
2020-05-27 20:09:36 +02:00
f79810bac1 Improve handling of Perl's block eval errors
Taken from `Perl::Critic`:

A common idiom in perl for dealing with possible errors is to use `eval`
followed by a check of `$@`/`$EVAL_ERROR`:

    eval {
        ...
    };
    if ($EVAL_ERROR) {
        ...
    }

There's a problem with this: the value of `$EVAL_ERROR` (`$@`) can change
between the end of the `eval` and the `if` statement. The issue are object
destructors:

    package Foo;

    ...

    sub DESTROY {
        ...
        eval { ... };
        ...
    }

    package main;

    eval {
        my $foo = Foo->new();
        ...
    };
    if ($EVAL_ERROR) {
        ...
    }

Assuming there are no other references to `$foo` created, when the
`eval` block in `main` is exited, `Foo::DESTROY()` will be invoked,
regardless of whether the `eval` finished normally or not. If the `eval`
in `main` fails, but the `eval` in `Foo::DESTROY()` succeeds, then
`$EVAL_ERROR` will be empty by the time that the `if` is executed.
Additional issues arise if you depend upon the exact contents of
`$EVAL_ERROR` and both `eval`s fail, because the messages from both will
be concatenated.

Even if there isn't an `eval` directly in the `DESTROY()` method code,
it may invoke code that does use `eval` or otherwise affects
`$EVAL_ERROR`.

The solution is to ensure that, upon normal exit, an `eval` returns a
true value and to test that value:

    # Constructors are no problem.
    my $object = eval { Class->new() };

    # To cover the possiblity that an operation may correctly return a
    # false value, end the block with "1":
    if ( eval { something(); 1 } ) {
        ...
    }

    eval {
        ...
        1;
    }
        or do {
            # Error handling here
        };

Unfortunately, you can't use the `defined` function to test the result;
`eval` returns an empty string on failure.

Various modules have been written to take some of the pain out of
properly localizing and checking `$@`/`$EVAL_ERROR`. For example:

    use Try::Tiny;
    try {
        ...
    } catch {
        # Error handling here;
        # The exception is in $_/$ARG, not $@/$EVAL_ERROR.
    };  # Note semicolon.

"But we don't use DESTROY() anywhere in our code!" you say. That may be
the case, but do any of the third-party modules you use have them? What
about any you may use in the future or updated versions of the ones you
already use?
2020-05-26 11:19:43 +02:00
e379628db0 hydra-eval-jobset: Pass --no-allow-import-from-derivation
https://github.com/NixOS/nixpkgs/issues/87592
2020-05-12 15:17:50 +02:00
96a514c169 Remove the "releases" feature
We haven't used this in many years (it was really only used for nix
and patchelf releases).
2020-05-06 12:39:21 +02:00
721c764951 Remove Hydra::Helper::nix::txn_do from the Perl code
To quote the function's comment:

  Awful hack to handle timeouts in SQLite: just retry the transaction.
  DBD::SQLite *has* a 30 second retry window, but apparently it
  doesn't work.

Since SQLite is now dropped entirely, this wrapper can be removed
completely.
2020-04-16 00:42:40 +02:00
efcbc08686 Get rid of dependency to SQLite
SQLite isn't properly supported by Hydra for a few years now[1], but
Hydra still depends on it. Apart from a slightly bigger closure this can
cause confusion by users since Hydra picks up SQLite rather than
PostgreSQL by default if HYDRA_DBI isn't configured properly[2]

[1] 78974abb69
[2] https://logs.nix.samueldr.com/nixos-dev/2020-04-10#3297342;
2020-04-16 00:42:40 +02:00
45ffe578b6 Keep track of the number of unsupported steps 2020-03-26 15:27:13 +01:00
d0688b93e1 Merge remote-tracking branch 'origin/master' into flake 2020-03-26 11:44:11 +01:00
12cc46cdb3 fixup: hydra-init: correct reference to hydra-backill-ids 2020-03-24 11:22:14 -04:00
4b5bb4e760 Merge remote-tracking branch 'origin/master' into flake 2020-03-04 15:28:23 +01:00
994430b94b treewide: allow nix command 2020-03-03 22:52:20 -05:00
2a50daa377 Update aggregate handling
(cherry picked from commit cf961ac893)
2020-02-20 10:13:39 +01:00
15187b059b Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.

(cherry picked from commit 23c9ca3e94)
2020-02-20 09:58:12 +01:00
6f1d68bda4 Revert "hydra-eval-jobs -> nix eval-hydra-jobs"
This reverts commit 345512a6d0.
2020-02-19 20:36:52 +01:00
cf961ac893 Update aggregate handling 2020-02-17 16:33:25 +01:00
345512a6d0 hydra-eval-jobs -> nix eval-hydra-jobs 2020-02-15 15:59:34 +01:00
23c9ca3e94 Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.
2020-02-15 15:59:34 +01:00
881b7449fd Merge remote-tracking branch 'origin/master' into flake 2020-02-11 14:23:16 +01:00
f0f41eaaff LatestSucceededForJob{,set}: Filter with jobset_id 2020-02-11 07:06:20 -05:00
100e09a5b3 Merge remote-tracking branch 'origin/master' into flake
Also update flake.lock
2020-02-10 17:58:10 +01:00
c4cc72f944 hydra-init: Warn about the schema version migration 2020-02-10 11:43:03 -05:00
f69260118b hydra-backfill-ids: create to add jobset_id values to Builds and Jobs
Vacuum every 10 iterations, update 10k at a time.
2020-02-10 11:43:03 -05:00
f3a561aecd Builds: populate Builds.jobset_id in hydra-eval-jobset 2020-02-10 11:43:02 -05:00
624f1d8d2d Jobs: populate Jobs.jobset_id field when writing from hydra-eval-jobset 2020-02-10 11:43:02 -05:00
511c2db8aa Merge remote-tracking branch 'origin/master' into flake 2019-12-28 20:22:54 -05:00
06abfd6b2f hydra-send-stats: Cleanup removed metrics
In 2946899504 these metrics got removed
due to refactoring of how notifications work.
2019-11-13 11:42:58 +01:00
55b0afa08f Merge remote-tracking branch 'origin/master' into flake 2019-11-07 18:42:15 +01:00
840e99f859 hydra-eval-jobset: $firstOutput is not used so can be removed 2019-11-05 13:58:40 +01:00
a816730d8b uri -> url 2019-10-23 20:31:27 +02:00
2de52d8538 Merge remote-tracking branch 'origin/master' into flake 2019-08-15 13:56:00 +02:00
92d8d6baa5 Avoid fetching Projects/Jobsets just to get the name column
In particular, doing a 'select * from Jobsets where ...' must be
avoided, because the 'errormsg' column can be very big.
2019-08-13 18:18:25 +02:00
f49a089fc0 hydra-notify: Don't do an unnecessary fetch of Jobsets 2019-08-13 18:18:24 +02:00
72c36373bb hydra-notify: Fix processing notifications 2019-08-13 18:18:24 +02:00