Commit Graph

338 Commits

Author SHA1 Message Date
24467a7bde Nix::getHydraNotifyPrometheusConfig: print errors if the configuration provided is invalid. 2021-08-24 11:35:38 -04:00
6d7ee27d25 hydra-notify: make the prometheus endpoint configurable, default-off 2021-08-24 10:56:13 -04:00
4632132d02 fixup perlcritic: indirect new
> Subroutine new called using indirect syntax at line 14, column 14.  See page 349 of PBP.  (Severity: 5)
2021-08-20 11:39:19 -04:00
503b0e0b6f Remove unnecessary comment 2021-07-26 17:16:40 +00:00
173ecb5d92 Apply requested changes to Config.pm 2021-07-26 17:16:40 +00:00
4f95774ca3 Factor out common options 2021-07-26 03:43:09 +00:00
e98bd0ec8d Comment around dupe to avoid drift 2021-07-22 23:27:21 -04:00
09ad52ab60 Document getEvals a bit
Also drop the $self parameter.

Co-authored-by: gustavderdrache
2021-06-16 13:15:01 -04:00
5d95abf540 getBuilds: clarify the names of evals vs. the query builder 2021-06-16 11:53:04 -04:00
bf5c76feb6 getEvals: order by the eval table's ID
I broke this when I added `me.` in f1e75c8bff

I added me. to disambiguate `id`, but:

* eval.id works on the per-build page
* me.id works on the other pages
* Just id works everywhere if I drop:

    , prefetch => { evaluationerror => [ ] },

  but this causes a query per row to collect the evaluationerror
  records later, this becomes significantly slow on non-trivial
  datasets.

Using evals->current_source_alias will use the correct alias
whether it is me or eval or something else.
2021-06-16 11:24:30 -04:00
719d0a6134 jobset overview: join jobsets to builds by jobset id 2021-06-01 11:19:33 -04:00
508d99d611 Join to builds via jobset_id when easy 2021-06-01 11:16:47 -04:00
47e19ba22c Return HTTP 400 when creating Project with nonexistent user 2021-04-26 11:32:39 -07:00
cc9c91fe12 jobsets: put hidden and enabled jobsets at the end
Allows for generally correct zebra striping
2021-03-31 14:33:20 +00:00
6b7ca554f9 Update src/lib/Hydra/Helper/Escape.pm: fewer ()s
Co-authored-by: Stig <stig@stig.io>
2021-03-18 16:27:21 -04:00
88e0198a8e Create a helper for dealing with nested attribute sets 2021-03-18 11:33:36 -04:00
d62a2c1657 NixExprs: extract the escape function and test it 2021-03-18 11:24:17 -04:00
a551fba346 statsd: add a chance to set hostname and port in hydra.conf
Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-03-08 10:03:16 -05:00
6de9c6540c Merge pull request #858 from Infinisil/fix-declarative-flakes
Fix transition from declarative non-flake to flake jobset
2021-02-02 11:04:05 -05:00
f1e75c8bff Move evaluation errors from evaluations to EvaluationErrors, a new table
DBIx likes to eagerly select all columns without a way to really tell
it so. Therefore, this splits this one large column in to its own
table.

I'd also like to make "jobsets" use this table too, but that is on hold
to stop the bleeding caused by the extreme amount of traffic this is
causing.
2021-02-01 21:33:14 -05:00
1d45b63516 Fix transition from declarative non-flake to flake jobset
The database has these constraints:

    check ((type = 0) = (nixExprInput is not null and nixExprPath is not null)),
    check ((type = 1) = (flake is not null)),

which prevented switching to flakes in a declarative jobspec, since the
nixexpr{path,input} fields were not nulled in such an update

Co-Authored-By: Graham Christensen <graham@grahamc.com>
2021-02-01 18:57:40 +01:00
54b8cb188e perl: jobsetevals -> jobset via by jobset_id
Frankly, this was suspiciously little work.
2021-01-26 13:51:39 -05:00
bde8d81876 Merge pull request #811 from helsinki-systems/fix/override-constraint
Stop violating not null constraint
2020-11-22 00:02:26 +01:00
bd0ab9a5fb Stop violating not null constraint
Fixes this error:

ERROR: failed to process declarative jobset test:inputs,
DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st
execute failed: ERROR:  null value in column "emailoverride" violates
not-null constraint
2020-11-21 22:04:40 +01:00
d9dc7ca18b getPreviousBuild: Get previous build in the job, not jobset
Broken since 8adb433e3.
2020-10-28 13:29:02 +01:00
7f16c0d243 declarative projects: support fully static, declarative configuration 2020-09-02 12:35:41 -04:00
48678df8b6 updateDeclarativeJobset: only set the emailresponsible column when defined (#788) 2020-07-08 19:08:11 -04:00
e34d40d4f8 Remove dead method from Nix.pm
This method has been moved to hydra-eval-jobset a long time ago.
2020-06-05 15:01:36 +02:00
8adb433e3b Remove the Jobs table
This table has been superfluous for a long time.
2020-05-27 20:09:36 +02:00
f79810bac1 Improve handling of Perl's block eval errors
Taken from `Perl::Critic`:

A common idiom in perl for dealing with possible errors is to use `eval`
followed by a check of `$@`/`$EVAL_ERROR`:

    eval {
        ...
    };
    if ($EVAL_ERROR) {
        ...
    }

There's a problem with this: the value of `$EVAL_ERROR` (`$@`) can change
between the end of the `eval` and the `if` statement. The issue are object
destructors:

    package Foo;

    ...

    sub DESTROY {
        ...
        eval { ... };
        ...
    }

    package main;

    eval {
        my $foo = Foo->new();
        ...
    };
    if ($EVAL_ERROR) {
        ...
    }

Assuming there are no other references to `$foo` created, when the
`eval` block in `main` is exited, `Foo::DESTROY()` will be invoked,
regardless of whether the `eval` finished normally or not. If the `eval`
in `main` fails, but the `eval` in `Foo::DESTROY()` succeeds, then
`$EVAL_ERROR` will be empty by the time that the `if` is executed.
Additional issues arise if you depend upon the exact contents of
`$EVAL_ERROR` and both `eval`s fail, because the messages from both will
be concatenated.

Even if there isn't an `eval` directly in the `DESTROY()` method code,
it may invoke code that does use `eval` or otherwise affects
`$EVAL_ERROR`.

The solution is to ensure that, upon normal exit, an `eval` returns a
true value and to test that value:

    # Constructors are no problem.
    my $object = eval { Class->new() };

    # To cover the possiblity that an operation may correctly return a
    # false value, end the block with &quot;1&quot;:
    if ( eval { something(); 1 } ) {
        ...
    }

    eval {
        ...
        1;
    }
        or do {
            # Error handling here
        };

Unfortunately, you can't use the `defined` function to test the result;
`eval` returns an empty string on failure.

Various modules have been written to take some of the pain out of
properly localizing and checking `$@`/`$EVAL_ERROR`. For example:

    use Try::Tiny;
    try {
        ...
    } catch {
        # Error handling here;
        # The exception is in $_/$ARG, not $@/$EVAL_ERROR.
    };  # Note semicolon.

"But we don't use DESTROY() anywhere in our code!" you say. That may be
the case, but do any of the third-party modules you use have them? What
about any you may use in the future or updated versions of the ones you
already use?
2020-05-26 11:19:43 +02:00
575113396d Handle missing values in declarative jobsets
The current implementation will pass all values to `create_or_update` method. The
missing values will end up as `undef` (or `NULL`) when assigned to `%update`.
Thus, for columns that are NOT NULL, when, for example, flakes are not used,
will result in a horrible:

    DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed:
    ERROR:  null value in column "type" violates not-null constraint

    DETAIL:  Failing row contains (.jobsets, 118, hydra, hydra jobsets, src, hydra/jobsets.nix, null,
    null, null, 1589536378, 1, 0, 0, , 3, 30, 100, null, null, 1589536379, null, null). [for Statement
    "UPDATE jobsets SET checkinterval = ?, description = ?, enableemail = ?, nixexprinput = ?,
    nixexprpath = ?, type = ? WHERE ( ( name = ? AND project = ? ) )" with ParamValues: 1='30',
    2='hydra jobsets', 3='0', 4='src', 5='hydra/jobsets.nix', 6=undef, 7='.jobsets', 8='hydra'] at
    /nix/store/lsf81ip9ybxihk5praf2n0nh14a6i9j0-hydra-0.1.19700101.DIRTY/libexec/hydra/lib/Hydra/Helper/AddBuilds.pm line 50

This change just omits adding such values to `%update`, which results in
PostgreSQL assigning the default values.
2020-05-15 20:33:54 +02:00
721c764951 Remove Hydra::Helper::nix::txn_do from the Perl code
To quote the function's comment:

  Awful hack to handle timeouts in SQLite: just retry the transaction.
  DBD::SQLite *has* a 30 second retry window, but apparently it
  doesn't work.

Since SQLite is now dropped entirely, this wrapper can be removed
completely.
2020-04-16 00:42:40 +02:00
4b5bb4e760 Merge remote-tracking branch 'origin/master' into flake 2020-03-04 15:28:23 +01:00
117b9ecef1 Nix.pm: readNixFile: pass «--experimental-features nix-command»
Declarative jobsets were broken by the Nix update, causing
nix cat-file to break silently.

This commit restores declarative jobsets, based on top of a commit
making it easier to see what broke.
2020-03-03 22:36:21 -05:00
113a312f67 handleDeclarativeJobsetBuild: handle errors from readNixFile 2020-03-03 22:32:13 -05:00
55b0afa08f Merge remote-tracking branch 'origin/master' into flake 2019-11-07 18:42:15 +01:00
841a47cabe Add cancel-build role 2019-11-05 22:56:01 +01:00
ce1e10c116 Add bump-to-front role 2019-11-05 19:32:06 +01:00
6f99d958bc Fix declarative flake builds 2019-10-27 14:57:53 +01:00
92d8d6baa5 Avoid fetching Projects/Jobsets just to get the name column
In particular, doing a 'select * from Jobsets where ...' must be
avoided, because the 'errormsg' column can be very big.
2019-08-13 18:18:25 +02:00
c741576563 improve the error messages when invalid declarative jobsets are defined
(cherry picked from commit 7568b89a1a9da3a58a0cdddc7b5bcea7bb6209d8)
2019-03-20 15:27:37 -04:00
2e225ba7c8 Do not attempt to report dir for grab command failure if not specified. 2019-03-17 23:15:24 -07:00
e523e8a643 Additional helpful information in error messages. 2018-06-29 17:41:23 -07:00
83a48d2d4f Add a restart-jobs role
Frequently users want Hydra access just to restart jobs. However,
prior to this commit the only way to grant that access was by giving
them full Admin access which isn't necessarily what we want to do.

By having a restart-jobs role, we can grant this privilege to users
who are known to the community and want to help, but aren't long-time
members.

I haven't tested this commit, but it looks good to me...
2017-12-30 08:28:23 -05:00
d8d71ea4a3 hydra-server: Use server_store_uri instead of store_uri, if set
The store_uri may contain parameters that cause Nix to fail when
running under the hydra-server user (e.g. the signing key).
2017-10-18 15:33:55 +02:00
bc60fccf78 Remove remaining references to store_mode etc. 2017-10-18 12:23:07 +02:00
4c26cf0c81 getHydraConfig: Allow multiple includes 2017-04-20 08:00:09 -04:00
7e3f12fe10 Merge branch 'decl-project-remote-store' 2017-04-10 11:36:41 -04:00
fec895a642 hydra-server: Support logs in S3 2017-04-05 17:55:56 +02:00
a738f826e8 declarative projects: Pull jobset spec build from the remote store
Fixes #443
2017-04-01 10:38:49 -04:00