ahuston-0/hydra
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
4,472 Commits 6 Branches 0 Tags
e851d9f9f6631f0a6fe48226581cb012b8ee3294
Commit Graph

475 Commits

Author SHA1 Message Date
Graham Christensen
21e1ff0da1 perlcritic: don't open files as bare words 2021-09-06 22:20:17 -04:00
Graham Christensen
c880888f1e File::Slurp -> File::Slurper 2021-09-06 22:13:33 -04:00
Your Name
4677a7c894 perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
Graham Christensen
ea3ae0693e Hook up the retryable tasks with hydra-notify 2021-09-02 10:06:26 -04:00
Graham Christensen
c0e86faa78 hydra-notify: call TaskRetries->save_task if a task fails 2021-09-02 10:06:26 -04:00
Graham Christensen
1f2adf61aa hydra-notify: extract runPluginsForEvent to a TaskDispatcher 2021-09-02 10:06:26 -04:00
Your Name
4ebdcc290e fixup! hydra-notify: pre-declare metrics 2021-08-24 10:57:23 -04:00
Your Name
45e8fa5319 hydra-notify: support sending diagnostic dumps to STDERR on request 2021-08-24 10:56:13 -04:00
Your Name
de2282bcf4 hydra-notify: print out log lines indicating it is or is not launching the exporter 2021-08-24 10:56:13 -04:00
Your Name
5c1228e141 hydra-notify: pre-declare metrics 2021-08-24 10:56:13 -04:00
Your Name
6d7ee27d25 hydra-notify: make the prometheus endpoint configurable, default-off 2021-08-24 10:56:13 -04:00
Your Name
5d0ad5f649 hydra-notify: initial scratch take of prometheus events 2021-08-24 10:56:12 -04:00
Your Name
6e65c3b320 hydra-notify: fixup printing of build IDs 
Used to print:

    sending notifications for build Hydra::Model::DB::Builds=HASH(0x124cf960)->id...

Now it prints:

    sending notifications for build 123...
 2021-08-16 16:09:05 -04:00
Your Name
2c50227082 hydra-notify: properly call new_event 2021-08-16 15:52:25 -04:00
Your Name
e572a5e576 hydra-notify: use Hydra::Event 2021-08-16 15:52:14 -04:00
Graham Christensen
fa6d7abc13 hydra-notify: move BuildFinished processing to an Event 2021-08-13 16:51:29 -04:00
Graham Christensen
4a1389e36e hydra-notify: move StepFinished processing to an Event 2021-08-13 16:51:29 -04:00
Graham Christensen
4fdb20d3bd hydra-notify: move BuildStarted processing to an Event 2021-08-13 16:51:29 -04:00
Graham Christensen
10e85e3422 hydra-notify: Create a helper for running each plugin on an event 2021-08-13 16:51:29 -04:00
Graham Christensen
a14c8ad5f8 Merge pull request #995 from DeterminateSystems/declarative-jobsets-plugin 
Declarative jobsets: move event handling to a plugin
 2021-08-12 15:56:13 -04:00
Graham Christensen
0f958f3425 Merge pull request #997 from DeterminateSystems/abstract-listener 
Abstract over postgres' LISTEN/NOTIFY
 2021-08-12 14:00:34 -04:00
Graham Christensen
5027003285 Abstract over postgres' LISTEN/NOTIFY 
This lets us test the event loop if we wanted, and lets us
test the listening behavior in isolation.
 2021-08-12 13:54:05 -04:00
Graham Christensen
593af41808 Declarative jobsets: move event handling to a plugin 
Declarative jobsets were sort of tucked in to the event hanlder
itself. It turned out that it could have been implemented as a
plugin without much trouble.
 2021-08-12 12:48:18 -04:00
Graham Christensen
9c5f317453 hydra-notify: move buildFinished query in to the function impl 
This is more consistent with the other event handlers, of dealing
with IDs and not objects.
 2021-08-12 12:30:35 -04:00
Graham Christensen
508d99d611 Join to builds via jobset_id when easy 2021-06-01 11:16:47 -04:00
Graham Christensen
05636de7d2 hydra-init: upgrade passwords to Argon2 on startup 2021-04-16 12:32:13 -04:00
Graham Christensen
79b0ddc27d hydra-create-user: re-hash sha1 as Argon2 2021-04-16 12:32:13 -04:00
Graham Christensen
1d956be61e hydra-create-user: support Argon2 
Co-authored-by: Graham Christensen <graham@grahamc.com>
 2021-04-15 11:35:16 -04:00
Graham Christensen
425c7ff17f hydra-send-stats: add a --once option for testing 2021-03-20 09:16:08 -04:00
Jörg Thalheim
6bb180a0f2 hydra-send-stats: fix imports 2021-03-20 09:16:04 -04:00
Janne Heß
3c86083d21 Fixup #717 "Add the project name to declarative inputs" 
```
Mar 10 16:22:35 hydra-b hydra-evaluator[41419]: DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR:  null value in column "type" violates not-null constraint
Mar 10 16:22:35 hydra-b hydra-evaluator[41419]: DETAIL:  Failing row contains (62358, projectName, 0, null, null, null, hackworthltd, null, , null). [for Statement "INSERT INTO jobsetevalinputs ( altnr, dependency, eval, name, path, revision, sha256hash, type, uri, value) VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ? )" with ParamValues: 1='0', 2=undef, 3='62358', 4='projectName', 5='', 6=undef, 7=undef, 8=undef, 9=undef, 10='hackworthltd'] at /nix/store/cmqblv437mp57yz5lwvkzcqca4ldf3r5-hydra-0.1.20210308.ebf1cd2/bin/.hydra-eval-jobset-wrapped line 793
Mar 10 16:22:35 hydra-b hydra-evaluator[25828]: evaluation of jobset ‘hackworthltd:.jobsets (jobset#1)’ failed with exit code 1
```

Use the abstraction for creating inputs for simulating the project
name input.

Co-authored-by: Graham Christensen <graham@grahamc.com>
 2021-03-16 09:52:36 -04:00
Janne Heß
9e018d5443 Add the project name to declarative inputs 
This allows for more generic declarative configurations which can be
shared between projects.
 2021-03-08 17:36:52 +01:00
Matej Cotman
a551fba346 statsd: add a chance to set hostname and port in hydra.conf 
Co-authored-by: Graham Christensen <graham@grahamc.com>
 2021-03-08 10:03:16 -05:00
Graham Christensen
3fda37f65a RunCommand: Test 2021-02-24 13:43:25 -05:00
Graham Christensen
f1e75c8bff Move evaluation errors from evaluations to EvaluationErrors, a new table 
DBIx likes to eagerly select all columns without a way to really tell
it so. Therefore, this splits this one large column in to its own
table.

I'd also like to make "jobsets" use this table too, but that is on hold
to stop the bleeding caused by the extreme amount of traffic this is
causing.
 2021-02-01 21:33:14 -05:00
Graham Christensen
091d58c128 hydra-dev-server: autoflush stderr/stdout 2021-01-26 13:51:39 -05:00
Graham Christensen
54b8cb188e perl: jobsetevals -> jobset via by jobset_id 
Frankly, this was suspiciously little work.
 2021-01-26 13:51:39 -05:00
Graham Christensen
9516b256f1 Normalize nixexpr{input,path} from builds to jobsetevals. 
Duplicating this data on every record of the builds table cost
approximately 4G of duplication.

Note that the database migration included took about 4h45m on an
untuned server which uses very slow rotational disks in a RAID5 setup,
with not a lot of RAM. I imagine in production it might take an hour
or two, but not 4. If this should become a chunked migration, I can do
that.

Note: Because of the question about chunked migrations, I have NOT
YET tested this migration thoroughly enough for merge.
 2021-01-22 09:10:18 -05:00
Graham Christensen
086eed5147 hydra-eval-jobs: write evaluation errorMsg to the jobseteval table 2021-01-21 13:10:41 -05:00
Jelle Besseling
bbd4891133 Implement GitHub logins 
Requires the following configuration options
enable_github_login = 1
github_client_id
github_client_secret
Or github_client_secret_file which points to a file with the secret
 2020-12-28 14:37:03 +01:00
Eelco Dolstra
be709d450b Fix sysbuild 
596f4cf4b9
 2020-10-22 13:27:52 +02:00
Janne Heß
24acb9d6bb Fix non-static declarative jobsets 
With the current implementation, if ANY hash was found inside the decl
spec, the spec would be treated as static. This is problematic since
`inputs` is a hash and hence any configuration would be handled as a
static one.
This fixes the code to match the documentation and only switch to static
processing when ALL values are hashes.
 2020-09-13 18:21:38 +02:00
Eelco Dolstra
e707990e2d Merge pull request #804 from grahamc/fully-static-jobsets 
declarative projects: support fully static, declarative configuration
 2020-09-02 21:09:01 +02:00
Graham Christensen
7f16c0d243 declarative projects: support fully static, declarative configuration 2020-09-02 12:35:41 -04:00
Graham Christensen
648eb980dd hydra-notify: autoflush stdout too 2020-09-02 12:35:18 -04:00
Nikola Knezevic
3acdd21569 Remove references to hydra-postgresql.sql 
As of https://github.com/NixOS/hydra/pull/737 (removal of sqlite
dependency), the only supported database is Postgresql.

This change removes all references to hydra-postgresql.sql file. This
file is generated using a cpp on hydra.sql, but doesn't differ from
hydra.sql at all.
 2020-06-05 13:42:56 +02:00
Maximilian Bosch
d4822a5f4b Fix syntax error in hydra-send-stats 2020-06-02 15:30:42 +02:00
Eelco Dolstra
750e2e618a Merge pull request #770 from NixOS/remove-jobs 
Remove the Jobs table
 2020-06-01 10:25:41 +02:00
Eelco Dolstra
d5844897da Merge pull request #771 from knl/fix-unprocessed-notificationpendingsince 
Fix: Set notificationpendingsince for dependent builds
 2020-05-28 11:15:24 +02:00
Nikola Knezevic
7d52946982 Set notificationpendingsince for dependent builds 
`build_finished` Postgres event will never be fired for the dependent builds.

For example, on our Hydra, the following query always returns increasing
numbers, even though all notifications have been delivered:

```
hydra=> select count(1) from builds where notificationpendingsince is not null;
 count
-------
  4583
(1 row)
```

Thus, we have to iterate over all dependent builds and mark their
`notificationpendingsince` as `null`, otherwise they will pile up until
the next restart of hydra-notify, when they will get delivered.
 2020-05-28 10:45:41 +02:00