Commit Graph

468 Commits

Author SHA1 Message Date
e499509595 Switch to new Nix bindings, update Nix for that
Implements support for Nix's new Perl bindings[1]. The current state
basically does `openStore()`, but always uses `auto` and doesn't support
stores at other URIs.

Even though the stores are cached inside the Perl implementation, I
decided to instantiate those once in the Nix helper module. That way
store openings aren't cluttered across the entire codebase. Also, there
are two stores used later on - MACHINE_LOCAL_STORE for `auto`,
BINARY_CACHE_STORE for the one from `store_uri` in `hydra.conf` - and
using consistent names should make the intent clearer then.

This doesn't contain any behavioral changes, i.e. the build product
availability issue from #1352 isn't fixed. This patch only contains the
migration to the new API.

[1] https://github.com/NixOS/nix/pull/9863
2024-02-12 18:50:56 +01:00
fcde5908d8 More CA derivations prep
Again, with care not to change the schema in any way.
2024-01-25 21:32:22 -05:00
b16470c544 nix flake info -> nix flake metadata
This gets rid of a deprecation warning.
2022-09-06 19:23:20 +02:00
5c90edd19f Merge pull request #1103 from DeterminateSystems/runcommand/dynamic
Dynamic RunCommand
2022-04-19 10:09:47 -04:00
e5393c2cf8 fixup: make id non-ambiguous 2022-03-19 23:56:47 -04:00
137be3452e Reduce the jobset cols on the remaining two queries 2022-03-19 23:56:47 -04:00
f353a7ac41 update-gc-roots: try subselecting the jobset table 2022-03-19 23:56:47 -04:00
145667cb53 hydra-update-gc-roots: allow cached refs to the build's jobset
Re-executing this search_related on every access turned out to
create very problematic performance. If a jobset had a lot of
error output stored in the jobset, and there were many hundreds
or thousands of active jobs, this could easily cause >1Gbps of
network traffic.
2022-03-19 23:56:47 -04:00
a582e4c485 HydraTestContext: add \n's to various dies 2022-03-19 14:46:53 -04:00
074a2f96bf hydra-eval-jobset: emit a useful error if constituents errored 2022-03-19 14:37:12 -04:00
a22a8fa62d AddBuilds: reject declarative jobsets with dynamic runcommand enabled if disabled elsewhere 2022-02-11 14:35:52 -05:00
f07fb7d279 LDAP support: include BC support for the YAML based loading
Includes a refactoring of the configuration loader.
2022-02-11 10:49:38 -05:00
845e6d4760 captureStdoutStderr*: move to Hydra::Helper::Exec which helps avoid some environment variable fixation problems 2022-02-09 14:28:50 -05:00
517dce285a eval_added event: change interface to traceID\tjobsetID\tevaluationID
I was not going to break the interface until I noticed
the current implementation uses the string literal \t.
2022-02-08 09:51:35 -05:00
d512e6220f eval_failed event: change interface to traceID\tjobsetID
I was not going to break the interface until I noticed the other eval_* events used literal \ts
2022-02-08 09:51:35 -05:00
2597fa8c11 eval_cached event: change interface to traceID\tjobsetID\tevaluationID
I was not going to break the interface until I noticed
the current implementation uses the string literal \t.
2022-02-08 09:51:35 -05:00
c30f084f32 eval_started event: change interface to traceID\tjobsetID
I was not going to break the interface until I noticed
the current implementation uses the string literal \t.
2022-02-08 09:51:35 -05:00
0eeced7f08 hydra-create-user: Warn that creating users with a plaintext password is deprecated 2022-01-21 12:56:15 -05:00
98928a4125 fixups 2022-01-21 12:52:06 -05:00
76fbde6d6b Set noecho when reading passwords 2022-01-21 11:11:09 -05:00
bb893d0bd5 hydra-create-user: support prompting for passwords
I'm not sure this is a good implementation as-is. It does work,
but the password gets echo'd to the screen. I tried to use IO::Prompt
but IO::Prompt really seems to want to read the password from ARGV.
2022-01-21 10:40:56 -05:00
4945306a2b hydra-create-user: make docs about using --password-hash better 2022-01-21 10:39:22 -05:00
9dc40e0816 evaluator: don't save project, jobset on builds 2022-01-15 15:58:02 -05:00
7544d4ff47 hydra-update-gc-roots: get project and jobset information from the project and jobset tables 2022-01-15 14:26:45 -05:00
8c3c573953 hydra-eval-jobset: fixup old reference to project / jobset columns 2022-01-15 12:32:16 -05:00
42edd3a9d8 hydra-notify: respond to cached_build_queued 2022-01-14 09:13:17 -05:00
6b7f1da11e hydra-notify: operate on cached_build_finished events 2022-01-14 09:13:17 -05:00
96b2200895 hydra-notify: sort subscriptions 2022-01-11 13:28:04 -05:00
4da80e736e hydra-eval-jobset: send notifications when cached queued / finished builds are submitted 2022-01-11 13:28:04 -05:00
a69693a832 hydra-eval-jobset: Fix two minor bugs
I'm honestly too lazy to create two commits for fixing these one-line
issues so here's one.

The first hunk fixes the name of the projectName input. This is relevant
now because it gets logged and the log message looks stupid when there
is an input without a name.

The second hunk fixes a warning when using declarative non-flake
jobsets. The implementation may look weird but it's just the same as the
logical implication operator of nix.
2022-01-10 18:28:41 +01:00
f3d77c3e6b hydra-eval-jobset: Print the jobset that is evaluated
This is useful for systems that use concurrent evals
2022-01-10 13:44:07 +01:00
e84bbc7f90 hydra-eval-jobset: notify build_queued 2021-12-21 20:57:53 -05:00
d195e545f5 hydra-notify: listen for build_queued events 2021-12-21 20:57:53 -05:00
cf514baaf2 hydra-notify: track the duration of handling an event
Currently we only track how long individual plugins take.
With #1083 we stop executing a lot of plugins, but we
don't have a way to measure its practical impact on the
execution time of handling events.
2021-12-21 19:46:19 -05:00
4dfe787bc2 perlcritic: each() called at line 752, column 35. The each function may cause undefined behavior when operating on the hash while iterating. Use a foreach loop over the hash's keys or values instead. 2021-12-14 10:16:25 -05:00
7dcf6a01c6 JSON -> JSON::MaybeXS 2021-12-13 15:37:56 -05:00
2cbeca5c44 Merge pull request #1071 from DeterminateSystems/log-fetches-evals
hydra-eval-jobset: log fetches and evaluations
2021-12-08 16:00:29 -05:00
264092169c hydra-eval-jobset: log fetches and evaluations 2021-12-08 09:25:27 -05:00
47cabac4bf hydra-eval-jobset: fix use of uninitialized value
This happens with flake jobsets for obvious reasons (namely, that nixexprinput
and nixexprpath may be undefined for a flake jobset).

12:38:59 hydra-evaluator.1    | Use of uninitialized value $args[0] in join or string at /home/vin/workspace/vcs/hydra/src/script/hydra-eval-jobset line 648.
12:38:59 hydra-evaluator.1    | Use of uninitialized value $args[1] in join or string at /home/vin/workspace/vcs/hydra/src/script/hydra-eval-jobset line 648.
2021-12-03 12:12:53 -08:00
0f8d02894a hydra-eval-jobset: Scalar value @declInputs[0] better written as $declInputs[0] at hydra-eval-jobset line 570. 2021-10-19 21:52:39 -04:00
2745226ada Merge pull request #1003 from DeterminateSystems/perlcritic-level-4
perlcritic: level 4
2021-09-27 20:23:55 +02:00
c603ae35f0 perlcritic: Don't conditionally declare variables 2021-09-07 21:35:01 -04:00
b1879132af ResultSet::TaskRetries: add missing method, get_retryable_task
Yet again, manual testing is proving to be insufficient. I'm pretty
sure I wrote this code but lost it in a rebase, or perhaps the switch
to result classes.

At any rate, this implements the actual "fetch a retry row and run it"
for the hydra-notify daemon.

Tested by hand.
2021-09-07 11:12:10 -04:00
21e1ff0da1 perlcritic: don't open files as bare words 2021-09-06 22:20:17 -04:00
c880888f1e File::Slurp -> File::Slurper 2021-09-06 22:13:33 -04:00
4677a7c894 perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
ea3ae0693e Hook up the retryable tasks with hydra-notify 2021-09-02 10:06:26 -04:00
c0e86faa78 hydra-notify: call TaskRetries->save_task if a task fails 2021-09-02 10:06:26 -04:00
1f2adf61aa hydra-notify: extract runPluginsForEvent to a TaskDispatcher 2021-09-02 10:06:26 -04:00
4ebdcc290e fixup! hydra-notify: pre-declare metrics 2021-08-24 10:57:23 -04:00