ahuston-0/hydra
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
4,173 Commits 3 Branches 0 Tags
Commit Graph

2985 Commits

Author SHA1 Message Date
Graham Christensen
145667cb53 hydra-update-gc-roots: allow cached refs to the build's jobset 
Re-executing this search_related on every access turned out to
create very problematic performance. If a jobset had a lot of
error output stored in the jobset, and there were many hundreds
or thousands of active jobs, this could easily cause >1Gbps of
network traffic.
 2022-03-19 23:56:47 -04:00
Graham Christensen
a582e4c485 HydraTestContext: add \n's to various dies 2022-03-19 14:46:53 -04:00
Graham Christensen
074a2f96bf hydra-eval-jobset: emit a useful error if constituents errored 2022-03-19 14:37:12 -04:00
Cole Helbling
c0f826b92d hydra-queue-runner: get the listening port from the exposer itself 
Otherwise, when the port is randomly chosen (e.g. by specifying no port,
or a port of 0), it will just show that the port is 0 and not the port
that is actually serving the metrics.
 2022-03-14 08:41:45 -07:00
Cole Helbling
52a29d43e6 hydra-queue-runner: make registry member of State, configurable metrics port 
Thanks to the updated prometheus-cpp library, specifying a port of 0
will cause it to pick a random (available) port -- ideal for tests.
 2022-03-11 11:58:10 -08:00
Cole Helbling
3bf31bd6a6 hydra-queue-runner: add simple "up" exporter 
There are probably better ways to achieve this (and will likely need to
be refactored a bit to support further metrics).
 2022-03-10 12:36:58 -08:00
Graham Christensen
9316544abf
src/hydra-eval-jobs/hydra-eval-jobs.cc: .get<std::string> for drvPath 
Co-authored-by: Kayla Fire <firestack@users.noreply.github.com>
 2022-02-21 12:41:21 -05:00
Graham Christensen
290e0653ad hydra-eval-jobs: GC root aggregate jobs 2022-02-20 12:28:40 -05:00
John Ericson
445bba337b Make copyClosureTo take a regular C++ ref to the store 
This is syntactically lighter wait, and demonstates there are no weird
dynamic lifetimes involved, just regular passing reference to callee
which it only borrows for the duration of the call.
 2022-02-20 17:22:43 +00:00
John Ericson
f14c583ce5 Use copyClosure instead of computeFSClosure + copyPaths 
It is more terse, and in the future it is possible `copyClosure` will
become more sophisticated.
 2022-02-19 11:59:17 -05:00
Graham Christensen
4c41ca08e1
Merge pull request #1155 from helsinki-systems/fix/graph-readability 
build-graphs: Fix readability in dark mode
 2022-02-14 11:27:37 -05:00
ajs124
1c84676527
Fit more content on screen 2022-02-13 18:33:37 +01:00
Janne Heß
6d146deaf0
build-graphs: Fix readability in dark mode 2022-02-13 14:00:17 +01:00
Graham Christensen
27ddde1e9e dynamic runcommand: print a notice on the build page if it is disabled 2022-02-11 15:04:54 -05:00
Cole Helbling
a22a8fa62d AddBuilds: reject declarative jobsets with dynamic runcommand enabled if disabled elsewhere 2022-02-11 14:35:52 -05:00
Cole Helbling
928ba9e854 Controller/{Jobset,Project}: error when enabling dynamic runcommand but it's disabled elsewhere 2022-02-11 14:35:52 -05:00
Cole Helbling
d680c209fe edit-project.tt: disable when disabled by server 
Also add a tooltip describing why it's disabled, to make it easier to
chase down.
 2022-02-11 14:35:52 -05:00
Cole Helbling
6053e5fd4b edit-jobset.tt: disable when disabled by project and server 
Also add a tooltip describing why it's disabled, to make it easier to
chase down.
 2022-02-11 14:35:52 -05:00
Cole Helbling
dfd3a67424 project.tt: more info on why Dynamic RunCommand is disabled 2022-02-11 14:35:52 -05:00
Cole Helbling
3f4f183792 jobset.tt: more info on why Dynamic RunCommand is disabled 2022-02-11 14:35:52 -05:00
Graham Christensen
71c06f2ce7 LDAP normalization errors: note that the error came while normalizing the roles. 2022-02-11 10:55:27 -05:00
Graham Christensen
f07fb7d279 LDAP support: include BC support for the YAML based loading 
Includes a refactoring of the configuration loader.
 2022-02-11 10:49:38 -05:00
Janne Heß
61d74a7194 Redo LDAP config in the main configuration and add role mappings 2022-02-11 10:49:38 -05:00
Graham Christensen
d0bc0d0eda
Merge pull request #1152 from DeterminateSystems/parallel-tests 
Parallel tests, fix a hydra-queue-runner race condition
 2022-02-10 12:11:20 -05:00
Graham Christensen
4acaf9c8b0 hydra-queue-runner: don't dispatch until the machines parser has completed one run 
Periodically, I have seen tests fail because of out of order queue runner behavior:

    checking the queue for builds > 0...
    loading build 1 (tests:basic:empty_dir)
    aborting unsupported build step '...-empty-dir.drv' (type 'x86_64-linux')
    marking build 1 as failed
    adding new machine ‘localhost’

This patch should prevent the dispatcher from running before any machines are
made available.
 2022-02-10 10:54:30 -05:00
Graham Christensen
9ae7c8bddc Hydra::Helper::Exec add an expectOkay which dies with stdout / stderr on exit 2022-02-09 20:56:10 -05:00
Graham Christensen
845e6d4760 captureStdoutStderr*: move to Hydra::Helper::Exec which helps avoid some environment variable fixation problems 2022-02-09 14:28:50 -05:00
Graham Christensen
517dce285a eval_added event: change interface to traceID\tjobsetID\tevaluationID 
I was not going to break the interface until I noticed
the current implementation uses the string literal \t.
 2022-02-08 09:51:35 -05:00
Graham Christensen
d512e6220f eval_failed event: change interface to traceID\tjobsetID 
I was not going to break the interface until I noticed the other eval_* events used literal \ts
 2022-02-08 09:51:35 -05:00
Graham Christensen
2597fa8c11 eval_cached event: change interface to traceID\tjobsetID\tevaluationID 
I was not going to break the interface until I noticed
the current implementation uses the string literal \t.
 2022-02-08 09:51:35 -05:00
Graham Christensen
c30f084f32 eval_started event: change interface to traceID\tjobsetID 
I was not going to break the interface until I noticed
the current implementation uses the string literal \t.
 2022-02-08 09:51:35 -05:00
Graham Christensen
8a18326f2b Sort notification classes / events 2022-02-07 16:08:27 -05:00
Graham Christensen
d8b56f022d RunCommand: print a warning if the hook isn't run because the project / jobset doens't have it enabled 2022-02-01 10:58:54 -05:00
Graham Christensen
3aa2393091 Jobsets: add a supportsDynamicRunCommand which also checks the project's dynamic runcommand support 2022-02-01 10:58:54 -05:00
Graham Christensen
daa6864a58 Project result: add a supportsDynamicRunCommand helper 2022-02-01 10:58:54 -05:00
Graham Christensen
bc1630bd27 fixup! RunCommand: Add a WIP execution of dynamic commands 2022-02-01 10:58:54 -05:00
Graham Christensen
8a96f07f58 Project: enable enabling dynamic runcommand per project 2022-02-01 10:58:54 -05:00
Graham Christensen
1affb1cfb1 jobset API: expose and check the enable_dynamic_run_command 2022-02-01 10:58:54 -05:00
Graham Christensen
726ea80e99 HTTP/Jobset: support setting / reading enable_dynamic_run_command 2022-02-01 10:58:54 -05:00
Graham Christensen
1802bd0113 Declarative Jobs: add support for the enable_dynamic_run_command flag 2022-02-01 10:58:54 -05:00
Graham Christensen
0810f5debc finish making the dynamic hooks only run on project & jobset agreement 2022-02-01 10:58:54 -05:00
Graham Christensen
aef11685a0 regenerate schema files after adding the flag to the projects 2022-02-01 10:58:54 -05:00
Graham Christensen
85a53694c8 sql: add enable_dynamic_run_command to the Project as well 2022-02-01 10:58:54 -05:00
Graham Christensen
a9bfabd672 sql: add a migration for enable_dynamic_run_command 2022-02-01 10:58:23 -05:00
Graham Christensen
3cce0c5ef6 Only run dynamic runcommand hooks if the jobset enables them 2022-02-01 10:57:30 -05:00
Graham Christensen
97a1d2d1d4 Jobsets: add enable_dynamic_run_command 2022-02-01 10:57:30 -05:00
Graham Christensen
216d8bee35 DynamicRunCommand: don't run if the build failed 2022-02-01 10:57:30 -05:00
Graham Christensen
1a30a0c2f1 Dynamic RunCommand: validate that the job's out exists, is a file (or points to a file) which is executable. 2022-02-01 10:57:30 -05:00
Graham Christensen
e7f68045f4 DynamicRunCommand: pull out the function determining if a build is 
eligible for execution under dynamic run commands.
 2022-02-01 10:57:30 -05:00
Graham Christensen
e56c49333f RunCommand: Add a WIP execution of dynamic commands 
This in-progress feature will run a dynamically generated set of
buildFinished hooks, which must be nested under the `runCommandHook.*`
attribute set. This implementation is not very good, with some to-dos:

1. Only run if the build succeeded
2. Verify the output is named $out and that it is an executable file
   (or a symlink to a file)
3. Require the jobset itself have a flag enabling the feature, since
   this feature can be a bit dangerous if various people of different
   trust levels can create the jobs.
 2022-02-01 10:57:30 -05:00