ahuston-0/hydra
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
4,223 Commits 6 Branches 0 Tags
cd925e876fd9d57231483b33332a6301889e5864
Commit Graph

4223 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Graham Christensen
c2be27e82b fanout.t: switch to makeAndEvaluateJobset 2022-02-01 10:57:30 -05:00
Graham Christensen
e7f68045f4 DynamicRunCommand: pull out the function determining if a build is 
eligible for execution under dynamic run commands.
 2022-02-01 10:57:30 -05:00
Graham Christensen
e56c49333f RunCommand: Add a WIP execution of dynamic commands 
This in-progress feature will run a dynamically generated set of
buildFinished hooks, which must be nested under the `runCommandHook.*`
attribute set. This implementation is not very good, with some to-dos:

1. Only run if the build succeeded
2. Verify the output is named $out and that it is an executable file
   (or a symlink to a file)
3. Require the jobset itself have a flag enabling the feature, since
   this feature can be a bit dangerous if various people of different
   trust levels can create the jobs.
 2022-02-01 10:57:30 -05:00
Graham Christensen
ea311a0eb4 RunCommand: enable the plugin if dynamicruncommand is set 2022-02-01 10:57:30 -05:00
Graham Christensen
6ffc93c01a RunCommand: write documentation for dynamic commands 2022-02-01 10:57:30 -05:00
Graham Christensen
4ea646130c RunCommand: split out documentation, fixup the matcher syntax 2022-02-01 10:57:30 -05:00
Graham Christensen
85b842e0ac Merge pull request #1137 from DeterminateSystems/runcommand-logs 
Store and display the output of RunCommands
 2022-01-31 16:26:31 -05:00
Cole Helbling
b57345ba1f hydra.sql: add IndexRunCommandLogsOnBuildID index 2022-01-31 12:56:34 -08:00
Cole Helbling
d0b6329aa8 sql/upgrade-81: remove unnecessary comment 2022-01-31 12:55:36 -08:00
Cole Helbling
9c4e6f78e7 hydra-module: don't bzip2 runcommand-logs 2022-01-31 12:55:36 -08:00
Cole Helbling
8c67e32480 RunCommand: ensure we reset the umask 2022-01-31 12:55:36 -08:00
Cole Helbling
34e4c119f4 build.tt: don't duplicate RunCommandLog buttons 2022-01-31 11:40:16 -08:00
Cole Helbling
61189ecca9 Helper/Nix: constructRunCommandLogPath: verify uuid is valid 
This shouldn't be possible normally, but it is possible to:

    $db->resultset('RunCommandLogs')->new({ uuid => "../etc/passwd" });

if you have access to the `$db`.
 2022-01-31 08:58:33 -08:00
Cole Helbling
e381751564 Helper/Nix: constructRunCommandLogPath: return undef in case of an error 
This allows us to give a web request to an invalid UUID a 404.
 2022-01-31 08:58:33 -08:00
Cole Helbling
2c6487b8d7 t/Helper: test constructRunCommandLogPath 2022-01-31 08:58:33 -08:00
Cole Helbling
8bf3cdbc67 t/Helper: switch to using test_context() 2022-01-31 08:58:33 -08:00
Cole Helbling
8eab7b8543 Helper/Nix: constructRunCommandLogPath: take RunCommandLog as input 
This way we ensure that it actually exists in the database, rather than
blindly trusting user-generated input.
 2022-01-31 08:58:33 -08:00
Cole Helbling
61914d56c6 runcommand-log.tt: escape the command 2022-01-31 08:58:33 -08:00
Cole Helbling
71bbb042db build.tt: link to the pretty, raw, and tail versions of the log 
Also split it out to a new div -- there are now 3 lines per
RunCommandLog -- the first saying when it started, the second saying how
long it ran for (or has been running), and the third with the buttons
for the pretty, raw, and tail versions of the log.
 2022-01-31 08:58:33 -08:00
Cole Helbling
3594ba942a Controller/Build: use showLog in view_runcommandlog 
This also adds the `runcommandlog` object to the stash so that we can
access its uuid as well as command run in order to display more useful
and specific information on the webpage.
 2022-01-31 08:58:33 -08:00
Cole Helbling
1d0076408b Controller/Build: pass log_uri to showLog in place of drvPath 
This way, we can reuse the `showLog` sub for other things, such as
`view_runcommandlog` (which doesn't have a drvPath attached).
 2022-01-31 08:58:33 -08:00
Cole Helbling
ff390e89a6 Controller/Build: remove unused parameter from showLog 2022-01-31 08:58:33 -08:00
Cole Helbling
38896db6b6 t/RunCommand: init http.t 
Test that we can indeed visit the pages of a valid runcommand log and
not of an invalid one.
 2022-01-31 08:58:33 -08:00
Cole Helbling
47c1f89d5a t/RunCommand: fixup basic.t to use uuid 2022-01-31 08:58:33 -08:00
Cole Helbling
fc3cf4ecb2 RunCommandLogs: identify and access via uuid 
Using a sha1 of the command combined with the build ID is not a
particularly good or unique identifier:

* A build could fail, be restarted, and then succeed -- assuming no
configuration changes, the sha1 hash of the command as well as the build
ID will be the same. This would lead to an overwritten log file.

* Allowing user input to influence filenames is not the best of ideas.
 2022-01-31 08:58:33 -08:00
Graham Christensen
dcb0c1425c RunCommandLogs: set a UUID automatically 2022-01-31 08:58:33 -08:00
Graham Christensen
89e8676d80 UUID4Tiny: init 2022-01-31 08:58:33 -08:00
Graham Christensen
cf49a05ff5 RunCommandLogs: add a uuid to each log entry 2022-01-31 08:58:33 -08:00
Graham Christensen
94ed9ed7ff Merge pull request #1136 from DeterminateSystems/github-status-cached-evals 
GithubStatus: try pushing statuses for cached buildqueued/buildfinished events
 2022-01-31 09:11:37 -05:00
Graham Christensen
3ae4b19d12 flake: update to postgres 13 for UUID commands 2022-01-28 13:07:11 -08:00
Cole Helbling
244300c1ad RunCommand: remove unused and problematic imports 
Since breaking the filename construction out to a helper function,
Hydra::Model::DB is no longer used. Importing Hydra::Helper::Nix,
however, has the potential to break tests, so just use the functions we
need without importing the entire module.
 2022-01-28 13:07:11 -08:00
Cole Helbling
fdf6f4d3da RunCommand: use IPC::Run3::run3 instead 
run3 just seems to do better handling for what we want to do, and
requires less deep-reaching changes to this plugin to get it to play
nice, as IPC::Run::run would.
 2022-01-28 13:07:11 -08:00
Cole Helbling
3e722f1d0a t/RunCommand: remove duplicate use 2022-01-28 13:07:11 -08:00
Cole Helbling
c8f3943d03 hydra-module: log files don't have a .drv extension 2022-01-28 13:07:11 -08:00
Cole Helbling
3432cd7636 build.tt: split runcommand logic across multiple lines 
Helps with readability.
 2022-01-28 13:07:11 -08:00
Cole Helbling
988e79c6e5 t/RunCommand: test that the log file exists on the filesystem 2022-01-28 13:07:11 -08:00
Cole Helbling
1554750acc RunCommand: use make_path over mkdir 
This will make all necessary parent directories a la `mkdir -p`.
 2022-01-28 13:03:15 -08:00
Cole Helbling
bf3c46ed43 RunCommand: use IPC::Run to spawn the command 
This allows `logPath`s with spaces and other characters that might
otherwise cause problems inside a `system()` call.
 2022-01-28 13:03:15 -08:00
Cole Helbling
bb16f4fb10 RunCommand: set umask when creating log paths 
This uses the somewhat restrictive umask of 0027 so that people outside
the user or group cannot read the files. This also helps to inhibit
TOCTOU where someone else has a handle to our file before we chmod it
and after we close it.
 2022-01-28 13:03:15 -08:00
Cole Helbling
5d3912962b RunCommand: use helper functions to ensure filenames and paths are the same 
Otherwise, it's possible someone updates the format in one place but not
the others, leading to broken or incorrect functionality.
 2022-01-28 13:03:15 -08:00
Cole Helbling
4a441b54ce hydra-module: /var/lib/hydra -> ${baseDir} 2022-01-28 13:03:15 -08:00
Cole Helbling
14090fbb86 runcommand-log.tt: init 2022-01-28 13:03:15 -08:00
Janne Heß
796ce165d4 RunCommand: Allow displaying command output 2022-01-28 13:03:15 -08:00
Janne Heß
4cb5e6cd94 RunCommand: Capture the output of the commands 2022-01-28 13:00:17 -08:00
Graham Christensen
ef362e92d1 GithubStatus: try pushing statuses for cached buildqueued/buildfinished events 2022-01-25 12:42:28 -05:00
Graham Christensen
001539c3d2 Merge pull request #1127 from NixOS/grahamc-patch-1 
Create a bug report issue template
 2022-01-25 09:57:17 -05:00
Graham Christensen
0a5f625746 Update bug_report.md 2022-01-24 20:36:08 -05:00
Graham Christensen
f6e86efc9f Merge pull request #1091 from Ma27/ssh-remote-store-location 
hydra-queue-runner: support store URIs declaring an alternate store location
 2022-01-24 14:10:54 -05:00
Graham Christensen
3a4ea6e563 Merge pull request #1124 from obsidiansystems/simplify--closure-of-path-set 
simplify, `computeFSClosure` can take a set now
 2022-01-24 14:09:35 -05:00
Graham Christensen
bb68b56f61 Merge pull request #1133 from helsinki-systems/doc/config-format 
doc: Document the file format of the config
 2022-01-21 20:49:18 -05:00