145667cb53
hydra-update-gc-roots: allow cached refs to the build's jobset
...
Re-executing this search_related on every access turned out to
create very problematic performance. If a jobset had a lot of
error output stored in the jobset, and there were many hundreds
or thousands of active jobs, this could easily cause >1Gbps of
network traffic.
2022-03-19 23:56:47 -04:00
22026da4f8
Merge pull request #1176 from DeterminateSystems/broken-constituent
...
Broken constituents: emit useful log messages on evaluation errors on constituents
2022-03-19 14:55:17 -04:00
a582e4c485
HydraTestContext: add \n's to various dies
2022-03-19 14:46:53 -04:00
074a2f96bf
hydra-eval-jobset: emit a useful error if constituents errored
2022-03-19 14:37:12 -04:00
0c51de6334
hydra-evaluate-jobset: assert it logs errored constituents properly
2022-03-19 14:35:30 -04:00
25f6bae847
HydraTestContext: make it easy to create a jobset without evaluating
2022-03-19 14:34:43 -04:00
b0c17112c9
flake: update to nixos-unstable-small
...
https://github.com/NixOS/nixpkgs/pull/163695 was merged, so no longer
need to use my commit!
2022-03-18 11:10:57 -07:00
c0f826b92d
hydra-queue-runner: get the listening port from the exposer itself
...
Otherwise, when the port is randomly chosen (e.g. by specifying no port,
or a port of 0), it will just show that the port is 0 and not the port
that is actually serving the metrics.
2022-03-14 08:41:45 -07:00
52a29d43e6
hydra-queue-runner: make registry member of State, configurable metrics port
...
Thanks to the updated prometheus-cpp library, specifying a port of 0
will cause it to pick a random (available) port -- ideal for tests.
2022-03-11 11:58:10 -08:00
6e6475d860
flake: replace aliases with their proper names
...
Newer Nixpkgs have added a throw for these aliases.
2022-03-11 11:58:10 -08:00
a0cb73579d
flake: update newNixpkgs for newer prometheus-cpp
2022-03-11 11:58:10 -08:00
3bf31bd6a6
hydra-queue-runner: add simple "up" exporter
...
There are probably better ways to achieve this (and will likely need to
be refactored a bit to support further metrics).
2022-03-10 12:36:58 -08:00
5bbaa18a8f
flake: fix foreman execution
...
[vin@scadrial:~/workspace/vcs/hydra]$ foreman -h
Warning: the running version of Bundler (2.1.4) is older than the version that created the lockfile (2.2.20). We suggest you to upgrade to the version that created the lockfile by running `gem install bundler:2.2.20`.
Traceback (most recent call last):
2: from /nix/store/ycshcdssxcj9sjf6yzb1ydw4fcglf66y-foreman-0.87.2/bin/foreman:20:in `<main>'
1: from /nix/store/ggqacj06n6qfm1iww0bih9ph0j89wcna-bundler-2.1.4/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/lib/bundler/rubygems_integration.rb:413:in `block in replace_bin_path'
/nix/store/ggqacj06n6qfm1iww0bih9ph0j89wcna-bundler-2.1.4/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/lib/bundler/rubygems_integration.rb:374:in `block in replace_bin_path': can't find executable foreman for gem foreman. foreman is not currently included in the bundle, perhaps you meant to add it to your Gemfile? (Gem::Exception)
2022-03-10 12:26:13 -08:00
28b682b85b
Merge pull request #1170 from NixOS/dependabot/github_actions/actions/checkout-3
...
build(deps): bump actions/checkout from 2 to 3
2022-03-09 21:13:47 -05:00
b9ec3a41d6
build(deps): bump actions/checkout from 2 to 3
...
Bumps [actions/checkout](https://github.com/actions/checkout ) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases )
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md )
- [Commits](https://github.com/actions/checkout/compare/v2...v3 )
---
updated-dependencies:
- dependency-name: actions/checkout
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com >
2022-03-07 15:01:23 +00:00
a2546121f0
Merge pull request #1163 from DeterminateSystems/constituents
...
Constituents: Add a GC root for indirect aggregates
2022-02-21 20:42:42 -05:00
9316544abf
src/hydra-eval-jobs/hydra-eval-jobs.cc: .get<std::string> for drvPath
...
Co-authored-by: Kayla Fire <firestack@users.noreply.github.com >
2022-02-21 12:41:21 -05:00
290e0653ad
hydra-eval-jobs: GC root aggregate jobs
2022-02-20 12:28:40 -05:00
e0921eba0a
Create a basic test which verifies we can't delete the derivation of aggregate jobs
2022-02-20 12:28:40 -05:00
be46f02164
tests: relocate evaluator tests
2022-02-20 12:28:40 -05:00
5d169e3a2e
Add a test validating direct and indirect constituents
2022-02-20 12:28:40 -05:00
445bba337b
Make copyClosureTo take a regular C++ ref to the store
...
This is syntactically lighter wait, and demonstates there are no weird
dynamic lifetimes involved, just regular passing reference to callee
which it only borrows for the duration of the call.
2022-02-20 17:22:43 +00:00
f14c583ce5
Use copyClosure instead of computeFSClosure + copyPaths
...
It is more terse, and in the future it is possible `copyClosure` will
become more sophisticated.
2022-02-19 11:59:17 -05:00
dfb3eccfaa
Merge pull request #1140 from Ma27/nix-update
...
Update Nix to 2.6
2022-02-19 08:38:34 -05:00
4c41ca08e1
Merge pull request #1155 from helsinki-systems/fix/graph-readability
...
build-graphs: Fix readability in dark mode
2022-02-14 11:27:37 -05:00
1124230d9f
Merge pull request #1151 from DeterminateSystems/ldap-tests-inc-mapping
...
ldap: support configurable roles with backwards compatibility
2022-02-14 09:30:54 -05:00
6d146deaf0
build-graphs: Fix readability in dark mode
2022-02-13 14:00:17 +01:00
78e9872251
ldap.t: write the password to an external .conf file
2022-02-11 11:27:10 -05:00
848fb3b265
ldap-legacy.t: specify the root password manually
2022-02-11 11:26:56 -05:00
0bd4a75918
HydraTestContext: support running a sub before running hydra-init.
2022-02-11 11:26:27 -05:00
05ca71069f
ldap config: document putting the password in a separate file
2022-02-11 11:24:45 -05:00
e13d80f5cf
LDAPContext: take a root_password argument or generate one
2022-02-11 11:02:04 -05:00
86bb16d07b
LDAPContext: sort $self hash keys
2022-02-11 11:01:25 -05:00
6637c03985
fixup normalization error regex
2022-02-11 10:59:24 -05:00
71c06f2ce7
LDAP normalization errors: note that the error came while normalizing the roles.
2022-02-11 10:55:27 -05:00
d6dea39912
ldap_role_map.t: fixup indentation
2022-02-11 10:53:08 -05:00
185100adb8
docs: fixup
2022-02-11 10:50:58 -05:00
f07fb7d279
LDAP support: include BC support for the YAML based loading
...
Includes a refactoring of the configuration loader.
2022-02-11 10:49:38 -05:00
61d74a7194
Redo LDAP config in the main configuration and add role mappings
2022-02-11 10:49:38 -05:00
76b4b43ac5
Move ldap.t to a legacy-ldap.t, make ldap.t use the new format config.
2022-02-11 10:49:38 -05:00
d0bc0d0eda
Merge pull request #1152 from DeterminateSystems/parallel-tests
...
Parallel tests, fix a hydra-queue-runner race condition
2022-02-10 12:11:20 -05:00
74a29e8ea9
Merge pull request #1150 from DeterminateSystems/ldap-tests
...
LDAP: Test without a VM
2022-02-10 12:05:19 -05:00
4f9aea9434
t/Hydra/Plugin/gitea.t: explain why we loop a few times
...
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com >
2022-02-10 12:02:29 -05:00
a1579007f4
Update t/test.pl: fixup the list of env vars yath looks at for job count
2022-02-10 11:53:10 -05:00
b1b1b4b74b
t/lib/LDAPContext.pm: error message nit
...
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com >
2022-02-10 11:31:49 -05:00
33f4c4c13d
build-locally-with-substitutable-path.t: give nix-store --delete a bit more time to run
...
Under high load, like 64-128 tests at once, this can take more than a second.
2022-02-10 11:13:31 -05:00
e709a17508
gitea.t: try opening the file a few times
2022-02-10 11:01:09 -05:00
4acaf9c8b0
hydra-queue-runner: don't dispatch until the machines parser has completed one run
...
Periodically, I have seen tests fail because of out of order queue runner behavior:
checking the queue for builds > 0...
loading build 1 (tests:basic:empty_dir)
aborting unsupported build step '...-empty-dir.drv' (type 'x86_64-linux')
marking build 1 as failed
adding new machine ‘localhost’
This patch should prevent the dispatcher from running before any machines are
made available.
2022-02-10 10:54:30 -05:00
09652475bd
Set the YATH_JOB_COUNt to NIX_BUILD_CORES if no parallelism variables are set
2022-02-10 10:00:32 -05:00
5506962537
Test::Postgresql: update to a tagged fork to improve paralellism
2022-02-10 09:37:48 -05:00
