ahuston-0/hydra
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
4,340 Commits 6 Branches 0 Tags
1cb1e139c43101df18295c16bd0038aab91589f2
Commit Graph

4340 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
John Ericson
1cb1e139c4 Fix build (due to C++ API changes) 2025-04-07 11:12:12 -04:00
John Ericson
6b97e3ab7b flake.lock: Update to nix and nix-eval-jobs 2.27 
Flake lock file updates:

• Updated input 'nix':
    'github:NixOS/nix/e310c19a1aeb1ce1ed4d41d5ab2d02db596e0918?narHash=sha256-q/RgA4bB7zWai4oPySq9mch7qH14IEeom2P64SXdqHs%3D' (2025-02-18)
  → 'github:NixOS/nix/d0f98c76f962147610489e84c10033ca92e9c532?narHash=sha256-u6RhBWQ1XohTZ4Ub5ml1PTcaxQgtqFNng6Sohy1rojw%3D' (2025-04-07)
• Updated input 'nix-eval-jobs':
    'github:nix-community/nix-eval-jobs/f7418fc1fa45b96d37baa95ff3c016dd5be3876b?narHash=sha256-Lo4KFBNcY8tmBuCmEr2XV0IUZtxXHmbXPNLkov/QSU0%3D' (2025-03-26)
  → 'github:nix-community/nix-eval-jobs/62f9c9e8d00d2ff6ab27a6197ab459a8e0808e59?narHash=sha256-PypQspB7h7EENe4RQQUQj2Ay8J1%2BO49AKNO9JbAU4Ek%3D' (2025-04-07)
 2025-04-07 11:02:52 -04:00
Jörg Thalheim
cad08f87d2 Merge pull request #1458 from NixOS/meson 
docs: fix contribution guide for new meson-based build
 2025-03-29 15:37:06 +01:00
Jörg Thalheim
3fef32b364 gitignore hydra-data as created by foreman 2025-03-29 14:31:18 +00:00
Jörg Thalheim
ae18a7b3ae fix development workflow after switching to meson-based build 2025-03-29 14:31:18 +00:00
Jörg Thalheim
b657bcdfb7 Merge pull request #1457 from dermetfan/fix-1429 
hydra-eval-jobset: do not wait on n-e-j inside transaction
 2025-03-29 11:36:13 +01:00
Jörg Thalheim
3b4c4972c2 Merge pull request #1449 from knedlsepp/fix-metrics-rendering-with-special-characters 
Fix rendering of metrics with special characters
 2025-03-29 08:52:07 +01:00
John Ericson
b2fe3f5218 Merge pull request #1455 from qowoz/226-constituent 
nix-eval-jobs + constituent globs
 2025-03-27 23:18:39 -04:00
Maximilian Bosch
9911f0107f Reimplement (named) constituent jobs (+globbing) based on nix-eval-jobs 
Depends on https://github.com/nix-community/nix-eval-jobs/pull/349 & #1421.

Almost equivalent to #1425, but with a small change: when having e.g. an
aggregate job with a glob that matches nothing, the jobset evaluation is
failed now. This was the intended behavior before (hydra-eval-jobset
fails hard if an aggregate is broken), the code-path was never reached
however since the aggregate was never marked as broken in this case
before.
 2025-03-28 11:12:54 +10:00
zowoq
feebb61897 flake.lock: Update 
Flake lock file updates:

• Updated input 'nix-eval-jobs':
    'github:nix-community/nix-eval-jobs/4b392b284877d203ae262e16af269f702df036bc?narHash=sha256-3wIReAqdTALv39gkWXLMZQvHyBOc3yPkWT2ZsItxedY%3D' (2025-02-14)
  → 'github:nix-community/nix-eval-jobs/f7418fc1fa45b96d37baa95ff3c016dd5be3876b?narHash=sha256-Lo4KFBNcY8tmBuCmEr2XV0IUZtxXHmbXPNLkov/QSU0%3D' (2025-03-26)
 2025-03-28 11:12:54 +10:00
zowoq
4bcbed2f1b hydraTest: remove outdated postgresql version 
error: postgresql_12 has been removed since it reached its EOL upstream
 2025-03-28 11:12:48 +10:00
Robin Stumm
987dad3371 hydra-eval-jobset: do not wait on n-e-j inside transaction 
fixes #1429
 2025-03-26 20:23:26 +01:00
John Ericson
d2db3c7446 Merge pull request #1450 from NixOS/hydra-compress-race 
Fix race condition in hydra-compress-logs
 2025-03-16 14:37:39 -04:00
John Ericson
97dcdae068 Merge pull request #1451 from NixOS/revert-to-fix-hangs 
Revert "Use `LegacySSHStore`"
 2025-03-03 10:18:28 -05:00
John Ericson
9a5bd39d4c Revert "Use LegacySSHStore" 
There were some hangs caused by this. Need to fix them, ideally
reproducing the issue in a test, before trying this again.

This reverts commit 4a4a0f901c.
 2025-03-03 10:12:38 -05:00
Martin Weinelt
f1deb22c02 Fix race condition in hydra-compress-logs 2025-03-02 03:08:26 +01:00
Josef Kemetmüller
d22d030503 Fix rendering of metrics with special characters 
My main motivation here is to get metrics with brackets to work in order
to support "pytest" test names:

- test_foo.py::test_bar[1]
- test_foo.py::test_bar[2]

I couldn't find an "HTML escape"-style function that would generate
valid html `id` attribute names from random strings, so I went with a
hash digest instead.
 2025-02-27 09:25:42 +01:00
John Ericson
18c0d76210 Merge pull request #1444 from NixOS/use-legacy-ssh-store 
Use `LegacySSHStore`
 2025-02-18 14:37:17 -05:00
John Ericson
4a4a0f901c Use LegacySSHStore 
In https://github.com/NixOS/nix/pull/10748 it is extended with
everything we need.
 2025-02-18 14:07:42 -05:00
John Ericson
881462bb4e Merge pull request #1447 from NixOS/newer-2.6 
Bump to newer 2.26.* Nix version
 2025-02-18 13:00:40 -05:00
John Ericson
af72b694d8 Bump to newer 2.26.* Nix version 
Needed one more thing before trying out using `LegacySSHStore` directly.

Flake lock file updates:

• Updated input 'nix':
    'github:NixOS/nix/674a87462cb93f605d4fbeef607d3453e7e5a7d8?narHash=sha256-TBoHqnIdVWhsBcL05vO2B1hSl9m//5Mz2NU%2BPMk3h3Y%3D' (2025-02-16)
  → 'github:NixOS/nix/e310c19a1aeb1ce1ed4d41d5ab2d02db596e0918?narHash=sha256-q/RgA4bB7zWai4oPySq9mch7qH14IEeom2P64SXdqHs%3D' (2025-02-18)
 2025-02-18 12:43:31 -05:00
John Ericson
c92342d12f Merge pull request #1446 from NixOS/newer-2.6 
Bump to newer 2.26.* Nix version
 2025-02-16 19:10:10 -05:00
John Ericson
df07670a21 Bump to newer 2.26.* Nix version 
Flake lock file updates:

• Updated input 'nix':
    'github:NixOS/nix/970942f45836172fda410a638853382952189eb9?narHash=sha256-jGFuyYKJjJZsBRoi7ZcaVKt1OYxusz/ld1HA7VD2w/0%3D' (2025-02-12)
  → 'github:NixOS/nix/674a87462cb93f605d4fbeef607d3453e7e5a7d8?narHash=sha256-TBoHqnIdVWhsBcL05vO2B1hSl9m//5Mz2NU%2BPMk3h3Y%3D' (2025-02-16)
 2025-02-16 18:44:32 -05:00
John Ericson
51944a5fa5 Merge pull request #1443 from NixOS/nix-2.26 
Nix 2.26
 2025-02-13 22:13:32 -05:00
John Ericson
341b2f1309 Update build system to depend on Nix 2.26 2025-02-13 21:54:35 -05:00
John Ericson
4dc0f11379 Update flake.nix for Nix 2.26 
Flake lock file updates:

• Removed input 'libgit2'
• Updated input 'nix':
    'github:NixOS/nix/d652513e4519ed4eb48c92f8670e5a71c7793fc3?narHash=sha256-mIpJgIwPS4o4xYhN1B%2B/fHESEXoxpu6nVoZTzZ0MfTg%3D' (2025-02-12)
  → 'github:NixOS/nix/970942f45836172fda410a638853382952189eb9?narHash=sha256-jGFuyYKJjJZsBRoi7ZcaVKt1OYxusz/ld1HA7VD2w/0%3D' (2025-02-12)
• Removed input 'nix/libgit2'
• Updated input 'nix-eval-jobs':
    'github:nix-community/nix-eval-jobs/6d4fd5a93d7bc953ffa4dcd6d53ad7056a71eff7?narHash=sha256-1dZLPw%2BnlFQzzswfyTxW%2B8VF1AJ4ZvoYvLTjlHiz1SA%3D' (2025-02-13)
  → 'github:nix-community/nix-eval-jobs/4b392b284877d203ae262e16af269f702df036bc?narHash=sha256-3wIReAqdTALv39gkWXLMZQvHyBOc3yPkWT2ZsItxedY%3D' (2025-02-14)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/dbebdd67a6006bb145d98c8debf9140ac7e651d0?narHash=sha256-Xc9lEtentPCEtxc/F1e6jIZsd4MPDYv4Kugl9WtXlz0%3D' (2024-09-18)
  → 'github:NixOS/nixpkgs/97a719c9f0a07923c957cf51b20b329f9fb9d43f?narHash=sha256-1o1qR0KYozYGRrnqytSpAhVBYLNBHX%2BLv6I39zGRzKM%3D' (2025-02-13)
 2025-02-13 21:54:31 -05:00
John Ericson
ea09952b7e Merge pull request #1442 from NixOS/clean-up-flake-lockfile 
Clean up flake lockfile stuff
 2025-02-13 20:52:40 -05:00
John Ericson
81d21979ef Clean up flake lockfile stuff 
The `flake = false;` for `nix-eval-jobs` didn't fully take before.

Flake lock file updates:

• Removed input 'nix-eval-jobs/flake-parts'
• Removed input 'nix-eval-jobs/flake-parts/nixpkgs-lib'
• Removed input 'nix-eval-jobs/nix-github-actions'
• Removed input 'nix-eval-jobs/nixpkgs'
• Removed input 'nix-eval-jobs/treefmt-nix'
• Removed input 'nix-eval-jobs/treefmt-nix/nixpkgs'
 2025-02-13 20:23:08 -05:00
John Ericson
0ed9a82912 Merge pull request #1441 from NixOS/nix-2.25 
Nix 2.25
 2025-02-13 19:53:07 -05:00
John Ericson
80241fc8be Make code change necessary for building with Nix 2.25 2025-02-13 19:10:09 -05:00
John Ericson
4347833f45 Rework to synchronize deps 
Bypass `nix-eval-job`'s flake, and just call-package it, for
fine-grained control.
 2025-02-13 19:03:37 -05:00
John Ericson
8835cbd10f flake.lock: Update 
Flake lock file updates:

• Updated input 'nix':
    'github:NixOS/nix/a7fdef6858dd45b9d7bda7c92324c63faee7f509?narHash=sha256-XFznzb8L4SdUm9u%2Bw3DPpMWJhffuv%2B/6%2BaiVl00slns%3D' (2024-09-19)
  → 'github:NixOS/nix/d652513e4519ed4eb48c92f8670e5a71c7793fc3?narHash=sha256-mIpJgIwPS4o4xYhN1B%2B/fHESEXoxpu6nVoZTzZ0MfTg%3D' (2025-02-12)
• Updated input 'nix-eval-jobs':
    'github:nix-community/nix-eval-jobs/889ea1406736b53cf165b6c28398aae3969418d1?narHash=sha256-3wwtKpS5tUBdjaGeSia7CotonbiRB6K5Kp0dsUt3nzU%3D' (2024-12-10)
  → 'github:nix-community/nix-eval-jobs/6d4fd5a93d7bc953ffa4dcd6d53ad7056a71eff7?narHash=sha256-1dZLPw%2BnlFQzzswfyTxW%2B8VF1AJ4ZvoYvLTjlHiz1SA%3D' (2025-02-13)
 2025-02-13 18:46:36 -05:00
John Ericson
9ad8ac586c Merge pull request #1440 from NixOS/legacy-ssh-expose-ssh-master 
Use new `CommonSSHStoreConfig::createSSHMaster`
 2025-02-13 18:30:41 -05:00
John Ericson
9a6928d93b Use new CommonSSHStoreConfig::createSSHMaster 
This avoids some duplicated code, leveraging the same `StoreReference`
type that also undergirds the machine file dedup we just did prior.

By using `LegacySSHStoreConfig`, we're also taking a baby step towards
using the store interface rather than messing around with the protocol
internals.
 2025-02-13 18:13:38 -05:00
John Ericson
810781a802 Merge pull request #1439 from NixOS/nix-next 
Dedup machine file parsing, and other improvements
 2025-02-13 18:10:08 -05:00
John Ericson
af9b0663f2 Merge branch 'master' into nix-next 2025-02-13 17:54:15 -05:00
Jörg Thalheim
c6f98202cd Merge pull request #1438 from NixOS/log-malformed-json 
Log malformed JSON received from `nix-eval-jobs`
 2025-02-12 12:58:18 +07:00
John Ericson
1dbc7f5845 Log malformed JSON received from nix-eval-jobs 2025-02-11 22:34:49 -05:00
John Ericson
c52845f560 Merge pull request #1421 from NixOS/nix-eval-jobs 
Use `nix-eval-jobs` and delete `hydra-eval-jobs`
 2025-02-07 19:41:38 -05:00
John Ericson
85383b9522 Render the nix-eval-jobs version too 2025-02-07 16:55:28 -05:00
Pierre Bourdon
2f92846e5a hydra-eval-jobs: remove, replaced by nix-eval-jobs 
(cherry picked from commit ed7c58708cd3affd62a598a22a500ed2adf318bf)
 2025-02-07 16:55:28 -05:00
Pierre Bourdon
d84ff32ce6 hydra-eval-jobset: Use nix-eval-jobs instead of hydra-eval-jobs 
incrementally ingest eval results

nix-eval-jobs streams output, unlike hydra-eval-jobs. Now that we've
migrated, we can use this to:

1. Use less RAM by avoiding buffering a whole eval's worth of metadata
   into a Perl string and an array of JSON objects.
2. Make evals latency a bit lower by allowing the queue runner to start
   ingesting builds faster.

Also use the newly-restored constituents support in `nix-eval-jobs`

Note, we pass --workers and --max-memory-size to n-e-j

Lost in the h-e-j -> n-e-j migration, causing evaluation to always be
single threaded and limited to 4GiB RAM. Follow the config settings like
h-e-j used to do (via C++ code).

`nix-eval-jobs` should check `hydraJobs` and then `checks` with flakes

(cherry picked from commit 6d4ccff43c41adaf6e4b2b9bced7243bc2f6e97b)
(cherry picked from commit b0e9b4b2f99f9d8f5c4e780e89f955c394b5ced4)
(cherry picked from commit cdfc5c81e8037d3e4818a3e459d0804b2c157ea9)
(cherry picked from commit 4b107e6ff36bd89958fba36e0fe0340903e7cd13)

Co-Authored-By: Maximilian Bosch <maximilian@mbosch.me>
 2025-02-07 16:55:28 -05:00
Pierre Bourdon
0c9726af59 flake: add nix-eval-jobs as input 
(cherry picked from commit 684cc50d86608cccf7500ce00af89ea34c488473)
 2025-02-07 16:55:28 -05:00
John Ericson
5100b85537 Merge pull request #1436 from obsidiansystems/test-aliased-constituents 
Improve tests around constituents
 2025-02-07 16:45:17 -05:00
John Ericson
141b5fd0b5 Improve tests around constituents 
- Test how shorter names are preferred when multiple jobs resolve to the
  same derivation.

- Test the exact aggregate map we get, by looking in the DB.
 2025-02-07 16:39:13 -05:00
John Ericson
8d78648e65 Merge pull request #1435 from obsidiansystems/flake-tests 
Test using Hydra with flakes
 2025-02-07 11:21:02 -05:00
John Ericson
8a8ac14877 Test using Hydra with flakes 
It seemed there was no self-contained end-to-end test actually doing
this?!

Among other things, this will help ensure that the switch-over to
`nix-eval-jobs` is correct.
 2025-02-06 21:30:49 -05:00
John Ericson
250668a19f Merge pull request #1426 from NixOS/queue-runner-wants-after 
Make hydra-queue-runner want network-online.target
 2024-12-05 19:28:15 -05:00
Martin Weinelt
efadb6a26c Make hydra-queue-runner want network-online.target 
Just ordering yourself after network-online.target will not guarantee
that it will be loaded. You'll have to either want or require it. Hence
the following trace on recent nixpkgs versions:

evaluation warning: hydra-queue-runner.service is ordered after 'network-online.target' but doesn't depend on it
 2024-12-03 01:44:55 +01:00
Janne Heß
3b16941b14 Merge pull request #1424 from AsterisMono/fix-darwin-tmp-path 
reproduce.tt: Use realpath for tmpDir to fix macOS compatibility
 2024-11-26 08:53:13 +01:00