ahuston-0/hydra
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity
4,343 Commits 3 Branches 0 Tags
1ca17faed4dccb94f217d054510497be670efae4
Commit Graph

4343 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Pierre Bourdon
1ca17faed4 web: include current step status on /machines 2025-04-09 11:31:47 -04:00
John Ericson
9c022848cf Fix the build 2025-04-09 11:31:47 -04:00
John Ericson
f58a752419 Fix Nix code 
Can now at least enter dev shell, but build is still broken.
 2025-04-09 11:31:47 -04:00
John Ericson
0769853dec flake.lock: Update to nix and nix-eval-jobs 2.28 
Flake lock file updates:

• Updated input 'nix':
    'github:NixOS/nix/d0f98c76f962147610489e84c10033ca92e9c532?narHash=sha256-u6RhBWQ1XohTZ4Ub5ml1PTcaxQgtqFNng6Sohy1rojw%3D' (2025-04-07)
  → 'github:NixOS/nix/a4962f73b5fc874d4b16baef47921daf349addfc?narHash=sha256-r%2BpsCOW77vTSTNbxTVrYHeh6OgB0QukbnyUVDwg8s4I%3D' (2025-04-07)
• Updated input 'nix-eval-jobs':
    'github:nix-community/nix-eval-jobs/62f9c9e8d00d2ff6ab27a6197ab459a8e0808e59?narHash=sha256-PypQspB7h7EENe4RQQUQj2Ay8J1%2BO49AKNO9JbAU4Ek%3D' (2025-04-07)
  → 'github:nix-community/nix-eval-jobs/cba718bafe5dc1607c2b6761ecf53c641a6f3b21?narHash=sha256-v5n6t49X7MOpqS9j0FtI6TWOXvxuZMmGsp2OfUK5QfA%3D' (2025-04-07)
 2025-04-09 11:31:47 -04:00
John Ericson
21c6afa83b Fix build (due to C++ API changes) 2025-04-09 11:31:47 -04:00
John Ericson
1022514027 flake.lock: Update to nix and nix-eval-jobs 2.27 
Flake lock file updates:

• Updated input 'nix':
    'github:NixOS/nix/e310c19a1aeb1ce1ed4d41d5ab2d02db596e0918?narHash=sha256-q/RgA4bB7zWai4oPySq9mch7qH14IEeom2P64SXdqHs%3D' (2025-02-18)
  → 'github:NixOS/nix/d0f98c76f962147610489e84c10033ca92e9c532?narHash=sha256-u6RhBWQ1XohTZ4Ub5ml1PTcaxQgtqFNng6Sohy1rojw%3D' (2025-04-07)
• Updated input 'nix-eval-jobs':
    'github:nix-community/nix-eval-jobs/f7418fc1fa45b96d37baa95ff3c016dd5be3876b?narHash=sha256-Lo4KFBNcY8tmBuCmEr2XV0IUZtxXHmbXPNLkov/QSU0%3D' (2025-03-26)
  → 'github:nix-community/nix-eval-jobs/62f9c9e8d00d2ff6ab27a6197ab459a8e0808e59?narHash=sha256-PypQspB7h7EENe4RQQUQj2Ay8J1%2BO49AKNO9JbAU4Ek%3D' (2025-04-07)
 2025-04-09 11:31:47 -04:00
Jörg Thalheim
2d4232475c gitignore hydra-data as created by foreman 2025-04-09 11:31:47 -04:00
Jörg Thalheim
d799742057 fix development workflow after switching to meson-based build 2025-04-09 11:31:47 -04:00
Robin Stumm
485aa93f2d hydra-eval-jobset: do not wait on n-e-j inside transaction 
fixes #1429
 2025-04-09 11:31:47 -04:00
Josef Kemetmüller
590e8d8511 Fix rendering of metrics with special characters 
My main motivation here is to get metrics with brackets to work in order
to support "pytest" test names:

- test_foo.py::test_bar[1]
- test_foo.py::test_bar[2]

I couldn't find an "HTML escape"-style function that would generate
valid html `id` attribute names from random strings, so I went with a
hash digest instead.
 2025-04-09 11:31:47 -04:00
Maximilian Bosch
90a8a0d94a Reimplement (named) constituent jobs (+globbing) based on nix-eval-jobs 
Depends on https://github.com/nix-community/nix-eval-jobs/pull/349 & #1421.

Almost equivalent to #1425, but with a small change: when having e.g. an
aggregate job with a glob that matches nothing, the jobset evaluation is
failed now. This was the intended behavior before (hydra-eval-jobset
fails hard if an aggregate is broken), the code-path was never reached
however since the aggregate was never marked as broken in this case
before.
 2025-04-09 11:31:47 -04:00
zowoq
eb17619ee5 flake.lock: Update 
Flake lock file updates:

• Updated input 'nix-eval-jobs':
    'github:nix-community/nix-eval-jobs/4b392b284877d203ae262e16af269f702df036bc?narHash=sha256-3wIReAqdTALv39gkWXLMZQvHyBOc3yPkWT2ZsItxedY%3D' (2025-02-14)
  → 'github:nix-community/nix-eval-jobs/f7418fc1fa45b96d37baa95ff3c016dd5be3876b?narHash=sha256-Lo4KFBNcY8tmBuCmEr2XV0IUZtxXHmbXPNLkov/QSU0%3D' (2025-03-26)
 2025-04-09 11:31:47 -04:00
zowoq
ebefdb0a3d hydraTest: remove outdated postgresql version 
error: postgresql_12 has been removed since it reached its EOL upstream
 2025-04-09 11:31:47 -04:00
Martin Weinelt
55349930f1 Fix race condition in hydra-compress-logs 2025-04-09 11:31:47 -04:00
John Ericson
847a8ae6cd Revert "Use LegacySSHStore" 
There were some hangs caused by this. Need to fix them, ideally
reproducing the issue in a test, before trying this again.

This reverts commit 4a4a0f901c.
 2025-04-09 11:31:47 -04:00
ahuston-0
86d0009448 add declaritive hydra spec 2025-04-01 15:02:44 -04:00
ahuston-0
a20f37b97f add gitea refs 
Signed-off-by: ahuston-0 <aliceghuston@gmail.com>
Reviewed-on: https://<censored>/ahuston-0/hydra/pulls/1
 2025-03-31 14:52:51 -04:00
ahuston-0
a94f84118c add Gitea pulls docs entry 
Signed-off-by: ahuston-0 <aliceghuston@gmail.com>
 2025-03-31 14:52:51 -04:00
Faye Chun
99e3ad325c Merge branch 'NixOS:master' into add-gitea-pulls 2025-03-01 22:04:13 -05:00
John Ericson
18c0d76210 Merge pull request #1444 from NixOS/use-legacy-ssh-store 
Use `LegacySSHStore`
 2025-02-18 14:37:17 -05:00
John Ericson
4a4a0f901c Use LegacySSHStore 
In https://github.com/NixOS/nix/pull/10748 it is extended with
everything we need.
 2025-02-18 14:07:42 -05:00
John Ericson
881462bb4e Merge pull request #1447 from NixOS/newer-2.6 
Bump to newer 2.26.* Nix version
 2025-02-18 13:00:40 -05:00
John Ericson
af72b694d8 Bump to newer 2.26.* Nix version 
Needed one more thing before trying out using `LegacySSHStore` directly.

Flake lock file updates:

• Updated input 'nix':
    'github:NixOS/nix/674a87462cb93f605d4fbeef607d3453e7e5a7d8?narHash=sha256-TBoHqnIdVWhsBcL05vO2B1hSl9m//5Mz2NU%2BPMk3h3Y%3D' (2025-02-16)
  → 'github:NixOS/nix/e310c19a1aeb1ce1ed4d41d5ab2d02db596e0918?narHash=sha256-q/RgA4bB7zWai4oPySq9mch7qH14IEeom2P64SXdqHs%3D' (2025-02-18)
 2025-02-18 12:43:31 -05:00
John Ericson
c92342d12f Merge pull request #1446 from NixOS/newer-2.6 
Bump to newer 2.26.* Nix version
 2025-02-16 19:10:10 -05:00
John Ericson
df07670a21 Bump to newer 2.26.* Nix version 
Flake lock file updates:

• Updated input 'nix':
    'github:NixOS/nix/970942f45836172fda410a638853382952189eb9?narHash=sha256-jGFuyYKJjJZsBRoi7ZcaVKt1OYxusz/ld1HA7VD2w/0%3D' (2025-02-12)
  → 'github:NixOS/nix/674a87462cb93f605d4fbeef607d3453e7e5a7d8?narHash=sha256-TBoHqnIdVWhsBcL05vO2B1hSl9m//5Mz2NU%2BPMk3h3Y%3D' (2025-02-16)
 2025-02-16 18:44:32 -05:00
John Ericson
51944a5fa5 Merge pull request #1443 from NixOS/nix-2.26 
Nix 2.26
 2025-02-13 22:13:32 -05:00
John Ericson
341b2f1309 Update build system to depend on Nix 2.26 2025-02-13 21:54:35 -05:00
John Ericson
4dc0f11379 Update flake.nix for Nix 2.26 
Flake lock file updates:

• Removed input 'libgit2'
• Updated input 'nix':
    'github:NixOS/nix/d652513e4519ed4eb48c92f8670e5a71c7793fc3?narHash=sha256-mIpJgIwPS4o4xYhN1B%2B/fHESEXoxpu6nVoZTzZ0MfTg%3D' (2025-02-12)
  → 'github:NixOS/nix/970942f45836172fda410a638853382952189eb9?narHash=sha256-jGFuyYKJjJZsBRoi7ZcaVKt1OYxusz/ld1HA7VD2w/0%3D' (2025-02-12)
• Removed input 'nix/libgit2'
• Updated input 'nix-eval-jobs':
    'github:nix-community/nix-eval-jobs/6d4fd5a93d7bc953ffa4dcd6d53ad7056a71eff7?narHash=sha256-1dZLPw%2BnlFQzzswfyTxW%2B8VF1AJ4ZvoYvLTjlHiz1SA%3D' (2025-02-13)
  → 'github:nix-community/nix-eval-jobs/4b392b284877d203ae262e16af269f702df036bc?narHash=sha256-3wIReAqdTALv39gkWXLMZQvHyBOc3yPkWT2ZsItxedY%3D' (2025-02-14)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/dbebdd67a6006bb145d98c8debf9140ac7e651d0?narHash=sha256-Xc9lEtentPCEtxc/F1e6jIZsd4MPDYv4Kugl9WtXlz0%3D' (2024-09-18)
  → 'github:NixOS/nixpkgs/97a719c9f0a07923c957cf51b20b329f9fb9d43f?narHash=sha256-1o1qR0KYozYGRrnqytSpAhVBYLNBHX%2BLv6I39zGRzKM%3D' (2025-02-13)
 2025-02-13 21:54:31 -05:00
John Ericson
ea09952b7e Merge pull request #1442 from NixOS/clean-up-flake-lockfile 
Clean up flake lockfile stuff
 2025-02-13 20:52:40 -05:00
John Ericson
81d21979ef Clean up flake lockfile stuff 
The `flake = false;` for `nix-eval-jobs` didn't fully take before.

Flake lock file updates:

• Removed input 'nix-eval-jobs/flake-parts'
• Removed input 'nix-eval-jobs/flake-parts/nixpkgs-lib'
• Removed input 'nix-eval-jobs/nix-github-actions'
• Removed input 'nix-eval-jobs/nixpkgs'
• Removed input 'nix-eval-jobs/treefmt-nix'
• Removed input 'nix-eval-jobs/treefmt-nix/nixpkgs'
 2025-02-13 20:23:08 -05:00
John Ericson
0ed9a82912 Merge pull request #1441 from NixOS/nix-2.25 
Nix 2.25
 2025-02-13 19:53:07 -05:00
John Ericson
80241fc8be Make code change necessary for building with Nix 2.25 2025-02-13 19:10:09 -05:00
John Ericson
4347833f45 Rework to synchronize deps 
Bypass `nix-eval-job`'s flake, and just call-package it, for
fine-grained control.
 2025-02-13 19:03:37 -05:00
John Ericson
8835cbd10f flake.lock: Update 
Flake lock file updates:

• Updated input 'nix':
    'github:NixOS/nix/a7fdef6858dd45b9d7bda7c92324c63faee7f509?narHash=sha256-XFznzb8L4SdUm9u%2Bw3DPpMWJhffuv%2B/6%2BaiVl00slns%3D' (2024-09-19)
  → 'github:NixOS/nix/d652513e4519ed4eb48c92f8670e5a71c7793fc3?narHash=sha256-mIpJgIwPS4o4xYhN1B%2B/fHESEXoxpu6nVoZTzZ0MfTg%3D' (2025-02-12)
• Updated input 'nix-eval-jobs':
    'github:nix-community/nix-eval-jobs/889ea1406736b53cf165b6c28398aae3969418d1?narHash=sha256-3wwtKpS5tUBdjaGeSia7CotonbiRB6K5Kp0dsUt3nzU%3D' (2024-12-10)
  → 'github:nix-community/nix-eval-jobs/6d4fd5a93d7bc953ffa4dcd6d53ad7056a71eff7?narHash=sha256-1dZLPw%2BnlFQzzswfyTxW%2B8VF1AJ4ZvoYvLTjlHiz1SA%3D' (2025-02-13)
 2025-02-13 18:46:36 -05:00
John Ericson
9ad8ac586c Merge pull request #1440 from NixOS/legacy-ssh-expose-ssh-master 
Use new `CommonSSHStoreConfig::createSSHMaster`
 2025-02-13 18:30:41 -05:00
John Ericson
9a6928d93b Use new CommonSSHStoreConfig::createSSHMaster 
This avoids some duplicated code, leveraging the same `StoreReference`
type that also undergirds the machine file dedup we just did prior.

By using `LegacySSHStoreConfig`, we're also taking a baby step towards
using the store interface rather than messing around with the protocol
internals.
 2025-02-13 18:13:38 -05:00
John Ericson
810781a802 Merge pull request #1439 from NixOS/nix-next 
Dedup machine file parsing, and other improvements
 2025-02-13 18:10:08 -05:00
John Ericson
af9b0663f2 Merge branch 'master' into nix-next 2025-02-13 17:54:15 -05:00
Jörg Thalheim
c6f98202cd Merge pull request #1438 from NixOS/log-malformed-json 
Log malformed JSON received from `nix-eval-jobs`
 2025-02-12 12:58:18 +07:00
John Ericson
1dbc7f5845 Log malformed JSON received from nix-eval-jobs 2025-02-11 22:34:49 -05:00
John Ericson
c52845f560 Merge pull request #1421 from NixOS/nix-eval-jobs 
Use `nix-eval-jobs` and delete `hydra-eval-jobs`
 2025-02-07 19:41:38 -05:00
John Ericson
85383b9522 Render the nix-eval-jobs version too 2025-02-07 16:55:28 -05:00
Pierre Bourdon
2f92846e5a hydra-eval-jobs: remove, replaced by nix-eval-jobs 
(cherry picked from commit ed7c58708cd3affd62a598a22a500ed2adf318bf)
 2025-02-07 16:55:28 -05:00
Pierre Bourdon
d84ff32ce6 hydra-eval-jobset: Use nix-eval-jobs instead of hydra-eval-jobs 
incrementally ingest eval results

nix-eval-jobs streams output, unlike hydra-eval-jobs. Now that we've
migrated, we can use this to:

1. Use less RAM by avoiding buffering a whole eval's worth of metadata
   into a Perl string and an array of JSON objects.
2. Make evals latency a bit lower by allowing the queue runner to start
   ingesting builds faster.

Also use the newly-restored constituents support in `nix-eval-jobs`

Note, we pass --workers and --max-memory-size to n-e-j

Lost in the h-e-j -> n-e-j migration, causing evaluation to always be
single threaded and limited to 4GiB RAM. Follow the config settings like
h-e-j used to do (via C++ code).

`nix-eval-jobs` should check `hydraJobs` and then `checks` with flakes

(cherry picked from commit 6d4ccff43c41adaf6e4b2b9bced7243bc2f6e97b)
(cherry picked from commit b0e9b4b2f99f9d8f5c4e780e89f955c394b5ced4)
(cherry picked from commit cdfc5c81e8037d3e4818a3e459d0804b2c157ea9)
(cherry picked from commit 4b107e6ff36bd89958fba36e0fe0340903e7cd13)

Co-Authored-By: Maximilian Bosch <maximilian@mbosch.me>
 2025-02-07 16:55:28 -05:00
Pierre Bourdon
0c9726af59 flake: add nix-eval-jobs as input 
(cherry picked from commit 684cc50d86608cccf7500ce00af89ea34c488473)
 2025-02-07 16:55:28 -05:00
John Ericson
5100b85537 Merge pull request #1436 from obsidiansystems/test-aliased-constituents 
Improve tests around constituents
 2025-02-07 16:45:17 -05:00
John Ericson
141b5fd0b5 Improve tests around constituents 
- Test how shorter names are preferred when multiple jobs resolve to the
  same derivation.

- Test the exact aggregate map we get, by looking in the DB.
 2025-02-07 16:39:13 -05:00
John Ericson
8d78648e65 Merge pull request #1435 from obsidiansystems/flake-tests 
Test using Hydra with flakes
 2025-02-07 11:21:02 -05:00
John Ericson
8a8ac14877 Test using Hydra with flakes 
It seemed there was no self-contained end-to-end test actually doing
this?!

Among other things, this will help ensure that the switch-over to
`nix-eval-jobs` is correct.
 2025-02-06 21:30:49 -05:00
Faye Chun
2f1fa2b069 Add a plugin to poll Gitea pull requests 
Based off the existing GithubPulls.pm and GitlabPulls.pm plugins.

Also adds an integration test for the new 'giteapulls' input type to
the existing 'gitea' test.
 2024-12-21 08:02:57 -05:00