Compare commits

...

4 Commits

Author SHA1 Message Date
github-actions[bot]
bd668ad109 automated: Update flake.lock
Some checks failed
Check flake.lock / Check health of `flake.lock` (pull_request) Successful in 18s
Check Nix formatting / Perform Nix format checks (pull_request) Successful in 3m30s
Check Nix flake / Build nix outputs (ubuntu-latest) (pull_request) Successful in 5m41s
Check Nix flake / Perform Nix flake checks (ubuntu-latest) (pull_request) Failing after 13m2s
- The following Nix Flake inputs were updated:

```
• Updated input 'nixpkgs':
    'github:nixos/nixpkgs/1750f3c1c89488e2ffdd47cab9d05454dddfb734?narHash=sha256-oDJGK1UMArK52vcW9S5S2apeec4rbfNELgc50LqiPNs%3D' (2025-03-24)
  → 'github:nixos/nixpkgs/0ab4a506520ea9247e673e9849ecf84c72f88be2?narHash=sha256-O3GFPU0Uyv80LKVMMukVqrfxSzWKkBwQIHN2UnRSCZk%3D' (2025-03-27)
• Updated input 'uv2nix':
    'github:pyproject-nix/uv2nix/f465e0c74d247545e57dc17cef72c973ff18ff13?narHash=sha256-GdxIZi77FZCz9nnkgjT6d50TtwlAjS1zn9kVffknQ0U%3D' (2025-03-24)
  → 'github:pyproject-nix/uv2nix/f5b0ec1f303c16e5819694f8f5185814177dacb2?narHash=sha256-dgohGOVoIju52AC1czMjRfAEqf9LZZSRpxZ3ekclEak%3D' (2025-03-27)
```

```
comparing packages.x86_64-linux.default:
<<< /nix/store/xxdhk7a0f19fwji43ny5nl2qr503flik-flupdt-env.drv
>>> /nix/store/nphnvwvbigyfaq5xpg4qsnz2fni8fp9d-flupdt-env.drv
Version changes:
[U.]  #1  uv  0.6.9, 0.6.9-vendor, 0.6.9-vendor-staging -> 0.6.10, 0.6.10-vendor, 0.6.10-vendor-staging
Added packages:
[A.]  #1  ar2ljidn95kdsdl1kc8745dj3h17brq9-source  <none>
Removed packages:
[R.]  #1  n65xmc5vjbzdr37kaa9yivflylmgb4qg-source  <none>
Closure size: 1473 -> 1473 (24 paths added, 24 paths removed, delta +0, disk usage +16B).

comparing checks.x86_64-linux.pre-commit-check:
<<< /nix/store/c63gqq7pcimhzg82lyb2g39yk211pgm0-pre-commit-run.drv
>>> /nix/store/w5y1zgyws0hixsjrfydrv0fldpih0zhh-pre-commit-run.drv
Added packages:
[A+]  #1  ar2ljidn95kdsdl1kc8745dj3h17brq9-source  <none>
Removed packages:
[R-]  #1  n65xmc5vjbzdr37kaa9yivflylmgb4qg-source  <none>
Closure size: 2604 -> 2604 (2 paths added, 2 paths removed, delta +0, disk usage +0B).

comparing packages.x86_64-linux.flupdt:
<<< /nix/store/xxdhk7a0f19fwji43ny5nl2qr503flik-flupdt-env.drv
>>> /nix/store/nphnvwvbigyfaq5xpg4qsnz2fni8fp9d-flupdt-env.drv
Version changes:
[U.]  #1  uv  0.6.9, 0.6.9-vendor, 0.6.9-vendor-staging -> 0.6.10, 0.6.10-vendor, 0.6.10-vendor-staging
Added packages:
[A.]  #1  ar2ljidn95kdsdl1kc8745dj3h17brq9-source  <none>
Removed packages:
[R.]  #1  n65xmc5vjbzdr37kaa9yivflylmgb4qg-source  <none>
Closure size: 1473 -> 1473 (24 paths added, 24 paths removed, delta +0, disk usage +16B).

comparing devShells.x86_64-linux.default:
<<< /nix/store/iks6vp5mfcbd0wyb3p7ijlaqn13apfza-nix-shell.drv
>>> /nix/store/qknh1grdkhk1hxj14if5mkal5b1zk033-nix-shell.drv
Version changes:
[U*]  #1  uv  0.6.9, 0.6.9-vendor, 0.6.9-vendor-staging -> 0.6.10, 0.6.10-vendor, 0.6.10-vendor-staging
Added packages:
[A.]  #1  ar2ljidn95kdsdl1kc8745dj3h17brq9-source  <none>
Removed packages:
[R.]  #1  n65xmc5vjbzdr37kaa9yivflylmgb4qg-source  <none>
Closure size: 2691 -> 2691 (38 paths added, 38 paths removed, delta +0, disk usage +16B).

comparing formatter.x86_64-linux:
<<< /nix/store/kpjm9ymg7d9h4mw5jf1ddzc40lv3wrbi-nixfmt-unstable-2025-03-03.drv
>>> /nix/store/kpjm9ymg7d9h4mw5jf1ddzc40lv3wrbi-nixfmt-unstable-2025-03-03.drv
No version or selection state changes.
Closure size: 1307 -> 1307 (0 paths added, 0 paths removed, delta +0, disk usage +0B).

```

Auto-generated by [update.yml][1] with the help of
[create-pull-request][2].

[1]: https://nayeonie.com/ahuston-0/nix-dotfiles/src/branch/main/.github/workflows/flake-update.yml
[2]: https://forgejo.stefka.eu/jiriks74/create-pull-request
2025-03-27 12:05:45 +00:00
7f648f7199 Merge pull request 'exit on failure, add per-class logging' (#12) from feature/logging-and-exit into main
Some checks failed
Check flake.lock / Check health of `flake.lock` (push) Successful in 13s
Check Nix formatting / Perform Nix format checks (push) Successful in 1m41s
Check Nix flake / Build nix outputs (ubuntu-latest) (push) Failing after 2m28s
Check Nix flake / Perform Nix flake checks (ubuntu-latest) (push) Successful in 5m26s
Update flakes / update_lockfile (push) Successful in 6m39s
Reviewed-on: #12
2025-03-26 12:04:28 -04:00
9103ece19c
default logging is now INFO
All checks were successful
Check Nix flake / Perform Nix flake checks (ubuntu-latest) (pull_request) Successful in 3m22s
Check Nix flake / Build nix outputs (ubuntu-latest) (pull_request) Successful in 2m21s
Check flake.lock / Check health of `flake.lock` (pull_request) Successful in 12s
Check Nix formatting / Perform Nix format checks (pull_request) Successful in 1m27s
2025-03-26 11:33:57 -04:00
607f3f9f76 exit on failure, add per-class logging
All checks were successful
Check Nix flake / Perform Nix flake checks (ubuntu-latest) (pull_request) Successful in 3m26s
Check flake.lock / Check health of `flake.lock` (pull_request) Successful in 20s
Check Nix flake / Build nix outputs (ubuntu-latest) (pull_request) Successful in 2m21s
Check Nix formatting / Perform Nix format checks (pull_request) Successful in 1m41s
Signed-off-by: ahuston-0 <aliceghuston@gmail.com>
2025-03-26 11:24:16 -04:00
6 changed files with 49 additions and 38 deletions

12
flake.lock generated
View File

@ -57,11 +57,11 @@
}, },
"nixpkgs": { "nixpkgs": {
"locked": { "locked": {
"lastModified": 1742800061, "lastModified": 1743039536,
"narHash": "sha256-oDJGK1UMArK52vcW9S5S2apeec4rbfNELgc50LqiPNs=", "narHash": "sha256-O3GFPU0Uyv80LKVMMukVqrfxSzWKkBwQIHN2UnRSCZk=",
"owner": "nixos", "owner": "nixos",
"repo": "nixpkgs", "repo": "nixpkgs",
"rev": "1750f3c1c89488e2ffdd47cab9d05454dddfb734", "rev": "0ab4a506520ea9247e673e9849ecf84c72f88be2",
"type": "github" "type": "github"
}, },
"original": { "original": {
@ -174,11 +174,11 @@
] ]
}, },
"locked": { "locked": {
"lastModified": 1742788087, "lastModified": 1743057427,
"narHash": "sha256-GdxIZi77FZCz9nnkgjT6d50TtwlAjS1zn9kVffknQ0U=", "narHash": "sha256-dgohGOVoIju52AC1czMjRfAEqf9LZZSRpxZ3ekclEak=",
"owner": "pyproject-nix", "owner": "pyproject-nix",
"repo": "uv2nix", "repo": "uv2nix",
"rev": "f465e0c74d247545e57dc17cef72c973ff18ff13", "rev": "f5b0ec1f303c16e5819694f8f5185814177dacb2",
"type": "github" "type": "github"
}, },
"original": { "original": {

View File

@ -11,6 +11,7 @@ from flupdt.common import bash_wrapper
drv_re = re.compile(r".*(/nix/store/.*\.drv).*") drv_re = re.compile(r".*(/nix/store/.*\.drv).*")
OUTPUT_DIR = mkdtemp(prefix="flupdt-outputs-") OUTPUT_DIR = mkdtemp(prefix="flupdt-outputs-")
logger = logging.getLogger(__name__)
def build_output(path: str, output: str) -> str | None: def build_output(path: str, output: str) -> str | None:
@ -20,16 +21,16 @@ def build_output(path: str, output: str) -> str | None:
:param output: flake output to be built :param output: flake output to be built
:returns the .drv path on success or None on failure :returns the .drv path on success or None on failure
""" """
logging.info(f"build {output}") logger.info(f"build {output}")
logging.debug(f"outputting to {OUTPUT_DIR}/{output}.nixoutput") logger.debug(f"outputting to {OUTPUT_DIR}/{output}.nixoutput")
out = bash_wrapper(f"nix build {path}#{output} -o {OUTPUT_DIR}/{output}.nixoutput") out = bash_wrapper(f"nix build {path}#{output} -o {OUTPUT_DIR}/{output}.nixoutput")
logging.debug("output") logger.debug("output")
logging.debug(out[0]) logger.debug(out[0])
logging.debug("error") logger.debug("error")
logging.debug(out[1]) logger.debug(out[1])
logging.debug("statuscode") logger.debug("statuscode")
logging.debug(out[2]) logger.debug(out[2])
if out[2] != 0: if out[2] != 0:
logging.warning(f"output {output} did not build correctly") logger.warning(f"output {output} did not build correctly")
return None return None
return "" return ""

View File

@ -5,6 +5,8 @@ import shutil
from flupdt.common import bash_wrapper from flupdt.common import bash_wrapper
logger = logging.getLogger(__name__)
def compare_derivations( def compare_derivations(
path_to_flake: str, path_to_pre_drv: str, path_to_post_drv: str, *, soft_failure: bool = True path_to_flake: str, path_to_pre_drv: str, path_to_post_drv: str, *, soft_failure: bool = True
@ -24,10 +26,10 @@ def compare_derivations(
) )
if returncode != 0: if returncode != 0:
log_func = logging.error log_func = logger.error
if soft_failure: if soft_failure:
log_func = logging.warning log_func = logger.warning
logging.warning(f"diff failed for {path_to_pre_drv} and {path_to_post_drv}") logger.warning(f"diff failed for {path_to_pre_drv} and {path_to_post_drv}")
log_func(stderr) log_func(stderr)
return stdout return stdout

View File

@ -9,6 +9,8 @@ from flupdt.common import bash_wrapper
drv_re = re.compile(r".*(/nix/store/.*\.drv).*") drv_re = re.compile(r".*(/nix/store/.*\.drv).*")
logger = logging.getLogger(__name__)
def evaluate_output(path: str, output: str) -> str | None: def evaluate_output(path: str, output: str) -> str | None:
"""Evaluates a given output in a flake. """Evaluates a given output in a flake.
@ -18,19 +20,19 @@ def evaluate_output(path: str, output: str) -> str | None:
:returns the .drv path on success or None on failure :returns the .drv path on success or None on failure
:raises RuntimeError: evaluation succeeds but no derivation is found :raises RuntimeError: evaluation succeeds but no derivation is found
""" """
logging.info(f"evaluating {output}") logger.info(f"evaluating {output}")
out = bash_wrapper(f"nix eval {path}#{output}") out = bash_wrapper(f"nix eval {path}#{output}")
logging.debug(out[0]) logger.debug(out[0])
logging.debug(out[1]) logger.debug(out[1])
logging.debug(out[2]) logger.debug(out[2])
if out[2] != 0: if out[2] != 0:
logging.warning(f"output {output} did not evaluate correctly") logger.warning(f"output {output} did not evaluate correctly")
return None return None
drv_match = drv_re.match(out[0]) drv_match = drv_re.match(out[0])
if drv_match is None: if drv_match is None:
out_msg = "derivation succeeded but output derivation does not contain a derivation" out_msg = "derivation succeeded but output derivation does not contain a derivation"
raise RuntimeError(out_msg) raise RuntimeError(out_msg)
drv = drv_match.group(1) drv = drv_match.group(1)
logging.debug(f"derivation evaluated to {drv}") logger.debug(f"derivation evaluated to {drv}")
return drv return drv

View File

@ -12,6 +12,7 @@ output_regexes = [
re.compile(r"checking derivation (.*)..."), re.compile(r"checking derivation (.*)..."),
re.compile(r"checking NixOS configuration \'(nixosConfigurations.*)\'\.\.\."), re.compile(r"checking NixOS configuration \'(nixosConfigurations.*)\'\.\.\."),
] ]
logger = logging.getLogger(__name__)
def traverse_json_base(json_dict: dict[str, typing.Any], path: list[str]) -> list[str]: def traverse_json_base(json_dict: dict[str, typing.Any], path: list[str]) -> list[str]:
@ -54,7 +55,7 @@ def get_derivations_from_check(nix_path: str, path_to_flake: str) -> list[str]:
""" """
flake_check = bash_wrapper(f"{nix_path} flake check --verbose --keep-going", path=path_to_flake) flake_check = bash_wrapper(f"{nix_path} flake check --verbose --keep-going", path=path_to_flake)
if flake_check[2] != 0: if flake_check[2] != 0:
logging.warning( logger.warning(
"nix flake check returned non-zero exit code, collecting all available outputs" "nix flake check returned non-zero exit code, collecting all available outputs"
) )
error_out = flake_check[1].split("\n") error_out = flake_check[1].split("\n")
@ -62,10 +63,10 @@ def get_derivations_from_check(nix_path: str, path_to_flake: str) -> list[str]:
derivations = [] derivations = []
for output in possible_outputs: for output in possible_outputs:
for r in output_regexes: for r in output_regexes:
logging.debug(f"{output} {r.pattern}") logger.debug(f"{output} {r.pattern}")
match = r.match(output) match = r.match(output)
if match is not None: if match is not None:
logging.debug(match.groups()) logger.debug(match.groups())
derivations += [match.groups()[0]] derivations += [match.groups()[0]]
return derivations return derivations
@ -84,8 +85,8 @@ def get_derivations(path_to_flake: str) -> list[str]:
raise RuntimeError(status_msg) raise RuntimeError(status_msg)
flake_show = bash_wrapper(f"{nix_path} flake show --json", path=path_to_flake) flake_show = bash_wrapper(f"{nix_path} flake show --json", path=path_to_flake)
if flake_show[2] != 0: if flake_show[2] != 0:
logging.error("flake show returned non-zero exit code") logger.error("flake show returned non-zero exit code")
logging.warning("falling back to full evaluation via nix flake check") logger.warning("falling back to full evaluation via nix flake check")
derivations = get_derivations_from_check(nix_path, path_to_flake) derivations = get_derivations_from_check(nix_path, path_to_flake)
else: else:
flake_show_json = json.loads(flake_show[0]) flake_show_json = json.loads(flake_show[0])

View File

@ -4,6 +4,7 @@
import logging import logging
import os import os
import sys
from argparse import Namespace from argparse import Namespace
from pathlib import Path from pathlib import Path
@ -14,6 +15,8 @@ from flupdt.flake_diff import compare_derivations
from flupdt.flake_eval import evaluate_output from flupdt.flake_eval import evaluate_output
from flupdt.flake_show import get_derivations from flupdt.flake_show import get_derivations
logger = logging.getLogger(__name__)
def batch_eval(args: Namespace, flake_path: str, derivations: list[str]) -> None: def batch_eval(args: Namespace, flake_path: str, derivations: list[str]) -> None:
"""Bulk run evaluations or builds on a derivation set. """Bulk run evaluations or builds on a derivation set.
@ -28,12 +31,14 @@ def batch_eval(args: Namespace, flake_path: str, derivations: list[str]) -> None
if args.evaluate: if args.evaluate:
drv_map[d] = evaluate_output(flake_path, d) drv_map[d] = evaluate_output(flake_path, d)
if args.build: if args.build:
build_output(flake_path, d) drv_map[d] = build_output(flake_path, d)
if args.json: if args.json:
with Path.open(args.json, "w+") as f: with Path.open(args.json, "w+") as f:
from json import dump from json import dump
dump(drv_map, f) dump(drv_map, f)
if any(x is None for x in drv_map.values()):
sys.exit(1)
def compare_drvs(args: Namespace) -> None: def compare_drvs(args: Namespace) -> None:
@ -52,8 +57,8 @@ def compare_drvs(args: Namespace) -> None:
pre_json_dict = load(pre) pre_json_dict = load(pre)
post_json_dict = load(post) post_json_dict = load(post)
logging.debug(f"pre-snapshot derivations: {pre_json_dict}") logger.debug(f"pre-snapshot derivations: {pre_json_dict}")
logging.debug(f"post-snapshot derivations: {post_json_dict}") logger.debug(f"post-snapshot derivations: {post_json_dict}")
pre_json_keys = set(pre_json_dict.keys()) pre_json_keys = set(pre_json_dict.keys())
post_json_keys = set(post_json_dict.keys()) post_json_keys = set(post_json_dict.keys())
@ -64,11 +69,11 @@ def compare_drvs(args: Namespace) -> None:
missing_pre_keys = post_json_keys.difference(common_keys_to_eval) missing_pre_keys = post_json_keys.difference(common_keys_to_eval)
if missing_pre_keys: if missing_pre_keys:
logging.warning(f"Following outputs are missing from pre-snapshot: {missing_pre_keys}") logger.warning(f"Following outputs are missing from pre-snapshot: {missing_pre_keys}")
if missing_post_keys: if missing_post_keys:
logging.warning(f"Following outputs are missing from post-snapshot: {missing_post_keys}") logger.warning(f"Following outputs are missing from post-snapshot: {missing_post_keys}")
logging.info(f"Evaluating the following outputs for differences: {common_keys_to_eval}") logger.info(f"Evaluating the following outputs for differences: {common_keys_to_eval}")
out_file: str = os.devnull out_file: str = os.devnull
if args.compare_output_to_file: if args.compare_output_to_file:
@ -97,11 +102,11 @@ def build_or_eval(args: Namespace) -> None:
lambda s: s.startswith("hydraJobs"), get_derivations(flake_path) lambda s: s.startswith("hydraJobs"), get_derivations(flake_path)
) )
derivations, hydra_jobs = list(derivations), list(hydra_jobs) derivations, hydra_jobs = list(derivations), list(hydra_jobs)
logging.info(f"derivations: {list(derivations)}") logger.info(f"derivations: {list(derivations)}")
batch_eval(args, flake_path, derivations) batch_eval(args, flake_path, derivations)
if not args.keep_hydra: if not args.keep_hydra:
logging.info("--keep-hydra flag is not specified, removing Hydra jobs") logger.info("--keep-hydra flag is not specified, removing Hydra jobs")
else: else:
batch_eval(args, flake_path, hydra_jobs) batch_eval(args, flake_path, hydra_jobs)
@ -112,7 +117,7 @@ def main() -> None:
:returns: None :returns: None
""" """
configure_logger("DEBUG") configure_logger("INFO")
args = parse_inputs() args = parse_inputs()
if args.compare_drvs: if args.compare_drvs:
compare_drvs(args) compare_drvs(args)