webhooks: implement authentication for GitHub and Gitea

- Add HMAC-SHA256 signature verification for webhooks - Support multiple secrets for rotation - Add security logging for authentication events - Maintain backward compatibility (auth optional during migration) - Add comprehensive test coverage Without authentication, anyone could trigger job evaluations by sending POST requests to webhook endpoints. This could lead to resource exhaustion through repeated requests or manipulation of build scheduling. While not a data breach risk, it allows unauthorized control over CI/CD operations.
2025-08-03 07:31:13 +02:00
parent 4d2d0f9722
commit f2cbf14f7e
7 changed files with 596 additions and 19 deletions
--- a/doc/manual/src/SUMMARY.md
+++ b/doc/manual/src/SUMMARY.md
@@ -10,6 +10,7 @@
  - [RunCommand](./plugins/RunCommand.md)
 - [Using the external API](api.md)
 - [Webhooks](webhooks.md)
+  - [Webhook Authentication Migration Guide](webhook-migration-guide.md)
 - [Monitoring Hydra](./monitoring/README.md)

 ## Developer's Guide
--- a/doc/manual/src/configuration.md
+++ b/doc/manual/src/configuration.md
@@ -266,6 +266,40 @@ default role mapping:
 Note that configuring both the LDAP parameters in the hydra.conf and via
 the environment variable is a fatal error.

+Webhook Authentication
+---------------------
+
+Hydra supports authenticating webhook requests from GitHub and Gitea to prevent unauthorized job evaluations.
+Webhook secrets should be stored in separate files outside the Nix store for security using Config::General's include mechanism.
+
+In your main `hydra.conf`:
+```apache
+<webhooks>
+  Include /var/lib/hydra/secrets/webhook-secrets.conf
+</webhooks>
+```
+
+Then create `/var/lib/hydra/secrets/webhook-secrets.conf` with your actual secrets:
+```apache
+<github>
+  secret = your-github-webhook-secret
+</github>
+<gitea>
+  secret = your-gitea-webhook-secret
+</gitea>
+```
+
+For multiple secrets (useful for rotation or multiple environments), use an array:
+```apache
+<github>
+  secret = your-github-webhook-secret-prod
+  secret = your-github-webhook-secret-staging
+</github>
+```
+
+**Important**: The secrets file should have restricted permissions (e.g., 0600) to prevent unauthorized access.
+See the [Webhooks documentation](webhooks.md) for detailed setup instructions.
+
 Embedding Extra HTML
 --------------------

--- a/doc/manual/src/webhook-migration-guide.md
+++ b/doc/manual/src/webhook-migration-guide.md
@@ -0,0 +1,168 @@
+# Webhook Authentication Migration Guide
+
+This guide helps Hydra administrators migrate from unauthenticated webhooks to authenticated webhooks to secure their Hydra instances against unauthorized job evaluations.
+
+## Why Migrate?
+
+Currently, Hydra's webhook endpoints (`/api/push-github` and `/api/push-gitea`) accept any POST request without authentication. This vulnerability allows:
+- Anyone to trigger expensive job evaluations
+- Potential denial of service through repeated requests
+- Manipulation of build timing and scheduling
+
+## Step-by-Step Migration for NixOS
+
+### 1. Create Webhook Configuration
+
+Create a webhook secrets configuration file with the generated secrets:
+
+```bash
+# Create the secrets configuration file with inline secret generation
+cat > /var/lib/hydra/secrets/webhook-secrets.conf <<EOF
+<github>
+  secret = $(openssl rand -hex 32)
+</github>
+<gitea>
+  secret = $(openssl rand -hex 32)
+</gitea>
+EOF
+
+# Set secure permissions
+chmod 0600 /var/lib/hydra/secrets/webhook-secrets.conf
+chown hydra:hydra /var/lib/hydra/secrets/webhook-secrets.conf
+```
+
+**Important**: Save the generated secrets to configure them in GitHub/Gitea later. You can view them with:
+```bash
+cat /var/lib/hydra/secrets/webhook-secrets.conf
+```
+
+Then update your NixOS configuration to include the webhook configuration:
+
+```nix
+{
+  services.hydra-dev = {
+    enable = true;
+    hydraURL = "https://hydra.example.com";
+    notificationSender = "hydra@example.com";
+
+    extraConfig = ''
+      <webhooks>
+        Include /var/lib/hydra/secrets/webhook-secrets.conf
+      </webhooks>
+    '';
+  };
+}
+```
+
+For multiple secrets (useful for rotation or multiple environments), update your webhook-secrets.conf:
+
+```apache
+<github>
+  secret = your-github-webhook-secret-prod
+  secret = your-github-webhook-secret-staging
+</github>
+<gitea>
+  secret = your-gitea-webhook-secret
+</gitea>
+```
+
+### 2. Deploy Configuration
+
+Apply the NixOS configuration:
+
+```bash
+nixos-rebuild switch
+```
+
+This will automatically restart Hydra services with the new configuration.
+
+### 3. Verify Configuration
+
+Check Hydra's logs to ensure secrets were loaded successfully:
+
+```bash
+journalctl -u hydra-server | grep -i webhook
+```
+
+You should not see warnings about webhook authentication not being configured.
+
+### 4. Update Your Webhooks
+
+#### GitHub
+1. Navigate to your repository settings: `https://github.com/<owner>/<repo>/settings/hooks`
+2. Edit your existing Hydra webhook
+3. In the "Secret" field, paste the content of `/var/lib/hydra/secrets/github-webhook-secret`
+4. Click "Update webhook"
+5. GitHub will send a ping event to verify the configuration
+
+#### Gitea
+1. Navigate to your repository webhook settings
+2. Edit your existing Hydra webhook
+3. In the "Secret" field, paste the content of `/var/lib/hydra/secrets/gitea-webhook-secret`
+4. Click "Update Webhook"
+5. Use the "Test Delivery" button to verify the configuration
+
+### 5. Test the Configuration
+
+After updating each webhook:
+1. Make a test commit to trigger the webhook
+2. Check Hydra's logs for successful authentication
+3. Verify the evaluation was triggered in Hydra's web interface
+
+## Troubleshooting
+
+### 401 Unauthorized Errors
+
+If webhooks start failing with 401 errors:
+- Verify the secret in the Git forge matches the file content exactly
+- Check file permissions: `ls -la /var/lib/hydra/secrets/`
+- Ensure no extra whitespace in secret files
+- Check Hydra logs for specific error messages
+
+### Webhook Still Unauthenticated
+
+If you see warnings about unauthenticated webhooks after configuration:
+- Verify the configuration syntax in your NixOS module
+- Ensure the NixOS configuration was successfully applied
+- Check that the webhook-secrets.conf file exists and is readable by the Hydra user
+- Verify the Include path is correct in your hydra.conf
+- Check the syntax of your webhook-secrets.conf file
+
+### Testing Without Git Forge
+
+You can test webhook authentication using curl:
+
+```bash
+# Read the secret
+SECRET=$(cat /var/lib/hydra/secrets/github-webhook-secret)
+
+# Create test payload
+PAYLOAD='{"ref":"refs/heads/main","repository":{"clone_url":"https://github.com/test/repo.git"}}'
+
+# Calculate signature
+SIGNATURE="sha256=$(echo -n "$PAYLOAD" | openssl dgst -sha256 -hmac "$SECRET" | cut -d' ' -f2)"
+
+# Send authenticated request
+curl -X POST https://your-hydra/api/push-github \
+  -H "Content-Type: application/json" \
+  -H "X-Hub-Signature-256: $SIGNATURE" \
+  -d "$PAYLOAD"
+```
+
+For Gitea (no prefix in signature):
+```bash
+# Read the secret
+SECRET=$(cat /var/lib/hydra/secrets/gitea-webhook-secret)
+
+# Create test payload
+PAYLOAD='{"ref":"refs/heads/main","repository":{"clone_url":"https://gitea.example.com/test/repo.git"}}'
+
+# Calculate signature
+SIGNATURE=$(echo -n "$PAYLOAD" | openssl dgst -sha256 -hmac "$SECRET" | cut -d' ' -f2)
+
+# Send authenticated request
+curl -X POST https://your-hydra/api/push-gitea \
+  -H "Content-Type: application/json" \
+  -H "X-Gitea-Signature: $SIGNATURE" \
+  -d "$PAYLOAD"
+```
--- a/doc/manual/src/webhooks.md
+++ b/doc/manual/src/webhooks.md
@@ -3,6 +3,58 @@
 Hydra can be notified by github or gitea with webhooks to trigger a new evaluation when a
 jobset has a github repo in its input.

+## Webhook Authentication
+
+Hydra supports webhook signature verification for both GitHub and Gitea using HMAC-SHA256. This ensures that webhook
+requests are coming from your configured Git forge and haven't been tampered with.
+
+### Configuring Webhook Authentication
+
+1. **Create webhook configuration**: Generate and store webhook secrets securely:
+   ```bash
+   # Create directory and generate secrets in one step
+   mkdir -p /var/lib/hydra/secrets
+   cat > /var/lib/hydra/secrets/webhook-secrets.conf <<EOF
+   <github>
+     secret = $(openssl rand -hex 32)
+   </github>
+   <gitea>
+     secret = $(openssl rand -hex 32)
+   </gitea>
+   EOF
+
+   # Set secure permissions
+   chmod 0600 /var/lib/hydra/secrets/webhook-secrets.conf
+   chown hydra:hydra /var/lib/hydra/secrets/webhook-secrets.conf
+   ```
+
+2. **Configure Hydra**: Add the following to your `hydra.conf`:
+   ```apache
+   <webhooks>
+     Include /var/lib/hydra/secrets/webhook-secrets.conf
+   </webhooks>
+   ```
+
+3. **Configure your Git forge**: View the generated secrets and configure them in GitHub/Gitea:
+   ```bash
+   grep "secret =" /var/lib/hydra/secrets/webhook-secrets.conf
+   ```
+
+### Multiple Secrets Support
+
+Hydra supports configuring multiple secrets for each platform, which is useful for:
+- Zero-downtime secret rotation
+- Supporting multiple environments (production/staging)
+- Gradual migration of webhooks
+
+To configure multiple secrets, use array syntax:
+```apache
+<github>
+  secret = current-webhook-secret
+  secret = previous-webhook-secret
+</github>
+```
+
 ## GitHub

 To set up a webhook for a GitHub repository go to `https://github.com/<yourhandle>/<yourrepo>/settings`
@@ -10,11 +62,16 @@ and in the `Webhooks` tab click on `Add webhook`.

 - In `Payload URL` fill in `https://<your-hydra-domain>/api/push-github`.
 - In `Content type` switch to `application/json`.
- The `Secret` field can stay empty.
+- In the `Secret` field, enter the content of your GitHub webhook secret file (if authentication is configured).
 - For `Which events would you like to trigger this webhook?` keep the default option for events on `Just the push event.`.

 Then add the hook with `Add webhook`.

+### Verifying GitHub Webhook Security
+
+After configuration, GitHub will send webhook requests with an `X-Hub-Signature-256` header containing the HMAC-SHA256
+signature of the request body. Hydra will verify this signature matches the configured secret.
+
 ## Gitea

 To set up a webhook for a Gitea repository go to the settings of the repository in your Gitea instance
@@ -22,6 +79,23 @@ and in the `Webhooks` tab click on `Add Webhook` and choose `Gitea` in the drop

 - In `Target URL` fill in `https://<your-hydra-domain>/api/push-gitea`.
 - Keep HTTP method `POST`, POST Content Type `application/json` and Trigger On `Push Events`.
+- In the `Secret` field, enter the content of your Gitea webhook secret file (if authentication is configured).
 - Change the branch filter to match the git branch hydra builds.

 Then add the hook with `Add webhook`.
+
+### Verifying Gitea Webhook Security
+
+After configuration, Gitea will send webhook requests with an `X-Gitea-Signature` header containing the HMAC-SHA256
+signature of the request body. Hydra will verify this signature matches the configured secret.
+
+## Troubleshooting
+
+If you receive 401 Unauthorized errors:
+- Verify the webhook secret in your Git forge matches the content of the secret file exactly
+- Check that the secret file has proper permissions (should be 0600)
+- Look at Hydra's logs for specific error messages
+- Ensure the correct signature header is being sent by your Git forge
+
+If you see warnings about webhook authentication not being configured:
+- Configure webhook authentication as described above to secure your endpoints