opencommit/roadmap

Request Mirror #3

New issue

Open

opened 2026-04-11 22:00:15 +02:00 by mvdkleijn · 0 comments

mvdkleijn commented

2026-04-11 22:00:15 +02:00

Owner

1. What

Development of a lightweight, non-blocking Traefik middleware plugin designed to asynchronously mirror HTTP request metadata and payloads to a user-defined webhook. This tool acts as a "data mirror", enabling platform internal services (such as security engines or audit loggers) to observe traffic without interfering with the primary request lifecycle.

Note: this is an enabler epic.

2. Why

Extensibility: Provides a standardized way to plug new functionality (security, analytics, auditing) into the OpenCommit ecosystem without modifying core services.
Decoupling: Separates the "detection" logic from the "application" logic, allowing for independent scaling and updates of security tools like the Anti-Spam Policy Engine.
Performance Preservation: By utilizing an asynchronous, non-blocking architecture, we enable advanced monitoring without introducing latency to the end-user's request path.

3. Boundaries

In-Scope:
- Development of the Go-based Traefik plugin module.
- Implementation of the Kubernetes Custom Resource Definition (CRD) for configuration.
- Payload extraction logic (Path, Method, Headers, Cookies, and limited Body).
- Asynchronous webhook dispatch mechanism with configurable timeouts.
Out-of-Scope:
- Full-body mirroring of multi-gigabyte uploads (strictly limited to a configurable 1MB safety threshold).
- Modification of the upstream Traefik binary or core routing logic.
- Persistent storage of mirrored data (the plugin is "pass-through" only).

4. Definition of Done

Non-Blocking Performance: Verified through load testing that the middleware does not delay the original HTTP response to the client, even during high-volume mirroring.
Configuration Accuracy: The plugin correctly honors the watchURL, methods, and cookieNames defined in the Traefik Middleware CRD.
Payload Integrity: The outgoing JSON webhook payload contains all required fields (timestamp, request metadata, and configured body/cookies) in a valid, preferably standardized, format.
Resource Safety: The plugin must strictly enforce the (configurable) 1MB body limit to prevent memory exhaustion within the Traefik process.
Deployment Readiness: A documented installation guide exists for enabling the plugin via Traefik static configuration and applying it via Kubernetes Ingress annotations.

## **1. What** Development of a lightweight, non-blocking Traefik middleware plugin designed to asynchronously mirror HTTP request metadata and payloads to a user-defined webhook. This tool acts as a "data mirror", enabling platform internal services (such as security engines or audit loggers) to observe traffic without interfering with the primary request lifecycle. Note: this is an **enabler epic**. ## **2. Why** * **Extensibility:** Provides a standardized way to plug new functionality (security, analytics, auditing) into the OpenCommit ecosystem without modifying core services. * **Decoupling:** Separates the "detection" logic from the "application" logic, allowing for independent scaling and updates of security tools like the Anti-Spam Policy Engine. * **Performance Preservation:** By utilizing an asynchronous, non-blocking architecture, we enable advanced monitoring without introducing latency to the end-user's request path. ## **3. Boundaries** * **In-Scope:** * Development of the Go-based Traefik plugin module. * Implementation of the Kubernetes Custom Resource Definition (CRD) for configuration. * Payload extraction logic (Path, Method, Headers, Cookies, and limited Body). * Asynchronous webhook dispatch mechanism with configurable timeouts. * **Out-of-Scope:** * Full-body mirroring of multi-gigabyte uploads (strictly limited to a configurable 1MB safety threshold). * Modification of the upstream Traefik binary or core routing logic. * Persistent storage of mirrored data (the plugin is "pass-through" only). ## **4. Definition of Done** * [ ] **Non-Blocking Performance:** Verified through load testing that the middleware does not delay the original HTTP response to the client, even during high-volume mirroring. * [ ] **Configuration Accuracy:** The plugin correctly honors the `watchURL`, `methods`, and `cookieNames` defined in the Traefik Middleware CRD. * [ ] **Payload Integrity:** The outgoing JSON webhook payload contains all required fields (timestamp, request metadata, and configured body/cookies) in a valid, preferably standardized, format. * [ ] **Resource Safety:** The plugin must strictly enforce the (configurable) 1MB body limit to prevent memory exhaustion within the Traefik process. * [ ] **Deployment Readiness:** A documented installation guide exists for enabling the plugin via Traefik static configuration and applying it via Kubernetes Ingress annotations.