Agent buffering & backpressure

A naive implementation would POST to core for every detection, every rate-limit hit, every ban. At any meaningful traffic, that floods the network, multiplies request latency, and falls apart the moment core has a hiccup. Buffering decouples emission from delivery.

Why buffer

The buffer is the shock absorber. Its size, flush cadence, and retry behavior are how you tune it.

Two buffers - events and metrics, independent

Bounded in-memory queues. Events and metrics each get their own. They fill independently, overflow independently, and flush independently to different endpoints.

Events buffer - detections, bans, rate-limit hits, behavioral rule trips. High-cardinality signal, one entry per security decision. Controlled by agent_enable_events.

Metrics buffer - aggregated counts and timings, pre-aggregated before emission. Lower-cardinality. Controlled by agent_enable_metrics.

Disabling one doesn't affect the other. agent_enable_metrics=False is a valid debugging move; events keep flowing.

Flush triggers - size OR time

Two conditions, evaluated continuously. When either crosses its threshold, the buffer flushes.

Size trigger - buffer fills to 80%. len(buffer) >= buffer_size * 0.8. Fires during traffic spikes when events arrive faster than they ship. Headroom before overflow - fires when 20% capacity remains, so a slow flush doesn't immediately push the buffer past its limit.

Time trigger - every flush_interval seconds. now - last_flush >= flush_interval. Bounds how stale a buffered event can get. Even at zero traffic, a sentinel flush runs on the interval.

The OR is what gives you both low latency under load AND predictable delivery during quiet periods.

Retry & backoff - exponential with a cap, 429 takes a different path

Each flush gets retry_attempts tries. Generic failures wait longer between attempts.

Attempt 1 → fail → wait 2s
Attempt 2 → fail → wait 4s
Attempt 3 → fail → give up

Formula: delay = backoff_factor × 2^attempt, capped at 60s. Default backoff_factor=1.0.

429 from core is special - the agent doesn't immediately retry. It hands control to its internal rate limiter, which paces subsequent flushes until core stops returning 429. A naive retry on 429 would amplify the load that triggered it. Self-throttling makes the agent a cooperative client instead of an adversarial one.

Overflow - what happens when the buffer fills

If events arrive faster than they can be flushed (sustained spike, prolonged core outage), the buffer fills. Three policies, picked via AgentConfig.buffer_overflow_policy (guard-agent 2.4.0+):

Opt-in - spill to Redis persistence. Independent of policy. When enabled, the agent persists overflowing entries to Redis at agent_events:* hashes (TTL 3600s). On recovery, the buffer drains from there as well as from memory. Trades a small amount of latency and Redis storage for delivery durability.

Sizing - survive one bad flush

Buffer sizing is a function of your peak rate and your flush cadence. The goal: a single flush failure shouldn't push you into overflow. Two flushes worth of headroom is the rule of thumb.

buffer_size ≥ peak_events_per_second × flush_interval × 2

Example - high-traffic API at 1,000 events/sec, flush_interval=2: buffer ≥ 4,000. The example app's agent_buffer_size=5000 covers this comfortably.

Wiring

SecurityConfig(
    agent_buffer_size=5000,
    agent_flush_interval=2,
    agent_retry_attempts=3,
    agent_timeout=30,
    agent_enable_events=True,
    agent_enable_metrics=True,
)

Troubleshooting