How a Missing useCallback Triggered 10,000 API Requests Per Minute in Production

11:40 AM, Tuesday. Our API dashboard goes red. Request volume on the product search endpoint jumps from a baseline of 800 req/min to 10,400 req/min in under 90 seconds. First instinct around the room: DDoS. We spin up incident response, check Cloudflare, start slicing by IP. Every single request was coming from authenticated, logged-in users. Normal users doing normal things. We were the attacker.

Production failure: the API that couldn't stop being called

Our FastAPI backend normally handled 800 to 1,200 req/min on the product search endpoint. It was receiving 10,400 req/min, a 13× spike in 90 seconds. P99 latency climbed from 210ms to 4.8s. The Postgres connection pool hit its ceiling of 100 connections. DigitalOcean's managed database started queuing.

Within 4 minutes, our rate limiter started blacklisting IP ranges. Users began receiving 429s, which included active paying customers mid-session. We rolled back the last deployment, a UI refresh to the product listing page, and the spike vanished inside 60 seconds. The backend exhaled. Then the investigation started.

False assumptions: hunting a timer that wasn't there

The rollback confirmed the culprit was in the last deploy. Our working theory was wrong. We assumed a background polling loop had been accidentally set to a 1-second interval instead of 30 seconds. That's the reflex: look for a timer gone haywire.

We combed through every setInterval and setTimeout in the diff. Nothing. Polling logic was unchanged. Then we checked for a misconfigured React Query refetchInterval. Also clean. The mistake was assuming the repetition was intentional rather than unintended re-rendering. Twenty minutes chasing a ghost.

Profiling the component tree: following the re-render trail

With the rollback live, I reproduced the issue locally by cherry-picking the offending commit onto a branch. React DevTools Profiler was the first tool: I opened the product listing page, typed a single character into the search input, and watched the flame graph.

The ProductSearch component was re-rendering continuously. Not once per keystroke. A tight loop. The profiler recorded 47 renders in 2,000ms after the initial keystroke, with no further user input. Each render triggered a fetch to /api/products/search. The component was burning CPU and network at the same time.

React Profiler — ProductSearch re-render storm
──────────────────────────────────────────────
Time →   0ms   40ms   80ms  120ms  160ms  200ms
         │      │      │      │      │      │
Render   ██     ██     ██     ██     ██     ██   (continuous loop)
         │      │      │      │      │      │
API      ↑      ↑      ↑      ↑      ↑      ↑
call  /search /search /search /search ...

47 renders in 2,000ms  = 23.5 renders/sec
1 API call per render  = 23.5 req/sec per user session
420 concurrent users   = ~9,870 req/sec ≈ 10,400 req/min observed
──────────────────────────────────────────────

Infinite re-render loop. The question was why. I isolated each useEffect and found one with a dependency array that looked correct at a glance. It wasn't.

Root cause: an unstable closure reference poisoning useEffect

The PR introduced a refactored ProductSearch component. A senior engineer had moved the search fetch logic into an inline async function, which read as clean separation from JSX. The code reviewed fine. I signed off on it myself. The bug was invisible unless you knew exactly how React's dependency comparison works, and in a code review at 4 PM on a Friday I did not.

// ❌ BEFORE: fetchProducts is recreated on every render
function ProductSearch({ filters }: Props) {
  const [query, setQuery] = useState('');
  const [results, setResults] = useState<Product[]>([]);

  // New function object allocated on EVERY render.
  // JavaScript closures don't cache — each render = new reference.
  const fetchProducts = async (q: string) => {
    const params = new URLSearchParams({ q, ...filters });
    const res = await fetch(`/api/products/search?${params}`);
    const data = await res.json();
    setResults(data.products); // ← triggers re-render
  };

  // React compares deps with Object.is().
  // fetchProducts is a NEW object each render → "changed" every time.
  // Effect fires → setResults → re-render → new fetchProducts → effect fires ...
  useEffect(() => {
    fetchProducts(query);
  }, [query, fetchProducts]); // ← fetchProducts is the infinite loop trigger

  return (
    <div>
      <input value={query} onChange={e => setQuery(e.target.value)} />
      <ProductList results={results} />
    </div>
  );
}

The chain. fetchProducts is defined inside the component body without useCallback. JS allocates a new function object on every render. React's useEffect does shallow reference comparison (Object.is) on every dependency; a new function object means the dependency "changed," so the effect re-runs. The effect calls setResults, which schedules a re-render. The re-render allocates a new fetchProducts. Repeat at roughly 24 Hz until the component unmounts or the server gives up.

At 420 active sessions on the product listing page, each generating 23.5 req/sec the moment any search interaction happened: 420 × 23.5 ≈ 9,870 req/sec. The math finally matched the 10,400 req/min spike.

Architecture fix: three layers and a lint rule

A single useCallback would stop the infinite loop. We needed two more layers to make the search production-safe, and a fourth change to keep this class of bug from merging again.

Fix Architecture — Three Layers
────────────────────────────────────────────────────────────
Layer 1: Stabilise the function reference
  fetchProducts = useCallback(fn, [filters])
  → Same JS object across renders unless `filters` prop changes
  → useEffect dep check sees no change → effect does NOT re-fire

Layer 2: Debounce user input (300ms)
  debouncedQuery = useDebounce(query, 300)
  useEffect deps: [debouncedQuery, fetchProducts]
  → API fires 300ms after user stops typing
  → 4-char query "shoe" = 1 request, not 4

Layer 3: Abort in-flight requests (race condition)
  AbortController per effect invocation
  → If debouncedQuery changes before response, cancel previous fetch
  → No stale results rendered out of order

Layer 4: CI lint enforcement
  eslint-plugin-react-hooks: exhaustive-deps → "error" (was "warn")
  → This exact bug fails the lint check before merging

────────────────────────────────────────────────────────────
Before: 1 search interaction → 96 API calls (4 keystrokes × 24/sec)
After:  1 search interaction → 1 API call (300ms after last keystroke)
────────────────────────────────────────────────────────────

// ✅ AFTER: stable reference + debounce + abort
function ProductSearch({ filters }: Props) {
  const [query, setQuery] = useState('');
  const [results, setResults] = useState<Product[]>([]);

  const debouncedQuery = useDebounce(query, 300); // only changes 300ms after last keystroke

  // useCallback returns the SAME function reference across renders.
  // Only reallocated when `filters` actually changes (prop equality).
  const fetchProducts = useCallback(async (q: string, signal: AbortSignal) => {
    if (!q.trim()) { setResults([]); return; }
    const params = new URLSearchParams({ q, ...filters });
    const res = await fetch(`/api/products/search?${params}`, { signal });
    if (!res.ok) throw new Error(`Search failed: ${res.status}`);
    const data = await res.json();
    setResults(data.products);
  }, [filters]); // ← only real dependency; stable across renders

  useEffect(() => {
    const controller = new AbortController();
    fetchProducts(debouncedQuery, controller.signal).catch(err => {
      if (err.name !== 'AbortError') console.error('Search error:', err);
    });
    return () => controller.abort(); // cancel on next effect or unmount
  }, [debouncedQuery, fetchProducts]); // ← both are now stable references

  return (
    <div>
      <input value={query} onChange={e => setQuery(e.target.value)} />
      <ProductList results={results} />
    </div>
  );
}

Why useCallback and not moving the function outside the component? Because fetchProducts closes over filters from props. It has to live inside the component. useCallback memoises it and returns the same reference until filters actually changes.

Why debounce on top? useCallback stops the infinite loop. Debounce stops the per-keystroke barrage. A 10-character query still triggers 10 API calls without debounce. With a 300ms debounce, it's one.

The lint change was the most operationally significant piece of the fix. We already had eslint-plugin-react-hooks configured. exhaustive-deps was set to "warn". The PR had a warning in CI that nobody acted on (including me). Bumping it to "error" caused the pipeline to fail on this exact file. This incident would not have merged under the stricter rule. Which tells you something about warnings in general.

Lessons learned

• Inline functions in useEffect deps are latent infinite loops. Any function defined in a component body without useCallback is a new object each render. If it's in a dependency array, you've got a loop waiting for the first state update to trigger it. The lint rule catches this automatically, if it's set to "error".
• React DevTools Profiler should have been the first tool I reached for. I wasted 20 minutes checking Cloudflare logs for DDoS patterns. The Profiler took 30 seconds and showed continuous re-renders immediately. For any "unexpected API volume" incident, profile the component tree before touching infrastructure.
• "warn" in CI is functionally the same as "off". Warnings that don't block merges get ignored. Hook dependency rules are correctness constraints. Treat them as "error" or remove them.
• Rollback first, investigate second. We spent 4 minutes investigating before pulling the rollback trigger. That's 4 minutes of 429s hitting paying customers. The right protocol is to roll back the moment you correlate a spike with a deploy, then investigate from a stable baseline.
• Rate limiting bought us survivability. Without it, the loop would have sustained 10,000 req/min until the DB connection pool was fully exhausted (a complete outage instead of degraded 429s). Rate limiting is a blast shield. Debounce and stable refs are the fix.

The ESLint rule has fired on two PRs since. Both were caught in CI. Neither reached production.