How an S3 Sync's --delete Flag Turned Every Open Tab Into a ChunkLoadError After Each Deploy

14:32 UTC. Sentry fires an alert group neither of us had seen cross a threshold before: ChunkLoadError, 40 events in three minutes and climbing. Support's queue starts filling with the same complaint in different words: "the site is frozen," "nothing happens when I click."

the setup

storefront-web is a Next.js 14 App Router app. SSR runs on ECS behind an internal load balancer; everything under /_next/static lives in an S3 bucket fronted by CloudFront, uploaded as part of every deploy by a step that had shipped, unquestioned, for over a year.

- name: Sync static assets to S3
  run: |
    aws s3 sync .next/static s3://storefront-static-prod/_next/static \
      --delete \
      --cache-control "public,max-age=31536000,immutable"

The --delete flag was there for a reason someone had a good argument for once: without it, the bucket only grows, hashed chunk from every build sitting there forever. Six months of daily deploys at a few megabytes of JS each adds up, and nobody wanted to explain a surprise S3 bill in a cost review. So every deploy synced the new build's output and removed anything not in it.

The deploy at 14:28 UTC was about as low-risk as they come: a copy change on the returns policy page, no logic touched. It shipped in four minutes, health checks green, no alerts. The alerts started four minutes after that.

the scramble

First theory: the deploy broke something. On-call rolled it back at 14:37, expecting the error rate to drop the way it always did for a bad release. It didn't. If anything, a second, smaller spike followed the rollback within a minute, which ruled out "the new code is broken" as cleanly as a graph can rule anything out, because the new code was already gone.

Second theory: CloudFront was serving garbage, a cache poisoning issue or a misconfigured behavior returning the wrong content type for JS. The CloudFront console showed normal cache hit ratios and no spike in 5xx from the distribution itself, only from the origin on a specific, narrow set of paths.

Third theory, and the one that wasted the most time: an ad blocker or browser extension pattern, since the reports weren't universal, some users were fine, others stuck. That story falls apart once you actually read the Sentry grouping instead of guessing from support tickets. Every single event carried the exact same chunk filename.

ChunkLoadError: Loading chunk 4521 failed.
(error: https://cdn.storefront.example.com/_next/static/chunks/4521-8f2a91cd0e3b4a7f.js)

47 events · affected sessions all opened the site before 14:28 UTC

Not random. Not extension noise. One file, and a sharp cutoff by session start time.

the hunt

A direct request to that URL returned a CloudFront 403, origin-generated, meaning S3 itself didn't have the object anymore, not a permissions or routing issue in front of it.

$ aws s3 ls s3://storefront-static-prod/_next/static/chunks/ | grep 4521-8f2a91cd0e3b4a7f
(no output)

CloudTrail's data events for the bucket answered when and why in one query.

14:28:41Z  DeleteObject  chunks/4521-8f2a91cd0e3b4a7f.js  user=github-actions-deploy-role

The deploy at 14:28 hadn't broken anything in the new build. It had deleted a file the previous build was still depending on. The returns-policy copy edit touched a component that, several imports down, changed the content of a shared chunk enough for webpack to give it a new hash. The old hash, 4521-8f2a91cd0e3b4a7f.js, wasn't in the new build's output directory, so --delete did exactly what it was told and removed it.

That alone wouldn't matter for a fresh page load, the new HTML references the new hash. It matters for every tab that was already open. Next.js's App Router doesn't reload the page shell on navigation, it fetches the next route's JS chunk by the hash baked into the build manifest the browser already has in memory. A tab open since before 14:28 still holds the old manifest, and the old manifest still points at 4521-8f2a91cd0e3b4a7f.js. The click works exactly as designed, right up until the fetch for that file returns a 403 instead of a script.

That's also why the rollback made things worse instead of better. Rolling back is just another deploy: a new sync, with its own --delete, against a build whose chunk hashes matched the original pre-14:28 code. It deleted the interim build's chunks, stranding the tabs that had loaded during the four minutes the bad deploy was live, on top of the tabs already stranded from before. Every deploy through this pipeline broke whichever set of open tabs didn't match whatever was currently in the bucket.

the find

Root cause: the static asset sync treated the S3 bucket as a mirror of the current build instead of an accumulating history of recent builds. --delete is correct for keeping a bucket in sync with a source directory. It is the wrong model for an asset store that browsers keep stale references into for as long as a tab stays open, which for an e-commerce site checking out over lunch or across a slow support call can be hours.

Vercel's own deployment model sidesteps this entirely: every deployment gets its own immutable asset namespace, and old ones stay resolvable until they're explicitly pruned well after the fact. Rolling your own static asset pipeline on S3 means rebuilding that guarantee by hand, and this one never had it.

the fix

The sync step lost --delete entirely. New builds only add.

- name: Sync static assets to S3
  run: |
    aws s3 sync .next/static s3://storefront-static-prod/_next/static \
      --cache-control "public,max-age=31536000,immutable"

Pruning still needs to happen, or the original cost concern comes back in six months. It just can't happen on the same clock as a deploy. A separate job runs nightly, keeps every chunk referenced by the last 10 build manifests no matter how old, and only removes anything outside that window past a 48-hour floor, comfortably longer than any real browser tab survives unattended.

KEEP_MANIFESTS = 10
MIN_AGE_HOURS = 48

referenced = set()
for manifest in recent_manifests(limit=KEEP_MANIFESTS):
    referenced.update(manifest["chunks"])

for obj in list_bucket_objects(BUCKET, prefix="_next/static/chunks/"):
    age_hours = (now() - obj.last_modified).total_seconds() / 3600
    if obj.key not in referenced and age_hours > MIN_AGE_HOURS:
        delete_object(BUCKET, obj.key)

On the client, a global handler now catches a chunk load failure and forces exactly one hard reload, which picks up the current HTML and manifest instead of leaving the user stuck on a dead click.

'use client';

import { useEffect } from 'react';

export function ChunkErrorHandler() {
  useEffect(() => {
    const handler = (event: ErrorEvent | PromiseRejectionEvent) => {
      const message = 'message' in event ? event.message : String((event as PromiseRejectionEvent).reason);
      if (!/ChunkLoadError|Loading chunk [\d]+ failed/.test(message)) return;

      const key = 'chunk-reload-attempted';
      if (sessionStorage.getItem(key)) return;
      sessionStorage.setItem(key, '1');
      window.location.reload();
    };

    window.addEventListener('error', handler);
    window.addEventListener('unhandledrejection', handler);
    return () => {
      window.removeEventListener('error', handler);
      window.removeEventListener('unhandledrejection', handler);
    };
  }, []);

  return null;
}

The sessionStorage guard matters as much as the reload itself: without it, a genuinely broken chunk (a bad deploy, not a stale one) reloads the page into the same error in an infinite loop instead of failing visibly.

the aftermath

Nobody's code was wrong in the sense that usually gets a fix shipped in fifteen minutes. The build was correct, the deploy was healthy, the CDN was doing exactly what it was configured to do. The bug lived in the gap between "what the current build needs" and "what browsers that loaded the previous build still expect," and a flag meant to keep a bucket tidy quietly assumed those were the same set.

The nightly prune job is what keeps the bucket from growing forever. The dropped --delete flag is what stopped deploys from being the thing that broke browser tabs in the first place, and it cost nothing more than a slightly bigger S3 bill nobody has actually complained about since.