A/B testing a headless Next.js store: proxy routing vs static generation

A headless store built with Next.js and Shopify is fast because its product pages are generated ahead of time and served from a cache. An A/B test seems to fight that: if the page is the same file for everyone, how does half your visitors see a different layout? Most teams end up either swapping content in the browser, which flickers, or rendering every product page per request, which throws away the static speed. This guide does neither. Both variations are built as static pages, and Next.js's proxy decides which one each visitor gets, before rendering, with VWO making the decision.

1. The short answer

2. Three ways to test a static storefront

Locally, the rewritten static page and the dynamic page both answered in about 35 to 40 ms, because the product data was cached either way. The difference shows under load and at a CDN: a static variation can be served from a cache, a dynamic page renders every time.

3. Middleware is now Proxy, and where "the edge" is

In Next.js 16 the middleware file convention is deprecated and renamed to proxy, and Proxy defaults to the Node.js runtime; the runtime option isn't available in a proxy file. It still runs before any route is rendered, and it can rewrite, redirect and set cookies. Whether that happens at a CDN edge or on your server depends on where you deploy. With next start, it's your Node process. The design here works either way, because the proxy never renders anything, it only picks a URL.

Two details from the docs matter for testing. Proxy can't see Next's internal RSC headers (rsc, next-router-prefetch), so it can't tell a client-side navigation or a prefetch from a page view. And NextResponse.rewrite() carries those requests through to the right variation.

A request for /products/mug

4. One VWO client per server

VWO's SDK starts with init(), which downloads your project's settings file and sends an SDK-init event. After that, getFlag() is a local calculation: a hash of the user ID against the settings. Calling init() per request puts a VWO round trip in front of every page view. VWO's edge guide warns about this and suggests keeping settings in KV or Redis on platforms that don't keep instances warm.

import {init} from 'vwo-fme-node-sdk'

export const FLAG = 'pdp_layout'      // VWO feature flag with a testing rule; its string variable "layout" names the page variant
export const VARIANTS = ['a', 'b']   // must match the "layout" values in VWO and the static pages under app/pdp/[variant]
export const COOKIE = 'ab_pdp_layout' // rename it for the next test, or visitors keep this test's assignment

const timeout = (ms: number) => new Promise<never>((_, reject) => setTimeout(() => reject(new Error(`VWO: no settings in ${ms} ms`)), ms))

type Client = Awaited<ReturnType<typeof init>>
let current: Client | null = null
let loading: Promise<Client> | null = null
let loadedAt = 0

// One client per server instance, refreshed every 5 minutes. init() downloads the settings file and sends an init
// event, so calling it per request would put a VWO round trip in front of every page view.
export function vwo(): Promise<Client> {
  if (current && Date.now() - loadedAt < 300_000) return Promise.resolve(current)
  loading ??= Promise.race([init({
    accountId: process.env.VWO_ACCOUNT_ID!,
    sdkKey: process.env.VWO_SDK_KEY!,
    logger: {level: 'ERROR'},
    ...(process.env.VWO_GATEWAY_URL && {gatewayService: {url: process.env.VWO_GATEWAY_URL}}),
  }), timeout(1000)]) // a hung settings download must not hold the next request too
    .then((c) => { [current, loadedAt] = [c, Date.now()]; return c })
    .finally(() => { loading = null })
  return current ? Promise.resolve(current) : loading // old settings keep answering while new ones load
}

Time to a getFlag() decision (median of 10)

The 1-second cap matters because of how it fails. With the settings endpoint hanging, the first request waited about 1 second and got the default variation, and so did the next one, instead of queueing behind a download that never finished. When the endpoint came back, the next request loaded the settings and bucketed the visitor.

5. proxy.ts

import {NextResponse, type NextFetchEvent, type NextRequest} from 'next/server'
import {COOKIE, FLAG, VARIANTS, vwo} from '@/lib/vwo'

const cookie = {httpOnly: true, secure: true, sameSite: 'lax' as const, path: '/'}

export async function proxy(req: NextRequest, event: NextFetchEvent) {
  const {pathname} = req.nextUrl

  const uid = req.cookies.get('vwo_uid')?.value ?? crypto.randomUUID()
  let variant = req.cookies.get(COOKIE)?.value
  const known = VARIANTS.includes(variant ?? '')
  if (!known) { // ask VWO once per visitor; later pages, prefetches and RSC requests reuse the cookie and send no impression
    try {
      const client = await vwo()
      const flag = await client.getFlag(FLAG, {id: uid, userAgent: req.headers.get('user-agent') ?? ''})
      variant = flag.isEnabled() ? flag.getVariable('layout', VARIANTS[0]) : undefined
      event.waitUntil(client.flushEvents()) // the SDK batches events by default: send this impression after the response
    } catch (e) {
      console.error('VWO unavailable, serving the default variant:', (e as Error).message)
    }
  }
  const inTest = VARIANTS.includes(variant ?? '')

  const url = req.nextUrl.clone()
  url.pathname = `/pdp/${inTest ? variant : VARIANTS[0]}${pathname.slice('/products'.length)}`
  const res = NextResponse.rewrite(url)
  if (!req.cookies.has('vwo_uid')) res.cookies.set('vwo_uid', uid, {...cookie, maxAge: 365 * 24 * 3600})
  if (inTest && !known) res.cookies.set(COOKIE, variant!, {...cookie, maxAge: 90 * 24 * 3600}) // only real decisions stick
  return res
}

// Only the public URL. Next runs the proxy again for the rewritten /pdp/... request when the matcher covers it,
// so a redirect for /pdp/ here would bounce every visitor; the variant pages point their canonical at /products/ instead.
export const config = {matcher: '/products/:handle'}

• VWO is asked once per visitor. getFlag() sends a vwo_variationShown impression every time it runs. (SDK 1.80's batch getFlags() takes a third argument that suppresses tracking, but then deciding when to send the impression is up to you.) Since the proxy can't tell prefetches and client navigations from page views, asking on every request would send an impression for each of them. With the cookie, one visitor's 13 page views of two products, plus an RSC navigation and a prefetch, produced exactly one impression.
• Only real decisions are stored. If VWO is unreachable or the visitor isn't in the test's traffic, the default page is served and no layout cookie is set, so an outage doesn't lock anyone into the default for 90 days.
• The impression is sent after the response. The SDK batches events by default; flushEvents() in waitUntil sends them without making the visitor wait, and keeps them from being lost when a serverless instance is frozen. In our test the mock was reached through gatewayService, which turns batching off, so events went out one by one.
• The matcher is only the public URL. We first matched /pdp/:path* too, to redirect anyone who found an internal URL. Every visitor got a 301 loop: under next start, the proxy ran a second time for the rewritten /pdp/a/mug request and redirected it. Leave the internal paths out of the matcher.

6. The static variant pages

import {notFound} from 'next/navigation'
import {getHandles, getProduct} from '@/lib/shopify'
import {ProductView} from '@/lib/product-view'
import {VARIANTS} from '@/lib/vwo'

// Every variant of every product is a static page, built ahead and revalidated hourly. proxy.ts only picks which one.
export const revalidate = 3600

export async function generateStaticParams() {
  const handles = await getHandles()
  return VARIANTS.flatMap((variant) => handles.map((handle) => ({variant, handle})))
}

export async function generateMetadata({params}: {params: Promise<{handle: string}>}) {
  const {handle} = await params
  return {alternates: {canonical: `/products/${handle}`}} // one URL for search engines, whatever the variant
}

export default async function Page({params}: {params: Promise<{variant: string; handle: string}>}) {
  const {variant, handle} = await params
  const product = await getProduct(handle)
  if (!product || !VARIANTS.includes(variant)) notFound()
  return <ProductView product={product} layout={variant} />
}

The page never reads cookies or headers, which is what keeps it static. The ProductView component takes the layout as a prop and renders on the server, so both variations are complete HTML at build time. The canonical link points at /products/{handle} on both variations. Don't add noindex to the variant route: it's the same HTML that's served at the public URL, so noindex would apply to your real product page too. A direct visit to /pdp/b/mug renders with the canonical pointing home, and nothing links to it.

7. The caching trap

The rewritten /products/mug response carried Cache-Control: s-maxage=3600, stale-while-revalidate=31532400, the variant page's ISR headers, under the public URL. A cache that keys on the URL, such as a "cache everything" rule at Cloudflare or a CloudFront distribution in front of next start, would store whichever variation it saw first and serve it to everyone, cookies or not.

• If you self-host behind a CDN, exclude tested paths from HTML caching at the CDN, or move the split to the CDN itself. Our WooCommerce guide does that with a Cloudflare Worker that puts the variation in the cache key.
• On a managed Next.js host, check how its cache treats rewrites from Proxy before launching. We didn't test any.

8. Counting purchases from Shopify

The checkout happens on Shopify, not in your Next.js app, so the purchase event has to come from Shopify. Put the visitor's VWO user ID on the cart as an attribute when you create it. Cart attributes become the order's note attributes:

mutation cartCreate($input: CartInput!) {
  cartCreate(input: $input) { cart { id checkoutUrl } }
}
# variables: { "input": { "lines": [...], "attributes": [{ "key": "_vwo_uid", "value": "<vwo_uid cookie>" }] } }

Then a route handler for the orders/create webhook verifies Shopify's HMAC, reads the attribute and sends VWO a purchase event with the order total:

import {createHmac, timingSafeEqual} from 'node:crypto'
import {after} from 'next/server'
import {vwo} from '@/lib/vwo'

const seen = new Set<string>() // ponytail: per-instance dedupe of Shopify retries; use Redis or a DB row if you run several instances

// Shopify orders/create webhook -> VWO "purchase" event for the visitor ID the storefront put on the cart.
export async function POST(req: Request) {
  const raw = await req.text()
  const sig = Buffer.from(req.headers.get('x-shopify-hmac-sha256') ?? '', 'base64')
  const want = createHmac('sha256', process.env.SHOPIFY_WEBHOOK_SECRET!).update(raw).digest()
  if (sig.length !== want.length || !timingSafeEqual(sig, want)) return new Response('bad signature', {status: 401})

  const eventId = req.headers.get('x-shopify-event-id') ?? ''
  const order = JSON.parse(raw)
  const uid = order.note_attributes?.find((a: {name: string}) => a.name === '_vwo_uid')?.value
  if (uid && !seen.has(eventId)) {
    seen.add(eventId)
    const client = await vwo()
    await client.trackEvent('purchase', {id: uid}, {amount: Number(order.current_total_price)})
    after(() => client.flushEvents())
  }
  return new Response(null, {status: 200})
}

Shopify retries webhooks, so the handler skips an event ID it has already processed; with more than one server instance, keep those IDs in Redis or a database instead of memory. We tested the handler, not the cartCreate call.

9. Test results

Not tested: Vercel or another managed host, the Edge runtime, a live VWO account and its reports, VWO's batching path (our mock route disables it), and the cartCreate mutation.

10. Our opinion: trade-offs

• Static variants are the right default for layout and content tests on product and collection pages. You keep the build-time speed and the test can't flicker.
• The cost is build size: two variations double the prerendered pages for the tested route. Test on the routes you mean to, and prerender only your top products if the catalog is large; the rest generate on first request with revalidate.
• Per-user personalization doesn't fit this model. If the page depends on who the visitor is, not which of two layouts they got, render it dynamically or stream that part.
• Keep the experiment logic in the proxy small. It runs before every matched request; a settings download or an extra network call there is felt on every page.
• Rename the cookie for each new test. Otherwise visitors carry the last test's assignment into the next one.

Related: ISR and on-demand revalidation for a headless Shopify store and adding reviews to headless Shopify without hurting LCP.

11. Tools

Top pick

Alternatives: free tools and references

We don't earn anything from these links.

• Flags SDK: Vercel's open-source library for feature flags in Next.js, with a "precompute" pattern for static variants.
• GrowthBook: open-source experimentation you can host yourself.
• Next.js: proxy.js reference: runtime, matchers, cookies and the RSC header note quoted here.

12. Pitfalls

• Reading cookies in the product page to choose the variation: the route stops being static.
• Swapping layouts in a client component: flicker, and hydration warnings when server and client disagree.
• Calling init() in the proxy on every request: a settings download per page view.
• Calling getFlag() on every request: an impression for every prefetch and navigation.
• No timeout on the settings download: a VWO outage becomes slow pages for everyone.
• Matching the internal variant paths in the proxy: it runs again on the rewrite.
• A URL-keyed CDN caching the public URL: one variation for everyone.
• noindex on the variant route: it lands on your real product page.
• No user ID on the cart: purchases can't be tied back to a variation.