Split service worker cache

[eugene.shul]

Hello, we have a problem with splitting service worker cache.
Example of out config

const key = new CustomCacheKey()
  .excludeAllQueryParametersExcept(
    'q'
  )
  .addCookie('cookieName')

SERVICE_WORKER: {
    key,
    browser: {
      maxAgeSeconds: SERVICE_WORKER,
    },
    edge: {
      maxAgeSeconds: SERVICE_WORKER,
      staleWhileRevalidateSeconds: 60 * 60 * 24,
    },
  },

And when we change the cookie value “cookieName”, and then navigating through the site we still getting cached pages from service worker.
Could you help me to solve the problem? What additional information should i provide?
Thanks, best regards

[mark.brocato]

What you’re doing looks correct. Please make sure you’ve cleared your service worker cache and unregistered the old service worker before testing any changes to the caching logic.

[eugene.shul]

@mark.brocato

make sure you’ve cleared y

Yeah, i did all this stuff. Also i tried with incognito mode as well.
But it is not working properly
Steps:

1. Clear service worker cache and unregistered the old service worker
2. Did some actions with the site
3. Change cookie value
4. I can see the cached value when navigation through the site
   But i expect then when i change cookie value i will not have cache, it should split the cache by “key”

[mark.brocato]

Alright, we will try to reproduce this internally and let you know what we find.

[eugene.shul]

Thanks, will wait your answer

[depnerd]

Any updates here @mark.brocato ?

[tristan.lee]

Eugene,

I have tried locally using our Next.js example app, configuring the cache key just as you described:

cache({
  key: new CustomCacheKey()
    .excludeAllQueryParametersExcept('q')
    .addCookie('cookieName'),
  browser: {
    maxAgeSeconds: 60 * 60 * 24 * 365 * 10,
    serviceWorkerSeconds: 60 * 60 * 24,
  },
  edge: {
    maxAgeSeconds: 60 * 60 * 24 * 365 * 10,
    staleWhileRevalidateSeconds: 60 * 60 * 24,
  },
});

I set my cookie to cookieName=foo, make requests to the different categories and observe a cached response. I close the browser (incognito), open again and set the cookie to cookieName=bar and make the same requests and get a different response.

Any more details you can provide such as a forked example of the above app or screenshots to show what you are experiencing?

[eugene.shul]

Thanks for trying But the main problem that we did’nt close the browser
Could you try this scenario but do not close the browser.
Example

1. Default Cookie: cookieName=foo
2. Make requests to the different categories and observe a cached response
3. Press some button change out cookie to cookieName=foo and DO NOT close the browser

[tristan.lee]

Are you sure the subsequent requests after you change the cookie value are going through the service worker? A screenshot of the network requests that you are seeing in the browser would be helpful here.

[eugene.shul]

Yes, you can see the video attached
When we toggle this switch we change cookie value (see video)

And then after changing as you can see request for “/rugs” comes from service worker

You can see all steps:

You can see full video here (DropMeFiles – free one-click file sharing service) link expire at 14 days

Is it clear now? Or maybe you need some more information?
Thanks!

[tristan.lee]

I think the issue may be that you have caching value for both the browser and service worker. Set the config to something like:

browser: {
  maxAgeSeconds: 0,
  serviceWorkerSeconds: 60 * 60 * 24, // whatever limit you desire
}

[eugene.shul]

Thanks!
I have added configuration as you described
But the issue still present here((
Do we have some another ways to fix it?

[tristan.lee]

Can you provide your entire router and any related source code that shows how this is configured?

Also make sure your browser cache is completely cleared to verify this change.

[eugene.shul]

sure,

const { CustomCacheKey } = require('@layer0/core/router')

/**
 * 24 hours
 */
const PAGE_TTL = 60 * 60 * 24

const IMAGE_TTL = 60 * 60 * 24

/**
 * 10 years
 */
const FAR_FUTURE_TTL = 60 * 60 * 24 * 365 * 10

const SERVICE_WORKER = 60 * 60 * 24 * 365

/**
 * 2 weeks
 */
const COUNTRIES_TTL = 60 * 60 * 24 * 7 * 2

/**
 * One week
 */
const ONE_WEEK_TTL = 60 * 60 * 24 * 7

/**
 * The custom cache key for all SSR and API responses.  Here we ignore all
 * query parameters except ones that we know our app specifically handles.  This
 * prevents cache fragmentation due to unexpected query parameters added in links
 * from 3rd parties.
 */
const key = new CustomCacheKey()
  .excludeAllQueryParametersExcept(
    'q',
    'sort',
    'filters',
    'color',
    'size',
    'amp',
    'page',
    'type',
    'productId',
    'productIds',
    'category',
    'token',
    'endpoint'
  )
  .addCookie('x-xdn-geo-country-code')
  .addCookie('countryCode')
  .addCookie('xRsfUserLevel')
  .addCookie('xRsfMsrpStatus')

const imagesCustomCacheKeys = new CustomCacheKey().excludeAllQueryParametersExcept(
  'cache',
  'width',
  'height',
  'quality',
  'format'
)

module.exports = {
  FAR_FUTURE_TTL,
  PAGE_TTL,
  IMAGE_TTL,
  key,

  /**
   * The cache config for all server side rendered pages
   */
  SSR: {
    key,
    browser: {
      maxAgeSeconds: 0,
    },
    edge: {
      forcePrivateCaching: true,
      maxAgeSeconds: PAGE_TTL,
      staleWhileRevalidateSeconds: PAGE_TTL,
    },
  },

  /**
   * The cache config for static assets
   */
  STATIC_ASSETS: {
    key,
    browser: {
      maxAgeSeconds: ONE_WEEK_TTL,
    },
    edge: {
      forcePrivateCaching: true,
      maxAgeSeconds: ONE_WEEK_TTL,
      staleWhileRevalidateSeconds: 60 * 60 * 24,
    },
  },

  /**
   * The cache config for all server side rendered pages
   */
  API: {
    key,
    browser: {
      maxAgeSeconds: 0,
      serviceWorkerSeconds: PAGE_TTL,
    },
    edge: {
      forcePrivateCaching: true,
      maxAgeSeconds: PAGE_TTL,
      staleWhileRevalidateSeconds: PAGE_TTL,
    },
  },

  /**
   * The cache config for all images of scene7
   */
  IMAGE: {
    key: imagesCustomCacheKeys,
    browser: {
      maxAgeSeconds: 0,
      serviceWorkerSeconds: IMAGE_TTL,
    },
    edge: {
      forcePrivateCaching: true,
      maxAgeSeconds: IMAGE_TTL,
      staleWhileRevalidateSeconds: IMAGE_TTL,
    },
  },

  /**
   * Cache config for static assets that never change and thus can be cached
   * at the edge and in the browser forever.
   */
  FAR_FUTURE: {
    key,
    browser: {
      maxAgeSeconds: FAR_FUTURE_TTL,
    },
    edge: {
      forcePrivateCaching: true,
      maxAgeSeconds: FAR_FUTURE_TTL,
      staleWhileRevalidateSeconds: 60 * 60 * 24,
    },
  },
  SERVICE_WORKER: {
    key,
    browser: {
      maxAgeSeconds: SERVICE_WORKER,
    },
    edge: {
      maxAgeSeconds: SERVICE_WORKER,
      staleWhileRevalidateSeconds: 60 * 60 * 24,
    },
  },
  NO_CACHE: {
    browser: false,
    edge: false,
  },

  /**
   * The cache config for shipping address countries
   */
  COUNTRIES: {
    key,
    browser: {
      maxAgeSeconds: 0,
      serviceWorkerSeconds: COUNTRIES_TTL,
    },
    edge: {
      maxAgeSeconds: COUNTRIES_TTL,
      staleWhileRevalidateSeconds: COUNTRIES_TTL,
    },
  },

  /**
   * Creates a route handler that caches based on the specified config
   * @param {Object} config A config for @layer0/core's cache function
   * @return {Function} a route handler
   */
  cacheResponse: config => ({ cache }) => cache(config),
}

and routes

const { Router } = require('@layer0/core/router')
const { nextRoutes, renderNextPage } = require('@layer0/next')
const {
  API,
  SSR,
  IMAGE,
  NO_CACHE,
  SERVICE_WORKER,
  COUNTRIES,
  STATIC_ASSETS,
  cacheResponse,
} = require('./cache')
const prerenderRequests = require('./layer0/prerenderRequests')

function checkCacheRules(res) {
  if (res?.request?.headers?.referer?.match(/token=/)) {
    return res.cache(NO_CACHE)
  }
  if (res?.request?.query?.token?.match(/.+/)) {
    return res.cache(NO_CACHE)
  }
  if (res?.request?.url?.includes('/api')) {
    return res.cache(API)
  }
  return res.cache(SSR)
}

module.exports = () => {
  const pwaRouter = new Router()

    pwaRouter
      .prerender(prerenderRequests)
      .match('/service-worker.js', async ({ cache, serveStatic }) => {
        cache(SERVICE_WORKER)
        await serveStatic('.next/static/service-worker.js')
      })


  pwaRouter
    .match(
      {
        path: '(.*)',
        headers: { referer: /token=/ },
      },
      cacheResponse(NO_CACHE)
    )
    .match(
      {
        path: '(.*)',
        query: { token: /.+/ },
      },
      cacheResponse(NO_CACHE)
    )
    .match('/api/p/:productId*', cacheResponse(API))
    .match('/api/p/:productId*/data', cacheResponse(API))
    .match('/p/:productId*', cacheResponse(SSR))
    .match('/api/c/:categorySlug*', cacheResponse(API))
    .match('/c/:categorySlug*', cacheResponse(SSR))
    .match('/api/sign-in', cacheResponse(NO_CACHE))
    .match('/api/signOut', cacheResponse(NO_CACHE))
    .match('/api/signUp', cacheResponse(NO_CACHE))
    .match('/api/session', cacheResponse(NO_CACHE))
    .match('/api/suggestions', cacheResponse(API))
    .match('/api/user/:id*', cacheResponse(NO_CACHE))
    .match('/api/subscribe', cacheResponse(NO_CACHE))
    .match('/api/getStores', cacheResponse(API))
    .match('/api/cart/getEntries', cacheResponse(NO_CACHE))
    .match('/api/cart/getCurrent', cacheResponse(NO_CACHE))
    .match('/api/cart/withPromoCode', cacheResponse(NO_CACHE))
    .match('/api/orders/getList', cacheResponse(NO_CACHE))
    .match('/api/s7image/:path*', cacheResponse(IMAGE))
    .match('/theme', cacheResponse(SSR))
    .match('/api/theme', cacheResponse(API))
    .match('/api/widgets/branded-products', cacheResponse(API))
    .match('/api/widgets/cart-dont-forget', cacheResponse(API))
    .match('/api/widgets/cart-recommended-products', cacheResponse(API))
    .match('/api/widgets/cart-style-with', cacheResponse(API))
    .match('/api/widgets/frequently-viewed-together', cacheResponse(API))
    .match('/api/widgets/more-like-this', cacheResponse(API))
    .match('/api/widgets/our-branded-products', cacheResponse(API))
    .match('/api/widgets/people-buy', cacheResponse(API))
    .match('/api/widgets/product-complete-room', cacheResponse(API))
    .match('/api/widgets/product-perfect-proportions', cacheResponse(API))
    .match('/api/widgets/related-searches', cacheResponse(API))
    .match('/api/widgets/style-with', cacheResponse(API))
    .match('/api/widgets/top-reviewed-products', cacheResponse(API))
    .match('/api/widgets/trending-products', cacheResponse(API))
    .match('/api/widgets/viewed-together', cacheResponse(API))
    .match('/api/widgets/you-may-also-like', cacheResponse(API))
    .match('/api/widgets/you-may-also-like-category', cacheResponse(API))
    .match('/api/countries', cacheResponse(COUNTRIES))
    .match('/api/helpers/categories', cacheResponse(API))
    .match('/:file*.js', cacheResponse(STATIC_ASSETS))
    .match('/:file*.css', cacheResponse(STATIC_ASSETS))
    .match('/:file*.aif', cacheResponse(STATIC_ASSETS))
    .match('/:file*.bmp', cacheResponse(STATIC_ASSETS))
    .match('/:file*.doc', cacheResponse(STATIC_ASSETS))
    .match('/:file*.gif', cacheResponse(STATIC_ASSETS))
    .match('/:file*.jpeg', cacheResponse(STATIC_ASSETS))
    .match('/:file*.jpg', cacheResponse(STATIC_ASSETS))
    .match('/:file*.png', cacheResponse(STATIC_ASSETS))
    .match('/:file*.otf', cacheResponse(STATIC_ASSETS))
    .match('/:file*.mov', cacheResponse(STATIC_ASSETS))
    .match('/:file*.pdf', cacheResponse(STATIC_ASSETS))
    .match('/:file*.txt', cacheResponse(STATIC_ASSETS))
    .match('/:file*.wbmp', cacheResponse(STATIC_ASSETS))
    .match('/:file*.tif', cacheResponse(STATIC_ASSETS))
    .match('/:file*.tiff', cacheResponse(STATIC_ASSETS))
    .match('/:file*.ttf', cacheResponse(STATIC_ASSETS))
    .match('/:file*.eot', cacheResponse(STATIC_ASSETS))
    .match('/:file*.woff', cacheResponse(STATIC_ASSETS))
    .match('/:file*.woff2', cacheResponse(STATIC_ASSETS))
    .match('/:file*.svg', cacheResponse(STATIC_ASSETS))

  pwaRouter
    .use(nextRoutes)
    .match('/(.*)', res => {
      checkCacheRules(res)
      if (res?.request?.url?.includes('/api')) {
        return renderNextPage('/api/proxy/[...params]', res, { url: res?.request?.url })
      }
      return renderNextPage('/proxy/[...params]', res, { url: res?.request?.url })
    })
    .fallback(({ proxy }) => proxy('legacy'))

  // For Split testing to legacy
  const legacyRouter = new Router()
    .match('/:file*.js', cacheResponse(STATIC_ASSETS))
    .match('/:file*.css', cacheResponse(STATIC_ASSETS))
    .match('/:file*.aif', cacheResponse(STATIC_ASSETS))
    .match('/:file*.bmp', cacheResponse(STATIC_ASSETS))
    .match('/:file*.doc', cacheResponse(STATIC_ASSETS))
    .match('/:file*.gif', cacheResponse(STATIC_ASSETS))
    .match('/:file*.jpeg', cacheResponse(STATIC_ASSETS))
    .match('/:file*.jpg', cacheResponse(STATIC_ASSETS))
    .match('/:file*.png', cacheResponse(STATIC_ASSETS))
    .match('/:file*.otf', cacheResponse(STATIC_ASSETS))
    .match('/:file*.mov', cacheResponse(STATIC_ASSETS))
    .match('/:file*.pdf', cacheResponse(STATIC_ASSETS))
    .match('/:file*.txt', cacheResponse(STATIC_ASSETS))
    .match('/:file*.wbmp', cacheResponse(STATIC_ASSETS))
    .match('/:file*.tif', cacheResponse(STATIC_ASSETS))
    .match('/:file*.tiff', cacheResponse(STATIC_ASSETS))
    .match('/:file*.ttf', cacheResponse(STATIC_ASSETS))
    .match('/:file*.eot', cacheResponse(STATIC_ASSETS))
    .match('/:file*.woff', cacheResponse(STATIC_ASSETS))
    .match('/:file*.woff2', cacheResponse(STATIC_ASSETS))
    .match('/:file*.svg', cacheResponse(STATIC_ASSETS))
    .fallback(async ({ proxy, cache }) => {
      cache({
        edge: false,
      })
      await proxy('legacy')
    })

  return new Router().destination('pwa', pwaRouter).destination('legacy', legacyRouter)
}

[tristan.lee]

Upon further investigation into this, the issue isn’t related to the cache splitting of the service worker itself, but rather the requests it handles. Service worker cache storage is keyed based on the URL path along with any query params that are black/white-listed. The cache key for these requests will not be split based on cookie because in this context, the request headers (aka cookies) are not made available to the service worker. So at this time, we currently do not support splitting cache of service worker requests based on cookie values.