BigCommerce on XDN - robots.txt and sitemap are not available

Issues
1

We recently launched a bigcommerce site using XDN Starter and noticed today the sitemap.xml and robots.txt are not being served from our site and are disallowing all traffic. Our XDN config is:

module.exports = {
  routes: './src/routes.ts',
  backends: {
    origin: {
      domainOrIp: 'store-abcdefg.mybigcommerce.com',
      hostHeader: 'store-abcdefg.mybigcommerce.com',
    },
  },
}
2

BigCommerce has a specific robots.txt and sitemap.xml for domains that aren’t the primary for a store and prevents crawling on secondary domains.

To resolve this you should specify that the XDN communicate with the bigcommerce store using the primary domain hostheader, so your xdn.config will look like:

module.exports = {
  routes: './src/routes.ts',
  backends: {
    origin: {
      domainOrIp: 'store-abcdefg.mybigcommerce.com',
      hostHeader: 'www.YourStore.com',
    },
  },
}

This will make sure the correct robots.txt and sitemap.xml pass through from BigCommerce.