How do I opt out of HTTP/2 server push when using fetch?

Question

I am writing a basic app in Javascript that uses the new fetch API. Here is a basic example of the relevant portion of the code:

function foo(url) {
  const options = {};
  options.credentials = 'omit';
  options.method = 'get';
  options.headers = {'Accept': 'text/html'};
  options.mode = 'cors';
  options.cache = 'default';
  options.redirect = 'follow';
  options.referrer = 'no-referrer';
  options.referrerPolicy = 'no-referrer';
  return fetch(url, options);
}

When making a fetch request I occasionally see errors appear in the console that look like the following:

Refused to load the script '<url>' because it violates the following Content Security Policy directive ...

After some reading and learning about HTTP/2, it looks like this message appears because the response is pushing back a preloaded script. Using devtools, I can see the following header in the response:

link:<path-to-script>; rel=preload; as=script

Here is the relevant portion of my Chrome extension's manifest.json file:

{
  "content_security_policy": "script-src 'self'; object-src 'self'"
}

Here is documentation on Chrome's manifest.json format, and how the content security policy is applied to fetches made by the extension: https://developer.chrome.com/extensions/contentSecurityPolicy

I did some testing and was able to determine that this error message happens during fetch, not later when parsing the response text. There is no issue where a script element gets loaded into a live DOM, this all happens at the time of the fetch.

What I was not able to find in my research was how to avoid this behavior. It looks like in the rush to support this great new feature, the people that made HTTP/2 and fetch did not consider the use case where I am not fetching the remote page for the purpose of displaying it or any of its associated resources like css/image/script. I (the app) will not ever later be using any associated resource; only the content of the resource itself.

In my use case, this push (1) is a total waste of resources and (2) is now causing a really annoying and stress-inducing message to sporadically appear in the console.

With that said, here is the question I would love some help with: Is there a way to signal to the browser, using manifest or script, that I have no interest in HTTP/2 push? Is there a header I can set for the fetch request that tells the web server to not respond with push? Is there a CSP setting I can use in my app manifest that somehow triggers a do-not-push-me response?

I've looked at https://w3c.github.io/preload/ section 3.3, it was not much help. I see that I can send headers like Link: </dont/want/to/push/this>; rel=preload; as=script; nopush. The problem is that I do not already know which Link headers will be in the response, and I am not sure if fetch even permits setting Link headers in the initial request. I wonder if I can send some type of request that can see the Link headers in the response but avoids them, then send a followup request that appends all the appropriate nopush headers?

Here is a simple test case to reproduce the issue:

1. Get a dev version of latest or near latest chrome
2. Create an extension folder
3. Create manifest with similar CSP
4. Load extension as unpacked into chrome
5. Open up the background page for the extension in devtools
6. In console type fetch('https://www.yahoo.com').
7. Examine the resulting error message that appears in the console: Refused to load the script 'https://www.yahoo.com/sy/rq/darla/2-9-20/js/g-r-min.js' because it violates the following Content Security Policy directive: "script-src 'self'".

Additional notes:

• I do not want to use a proxy server. A clear explanation as to why that would be my only option would be an acceptable answer.
• I do not know the urls that will be fetched at the time of configuring the CSP.
• See https://www.rfc-editor.org/rfc/rfc7540#section-6.5.1 which states in relevant part that "SETTINGS_ENABLE_PUSH (0x2): This setting can be used to disable server push (Section 8.2). An endpoint MUST NOT send a PUSH_PROMISE frame if it receives this parameter set to a value of 0." Is there a way to specify this setting from script or manifest or is it baked into Chrome?

Cami

Why can't you just remove the offending header in the response? – Foskett

@Foskett According to developer.mozilla.org/en-US/docs/Web/API/Response/headers, Response.headers is read-only. Also, my understanding is that this happens as the response is received, there is no time or place for my script to jump into the middle of the processing and remove the offending header. The browser handles all of it. I only get access at the end. – Cami

Sorry for not being clear. I meant, why can't you remove the offending header on the server-side? – Foskett

Ah. Well I have no control over the server, or any other server. I am writing an app in javascript that sends fetch requests to a variety of other servers. There is not even a pre-determined or apriori-compiled list of which other servers are contacted. – Cami

Okey, I see your pain now. Are you sure then that you not going against plain-old CORS protections? – Foskett

@Foskett I am not sure. – Cami

This answer may give some hints. – Walke

@K3N Thanks! I checked the question, did not help. I am aware of how to add custom urls to csp. Do not want to do that, and cannot do that because the app does not know which urls will be accessed. I do not want to suppress the cors error message to allow for JS. I want to opt out of server push entirely. – Cami

Why do you believe HTTP/2 is related to Question? – Rexford

@Rexford My understanding is that server push is a feature of HTTP/2 – Cami

Does the server that you are requesting from implement HTTP/2? Is the issue the preflight request Why does Fetch API Send the first PUT request as OPTIONS? – Rexford

@Rexford I am having a harder time answering this than expected. I looked at the response headers and the only mention of the protocol used is in the via header: http/1.1 ir9.fp.bf1.yahoo.com (ApacheTrafficServer). I have noticed that OPTIONS requests sometimes appear on the Network tab of devtools in Chrome after running my script. I am specifying cors mode in fetch options as shown in my question, so this is expected, and is not the thing giving me trouble. – Cami

@Rexford Oops. Actually the request is http/2, I see "h2" in the protocol column for the request. I do not know if this means the server implements HTTP/2, but I assume so. – Cami

@Cami Have you tried making an OPTIONS request first #42311518? – Rexford

@Rexford I have not tried that. I was hoping to avoid explicitly making additional requests given that I view this as the browser's responsibility, but I am out of options so I guess I will try it out. – Cami

@Rexford The same issue of 'refused to load script...' occurs when sending an OPTIONS request. – Cami

@Cami Not entirely sure what requirement is? – Rexford

What are you trying to achieve? – Rexford

@Rexford I am trying to avoid making requests that result in responses being sent with Link headers. – Cami

Have you tried using YQL as a proxy to make request? – Rexford

I have not tried that – Cami

Answer 1

After following along with your test case I was able to resolve this (example of the) issue in the following way, though I don't know that it applies to all more general cases:

1. Use chrome.webRequest to intercept responses to the extension's requests.
2. Use the blocking form of onHeadersRecieved to strip out headers containing rel=preload
3. Allow the response to proceed with the updated headers.

I have to admit I spent a lot of time trying to figure out why this seemed to work, as I don't think stripping the Link headers should work in all cases. I thought that Server Push would just start pushing files after the request is sent.

As you mentioned in your additional note about SETTINGS_ENABLE_PUSH much of this is in fact baked into chrome and hidden from our view. If you want to dig deeper I found the details at chrome://net-internals/#http2. Perhaps Chrome is killing files sent by Server Push that don't have a corresponding Link header in the initial response.

This solution hinges on chrome.webRequest Docs

---

The extension's background script:

let trackedUrl;

function foo(url) {
  trackedUrl = url;
  const options = {};
  options.credentials = 'omit';
  options.method = 'get';
  options.headers = { 'Accept': 'text/html' };
  options.mode = 'cors';
  options.cache = 'default';
  options.redirect = 'follow';
  options.referrer = 'no-referrer';
  options.referrerPolicy = 'no-referrer';
  return fetch(url, options)
}

chrome.webRequest.onHeadersReceived.addListener(function (details) {
  let newHeaders;
  if (details.url.indexOf(trackedUrl) > -1) {
    newHeaders = details.responseHeaders.filter(header => {
      return header.value.indexOf('rel=preload') < 0;
    })
  }

  return { responseHeaders: newHeaders };
}, { urls: ['<all_urls>'] }, ['responseHeaders', 'blocking']);

---

The extension's manifest:

{
  "manifest_version": 2,
  "name": "Example",
  "description": "WebRequest Blocking",
  "version": "1.0",
  "browser_action": {
    "default_icon": "icon.png"
  },
  "background": {
    "scripts": [
      "back.js"
    ]
  },
  "content_security_policy": "script-src 'self'; object-src 'self'",
  "permissions": [
    "<all_urls>",
    "background",
    "webRequest",
    "webRequestBlocking"
  ]
}

Additional Notes:

• I'm just naively limiting this to the latest request url from the extension, there are webRequest.requestFilters baked into chrome.webRequest you can check out here

• You'll probably also want to be much more specific about which headers you strip. I feel like stripping all the Links will have some additional effects.

• This avoids proxys and does not require setting a Link header in the request.

• This makes for a pretty powerful extension, personally I avoid extensions with permissions like <all_urls>, hopefully you can narrow the scope.

• I did not test for delays caused by blocking the responses to delete headers.

Galla

Thank for you the great effort. I am running into a few problems. chrome.webRequest requires the permissions. So I try adding the permissions, and now Chrome refuses to load the extension and says that webRequest cannot be used for event pages. My background page is an event page. Hmm. – Cami

My understanding of event pages is that they load on demand, and chrome.webRequest has the potential to fire so often that it doesn't make sense on a lazy-loading page. Could you convert to a background page? Although event pages are preferred, background pages still work with the "persistent": true flag more info – Galla

Thanks, I am looking into this. Cannot do much right now but if it eventually works will mark as accepted. – Cami

Any chance you see how to easily restrict urls to only particular fetches made by the extension, and not all requests? – Cami

Sure, it wouldn't be difficult, you'd just set trackedUrl = url in foo() when you want to clear Link headers from the following fetch. For a fetch where you don't want to do this, set trackedUrl = '<>' or any other thing which doesn't occur in urls. The result is the the the trackedUrl with invalid characters will never be matched, and thus the Link headers will not be cleared. There are better ways to do all this if you have more defined requirements though, or if you have multiple simultaneous requests. The performance overhead of this tiny snippet is next to nothing. – Galla

Eh, I cannot get it working with declarativeRequest, as the documentation is hilariously confusing, but I'll accept for now since I am going down this path. Really wanted to avoid a hack but this is good enough and I cannot deny I am in an extension context in the first place. – Cami

Yeah, declarativeRequest is in some form of permanent beta. Unfortunately since you are trying to achieve something that is contrary to the direction Chrome is headed (all things HTTP/2) - but which is dependent on and built within Chrome - you've got to hack your solution a bit. I agree that it isn't very satisfying to have a solution without knowing why it works or if it will continue to work. – Galla