How secure is it to use fragment identifiers to hold private data in URLs?

Question

We know the URL itself is not a secure way to pass or store information. Too many programs will perform unexpected processing on the URL or even ship it over the network, and generally speaking, the URL is not treated with a high regard for its privacy.

In the past we've seen Bitcoin wallets, for example, which have relied on keeping a URL secret, but they found out the hard way there are too many ways in which a URL (sent via Skype, or emailed, or even just typing it into the Google Chrome omnibar) will get stored by a remote server, and possibly displayed publicly.

And so I thought URL would be forsaken forever as a means for carrying any private data... despite being extremely convenient, except now I've seen a few sites which are using URL fragments -- the portion of the URL after the '#' -- as a kind of 'secure' storage. I think the expectation is that Google won't parse the fragment and allow it to show up in search results, so that data shouldn't be published.

But that seems like a pretty weak basis for the security of your product. There would be a huge benefit to having a way to securely move data in URL fragments, but can we really rely on that?

So, I would really like to understand... Can anyone explain, what is the security model for fragment identifiers?

Hebdomad

I am having no idea of what you are asking, are you are talking about the anchor, or you are talking about GET parameters – Jumada

Anchor text can most certainly be indexed by Google, and show up in search results. It is a very important SEO strategy. What? – Chiliarch

I'm talking about the component of the URL which comes after '#'. OK, apparently it's called a named anchor or also a 'fragment identifier'. en.wikipedia.org/wiki/Fragment_identifier – Hebdomad

Answer 1

Tyler Close and others who did the security architecture for Waterken did the relevent research form this. They use unguessable strings in URI fragments as web-keys:

This leakage of a permission bearing URL via the Referer header is only a problem in practice if the target host of a hyperlink is different from the source host, and so potentially malicious. RFC 2616 foresaw the danger of such leakage of information and so provided security guidance in section 15.1.3:

"Because the source of a link might be private information or might reveal an otherwise private information source, … Clients SHOULD NOT include a Referer header field in a (non-secure) HTTP request if the referring page was transferred with a secure protocol."

Unfortunately, clients have implemented this guidance to the letter, meaning the Referer header is sent if both the referring page and the destination page use HTTPS, but are served by different hosts.

This enthusiastic use of the Referer header would present a significant barrier to implementation of the web-key concept were it not for one unrelated, but rather fortunate, requirement placed on use of the Referer header. Section 14.36 of RFC 2616, which governs use of the Referer header, states that: "The URI MUST NOT include a fragment." Testing of deployed web browsers has shown this requirement is commonly implemented.

Putting the unguessable permission key in the fragment segment produces an https URL that looks like: <https://www.example.com/app/#mhbqcmmva5ja3>.

Fetching a representation

Placing the key in the URL fragment component prevents leakage via the Referer header but also complicates the dereference operation, since the fragment is also not sent in the Request-URI of an HTTP request. This complication is overcome using the two cornerstones of Web 2.0: JavaScript and XMLHttpRequest.

---

So, yes, you can use fragment identifiers to hold secrets, though those secrets could be stolen and exfiltrated if your application is susceptible to XSS, and there is no equivalent of http-only cookies for fragment identifiers.

I believe Waterken mitigates this by removing the secret from the fragment before it runs any application code in the same way many sensitive daemons zero-out their argv.

Hargreaves

Answer 2

The part after the # is not any more secure than any other part of the URL. The only difference is that it MAY be omitted from the web server access log. But the web server is not the threat.

As long as you store the secret, either in a URL or somewhere else where it can become public it is insecure. That is why we invented passwords, because they are supposed to only exist in peoples head.

The problem is not to find a way to store a secret in a URL. That is impossible, because as you say: The probably will become public. If all you need is the URL, and it gos public, nobody cares what the original data is. Bacuse they have what they need, the URL. So to rely on the URL alone for authentication is.. moronic.

The The problem is to store your secrets in a secure way, and to create secure systems.

Biffin

What do you mean by “anchor tag”, like <a>? – Browne

@Browne Sorry about that. I was off course talking about the part after the #. – Biffin

That would be known as fragment identifier. – Browne

The fragment is not sent to the server. – Hargreaves

No, but usually you want to give the secret to the server at some point. So the server is not who you want to protect it from. – Biffin

Secrets in fragment identifiers are not hashed, encrypted, or obfuscated in any way. That doesn't mean they're useless, but it does put limits around their use. But if simply pasting the fragment into a browser reveals the secret, e.g. by getting it indexed by Google, certainly then it is completely useless. – Hebdomad

'The part after the # is not any more secure than any other part of the URL.' That is incorrect. As another answer cites, 'The URI MUST NOT include a fragment' in a Referer header. Since it is stripped during referral (in theory as well as in practice), it is more secure than the rest of the URL. The use of the word 'moronic' is unconstructive. – Walling