Authoritative position of duplicate HTTP GET query keys
Asked Answered
C

7

160

I am having trouble on finding authoritative information about the behavior with HTTP GET query string duplicate fields, like

http://example.com/page?field=foo&field=bar 

and in particular if the order is kept or not. Most web-oriented languages produce an array containing both foo and bar associated to a key "field", but I would like to know if authoritative statement exist (e.g. on a RFC) about this point. RFC 3986 has a section 3.4. Query, which refers to key=value pairs, but nothing is said on how to interpret order and duplicate fields and so on. This makes sense, since it's backend dependent, and not in the scope of that RFC...

Although a de-facto standard exists, I'd like to see an authoritative source for it, just out of curiosity.

Computerize answered 17/11, 2009 at 4:4 Comment(3)
Been wondering about that, too. The other thing is the spec about merging the parameters from the query string with those in the POST body.Charily
Over at the code ranch, people say there is no order guarantee. But that thread is old and no one backs it up in any way: coderanch.com/t/357197/Servlets/java/getParameterValues-orderCharily
In addition to the server keeping the order of the query string, there is also the question about the browser sending them in DOM (or some other fixed) order.Charily
C
21

The situation seems to have changed since this question was asked and the accepted answer was written 12 years ago. I believe we now have an authoritative source: The WHATWG URL Standard describes the process of extracting and parsing a query string in detail in section 6.2 (https://url.spec.whatwg.org/#interface-urlsearchparams) and section 5.1 on x-www-form-urlencoded parsing (https://url.spec.whatwg.org/#urlencoded-parsing). The parsing output is "an initially empty list of name-value tuples where both name and value hold a string", where a list is defined as a finite ordered sequence, and the key-value pairs are added to this list in the order they appear in the URL. At first there is no mention of repeated keys, but some methods on the URLSearchParams class in section 6.2 (https://url.spec.whatwg.org/#interface-urlsearchparams) set clear expectations on ordering: "The getAll(name) method steps are to return the values of all name-value pairs whose name is name... in list order"; The sort() method specifies that "The relative order between name-value pairs with equal names must be preserved." (Emphasis mine). Examining the Github issue referenced in the commit where the sort method was added, we see that the original proposal was to sort on values where keys were identical, but this was changed: "The reason for the default sort not affecting the value order is that ordering of the values can be significant. We should not assume that it's ok to move the order of the values around." (https://github.com/whatwg/url/issues/26#issuecomment-271600764)

Cyst answered 7/12, 2021 at 17:39 Comment(0)
B
132

There is no spec on this. You may do what you like.

Typical approaches include: first-given, last-given, array-of-all, string-join-with-comma-of-all.

Suppose the raw request is:

GET /blog/posts?tag=ruby&tag=rails HTTP/1.1
Host: example.com

Then there are various options for what request.query['tag'] should yield, depending on the language or the framework:

request.query['tag'] => 'ruby'
request.query['tag'] => 'rails'
request.query['tag'] => ['ruby', 'rails']
request.query['tag'] => 'ruby,rails'
Bodnar answered 17/11, 2009 at 4:21 Comment(6)
More to the point of the question, there is also the option of ['rails', 'ruby'] (different order).Charily
.NET will give you as an array (I have not cared about the order when I tested that), PHP will give you always the last and Java (at least the system I worked with based on Java) always the first value. #1809994Thickleaf
This is based on an attack called HTTP Parameter Pollution and has been analyzed by OWASP: owasp.org/images/b/ba/AppsecEU09_CarettoniDiPaola_v0.8.pdf At page 9 you'll find a list of 20 systems and a description how they handle this issue.Thickleaf
@Thickleaf in addition to that, PHP will actually create an array if you append square brackets with an optional index to the parameter name.Highsmith
@SimonSimCity, Java will return the first value or the array of values depending on whether you call getParameter or getParameterValues.Psittacosis
how can i get it as an array in js?Slipsheet
C
21

The situation seems to have changed since this question was asked and the accepted answer was written 12 years ago. I believe we now have an authoritative source: The WHATWG URL Standard describes the process of extracting and parsing a query string in detail in section 6.2 (https://url.spec.whatwg.org/#interface-urlsearchparams) and section 5.1 on x-www-form-urlencoded parsing (https://url.spec.whatwg.org/#urlencoded-parsing). The parsing output is "an initially empty list of name-value tuples where both name and value hold a string", where a list is defined as a finite ordered sequence, and the key-value pairs are added to this list in the order they appear in the URL. At first there is no mention of repeated keys, but some methods on the URLSearchParams class in section 6.2 (https://url.spec.whatwg.org/#interface-urlsearchparams) set clear expectations on ordering: "The getAll(name) method steps are to return the values of all name-value pairs whose name is name... in list order"; The sort() method specifies that "The relative order between name-value pairs with equal names must be preserved." (Emphasis mine). Examining the Github issue referenced in the commit where the sort method was added, we see that the original proposal was to sort on values where keys were identical, but this was changed: "The reason for the default sort not affecting the value order is that ordering of the values can be significant. We should not assume that it's ok to move the order of the values around." (https://github.com/whatwg/url/issues/26#issuecomment-271600764)

Cyst answered 7/12, 2021 at 17:39 Comment(0)
T
15

I can confirm that for PHP (at least in version 4.4.4 and newer) it works like this:

GET /blog/posts?tag=ruby&tag=rails HTTP/1.1
Host: example.com

results in:

request.query['tag'] => 'rails'

But

GET /blog/posts?tag[]=ruby&tag[]=rails HTTP/1.1
Host: example.com

results in:

request.query['tag'] => ['ruby', 'rails']

This behavior is the same for GET and POST data.

Thickleaf answered 23/1, 2012 at 12:20 Comment(3)
The [] suffix seems like really strange behaviour, but if you try to send an Array as an argument via jQuery's .ajax(), then it'll automatically add them for you in the same way. It looks like this is to the benefit of PHP users.Viridissa
@IanClark It's intuitive to PHP coders - in plain PHP, $foo[] = 1 appends to an array. Django (Python) also does the same thing.Gibbie
Can verify on Apache Tomcat it returns comma-concatenated strings.Sides
S
9

yfeldblum's answer is perfect.

Just a note about a fifth behavior I noticed recently: on Windows Phone, opening an application with an uri with a duplicate query key will result in NavigationFailed with:

System.ArgumentException: An item with the same key had already been added.

The culprit is System.Windows.Navigation.UriParsingHelper.InternalUriParseQueryStringToDictionary(Uri uri, Boolean decodeResults).

So the system won't even let you handle it the way you want, it will forbid it. You are left with the only solution to choose your own format (CSV, JSON, XML, ...) and uri-escape-it.

Shrunken answered 13/10, 2014 at 13:55 Comment(3)
That seems like an internal bug of that function, rather than a design choice. The probably function doesn't check for duplicate keys in the Dictionary it's creating. Dictionaries, of course, require unique keys.Fennell
So the client browser -- not the server -- is throwing an error in this situation? It does seem like a bug. I wonder if this bug still exists today?Ahn
@JonSchneider Yes, the client is throwing NavigationFailed for such URI. But, forgive me, I dropped Windows (Phone) development a month after this post and I moved to macOS (iOS), so I can't help anymore to track this issue nowadays.Polysyllable
M
6

Most (all?) of the frameworks offer no guarantees, so assume they will be returned in random order.

Always take the safest approach.

For example, java HttpServlet interface: ServletRequest.html#getParameterValues

Even the getParameterMap method leaves out any mention about parameter order (the order of a java.util.Map iterator cannot be relied on either.)

Medamedal answered 17/11, 2009 at 4:18 Comment(1)
"offer no guarantees" or perhaps severely under-documented sprinkled with a bit of cowboy coding?Hellenic
P
4

Typically, duplicate parameter values like

http://example.com/page?field=foo&field=bar

result in a single queryString parameter that is an array:

field[0]=='foo'
field[1]=='bar'

I've seen this behavior in ASP, ASP.NET and PHP4.

Procuration answered 17/11, 2009 at 4:9 Comment(2)
exactly, this is the de-facto standard, but as far as I see there's no authoritative decision on it. Since I don't believe this is the case, I am just inept to find it.Computerize
Yes, probably everyone has seen that behaviour. The question was if that is actually specified somewhere.Charily
B
1

The ?array[]=value1&array[]=value2 approach is certainly a very popular one.

  • supported by most Javascript frameworks
  • supported by Java Spring
  • supported by PHP
Bascio answered 9/3, 2021 at 8:44 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.