Tomcat 8 is not able to handle get request with '|' in query parameters?

Question

I am using Tomcat 8. In one case I need to handle external request coming from external source where the request has a parameters where it is separated by |.

Request is looks like this:

http://localhost:8080/app/handleResponse?msg=name|id|

In this case I am getting following error.

java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986
    at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:467)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:667)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:789)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:745)

EDIT 1

It works with Apache Tomcat 8.0.30 but not with Tomcat 8.5

Wilkens

OK, I guess you'll have to escape it – Profess

Have you took a look into the referenced RFC? – Rickettsia

RFCs are meant to be improved. – Maidenhair

Answer 1

This behavior is introduced in all major Tomcat releases:

• Tomcat 7.0.73, 8.0.39, 8.5.7

To fix, do one of the following:

• set relaxedQueryChars to allow this character (recommended, see Lincoln's answer)
• set requestTargetAllow option (deprecated in Tomcat 8.5) (see Jérémie's answer).
• you can downgrade to one of older versions (not recommended - security)

---

Based on changelog, those changes could affect this behavior:

Tomcat 8.5.3:

Ensure that requests with HTTP method names that are not tokens (as required by RFC 7231) are rejected with a 400 response

Tomcat 8.5.7:

Add additional checks for valid characters to the HTTP request line parsing so invalid request lines are rejected sooner.

---

The best option (following the standard) - you want to encode your URL on client:

encodeURI("http://localhost:8080/app/handleResponse?msg=name|id|")
> http://localhost:8080/app/handleResponse?msg=name%7Cid%7C

or just query string:

encodeURIComponent("msg=name|id|")
> msg%3Dname%7Cid%7C

It will secure you from other problematic characters (list of invalid URI characters).

Fanchon

Thanks, I just downgraded my apache-tomcat-8.5.11 to apache-tomcat-8.0.30 and it worked like a charm. – Windward

@BhuwanGautam I would not recommend downgrading tomcat as latest tomcat versions have all the necessary CVE patches which might have affected your downgraded versions already. Encoding URI parameters would be a fix - or using relaxedQueryChars - In long term, anywhere in code that uses reserved RFC 3986 chars should be refactored. – Ramburt

It seems relaxedQueryChars is still hackable only. Is there any proper solution provided by Apache? – Apart

For me "encodeURIComponent" worked like a charm! Tnx for describing alternative fixes. – Ruckman

is there any properties file change in spring boot case? – Sentry

Answer 2

Since Tomcat 7.0.76, 8.0.42, 8.5.12 you can define property requestTargetAllow to allow forbiden characters.

Add this line in your catalina.properties

tomcat.util.http.parser.HttpParser.requestTargetAllow=|{}

Lenoralenore

From tomcat 8.5 onwards # WARNING: Using this option may expose the server to CVE-2016-6816 #tomcat.util.http.parser.HttpParser.requestTargetAllow=| – Waki

Answer 3

The parameter tomcat.util.http.parser.HttpParser.requestTargetAllow is deprecated since Tomcat 8.5: tomcat official doc.

You can use relaxedQueryChars / relaxedPathChars in the connectors definition to allow these chars: tomcat official doc.

Coral

In 8.5 I tried these and it didn't have any effect. The best option is to always encode your urls. – Nucleolus

Answer 4

Issue: Tomcat (7.0.88) is throwing below exception which leads to 400 – Bad Request.

java.lang.IllegalArgumentException: Invalid character found in the request target. 
The valid characters are defined in RFC 7230 and RFC 3986.

This issue is occurring most of the tomcat versions from 7.0.88 onwards.

Solution: (Suggested by Apache team):

Tomcat increased their security and no longer allows raw square brackets in the query string. In the request we have [,] (Square brackets) so the request is not processed by the server.

Add relaxedQueryChars attribute under tag under server.xml (%TOMCAT_HOME%/conf):

<Connector port="80" 
           protocol="HTTP/1.1"
           maxThreads="150"
           connectionTimeout="20000"
           redirectPort="443"
           compression="on"
           compressionMinSize="2048"
           noCompressionUserAgents="gozilla, traviata"
           compressableMimeType="text/html,text/xml"
                                     relaxedQueryChars="[,]"
             />

If application needs more special characters that are not supported by tomcat by default, then add those special characters in relaxedQueryChars attribute, comma-separated as above.

Cycloplegia

Works for me latest tomcat 8 for DHIS2 ,because this helps some one !!! – Aronow

Answer 5

The URI is encoded as UTF-8, but Tomcat is decoding them as ISO-8859-1. You need to edit the connector settings in the server.xml and add the URIEncoding="UTF-8" attribute.

or edit this parameter on your application.properties

server.tomcat.uri-encoding=utf-8

Orthodontia

URIEncoding attribute on the <Connector> element in server.xml to something specific (e.g. URIEncoding="UTF-8") (wiki.apache.org/tomcat/FAQ/CharacterEncoding) – Leslee

@Leslee your link is down – Flatling

found similar link cwiki.apache.org/confluence/display/TOMCAT/Character+Encoding – Leslee

Answer 6

Escape it. The pipe symbol is one that has been handled differently over time and between browsers. For instance, Chrome and Firefox convert a URL with pipe differently when copy/paste them. However, the most compatible, and necessary with Tomcat 8.5 it seems, is to escape it:

http://localhost:8080/app/handleResponse?msg=name%7Cid%7C

Orrery

Answer 7

Adding "relaxedQueryChars" attribute to the server.xml worked for me :

<Connector connectionTimeout="20000" port="8080" protocol="HTTP/1.1" redirectPort="443" URIEncoding="UTF-8" relaxedQueryChars="[]|{}^&#x5c;&#x60;&quot;&lt;&gt;"/>

Chrystalchryste

how to allow # in the url ? relaxedQueryChars="^{}|[]_#" and im using tomcat 8.5.27 – Dorthea

Try : &#35; So Maybe something like this : relaxedQueryChars="[]|{}^&#x5c;&#x60;&quot;&lt;&gt;&#35;" – Chrystalchryste