Spring MVC @PathVariable with dot (.) is getting truncated

Question

This is continuation of question Spring MVC @PathVariable getting truncated

Spring forum states that it has fixed(3.2 version) as part of ContentNegotiationManager. see the below link.
https://github.com/spring-projects/spring-framework/issues/10832 (formerly SPR-6164)
https://github.com/spring-projects/spring-framework/issues/12288 (formerly SPR-7632)

In my application requestParameter with .com is truncated.

Could anyone explain me how to use this new feature? how is it configurable at xml?

Note: spring forum- #1 Spring MVC @PathVariable with dot (.) is getting truncated

Altercation

Answer 1

As far as i know this issue appears only for the pathvariable at the end of the requestmapping.

We were able to solve that by defining the regex addon in the requestmapping.

 /somepath/{variable:.+}

Equipage

Thanks, I think this fix available earlier also (before 3.2V)?. However I don't like this fix; since it is needed at all the url which has to be handled in my application... and future URL implementation also to be taken care of this... – Altercation

here is how i solved the issue in spring 3.0.5 <!-- Spring Configuration needed to avoid URI using dots to be truncated --> <bean class="org.springframework.web.servlet.mvc.annotation.DefaultAnnotationHandlerMapping"> <property name="useDefaultSuffixPattern" value="false" /> </bean> – Gluey

@Martin Frey Could you explain your expression? I googled spring expression language and regular expression and I don't understand it.Mayby it is RequestMappingHandlerMapping or PathVariable unique expression? – Scharaga

@Mariusz, the syntax is {variable_name:regular_expression}, so here we have variable named variable, which value will be matched using regex .+ (where . means 'any character' and + means 'one or more times'). – Lifeline

@MichałRybak I understand the syntax of the regular expression. What puzzles me is, why does this solve the truncation problem? Which it does, I'm just curious why. – Aftercare

@StefanHaberl if you match variable in a regular way, Spring uses its suffix detection features and truncates everything after dot. When you use regexp matching, those features are not used - variable is only matched to regexp that you provide. – Lifeline

For me Spring 3.1.x doesn't seem to work. The RequestMapping is not matched. scratch that. I was excluding some extensions in web.xml – Sulphonate

@martin "variable:.+" doesn't work when there's more than one dot in the variable. eg putting emails at the end of restful paths like /path/[email protected]. The controller doesn't even get called, but it works when there's only one dot /path/[email protected]. Any idea why and/or a workaround? – Incurrence

@Incurrence : Make sure your @ is URL-encoded, otherwise you may run into issues with some/[email protected] being interpreted as user some/path at host foo.bar. – Summer

@Matt if you mean dot characters, you need to escape the dot: /somepath/{variable:(?:\\.\\w+)*}, but I think you need /somepath/{variable:(?:\\w+(\\.\\w+)*} otherwise the value must start with a dot. – Incurrence

Answer 2

Spring considers that anything behind the last dot is a file extension such as .jsonor .xml and trucate it to retrieve your parameter.

So if you have /somepath/{variable} :

• /somepath/param, /somepath/param.json, /somepath/param.xml or /somepath/param.anything will result in a param with value param
• /somepath/param.value.json, /somepath/param.value.xml or /somepath/param.value.anything will result in a param with value param.value

if you change your mapping to /somepath/{variable:.+} as suggested, any dot, including the last one will be consider as part of your parameter :

• /somepath/param will result in a param with value param
• /somepath/param.json will result in a param with value param.json
• /somepath/param.xml will result in a param with value param.xml
• /somepath/param.anything will result in a param with value param.anything
• /somepath/param.value.json will result in a param with value param.value.json
• ...

If you don't care of extension recognition, you can disable it by overriding mvc:annotation-driven automagic :

<bean id="handlerMapping"
      class="org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping">
    <property name="contentNegotiationManager" ref="contentNegotiationManager"/>
    <property name="useSuffixPatternMatch" value="false"/>
</bean>

So, again, if you have /somepath/{variable} :

• /somepath/param, /somepath/param.json, /somepath/param.xml or /somepath/param.anything will result in a param with value param
• /somepath/param.value.json, /somepath/param.value.xml or /somepath/param.value.anything will result in a param with value param.value

note : the difference from the default config is visible only if you have a mapping like somepath/something.{variable}. see Resthub project issue

if you want to keep extension management, since Spring 3.2 you can also set the useRegisteredSuffixPatternMatch property of RequestMappingHandlerMapping bean in order to keep suffixPattern recognition activated but limited to registered extension.

Here you define only json and xml extensions :

<bean id="handlerMapping"
      class="org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping">
    <property name="contentNegotiationManager" ref="contentNegotiationManager"/>
    <property name="useRegisteredSuffixPatternMatch" value="true"/>
</bean>

<bean id="contentNegotiationManager" class="org.springframework.web.accept.ContentNegotiationManagerFactoryBean">
    <property name="favorPathExtension" value="false"/>
    <property name="favorParameter" value="true"/>
    <property name="mediaTypes">
        <value>
            json=application/json
            xml=application/xml
        </value>
    </property>
</bean>

Note that mvc:annotation-driven accepts now a contentNegotiation option to provide a custom bean but the property of RequestMappingHandlerMapping has to be changed to true (default false) (cf. https://jira.springsource.org/browse/SPR-7632).

For that reason, you still have to override the all mvc:annotation-driven configuration. I opened a ticket to Spring to ask for a custom RequestMappingHandlerMapping : https://jira.springsource.org/browse/SPR-11253. Please vote if you are intereted in.

While overriding, be carreful to consider also custom Execution management overriding. Otherwise, all your custom Exception mappings will fail. You will have to reuse messageCoverters with a list bean :

<bean id="validator" class="org.springframework.validation.beanvalidation.LocalValidatorFactoryBean" />
<bean id="conversionService" class="org.springframework.format.support.FormattingConversionServiceFactoryBean" />

<util:list id="messageConverters">
    <bean class="your.custom.message.converter.IfAny"></bean>
    <bean class="org.springframework.http.converter.ByteArrayHttpMessageConverter"></bean>
    <bean class="org.springframework.http.converter.StringHttpMessageConverter"></bean>
    <bean class="org.springframework.http.converter.ResourceHttpMessageConverter"></bean>
    <bean class="org.springframework.http.converter.xml.SourceHttpMessageConverter"></bean>
    <bean class="org.springframework.http.converter.xml.XmlAwareFormHttpMessageConverter"></bean>
    <bean class="org.springframework.http.converter.xml.Jaxb2RootElementHttpMessageConverter"></bean>
    <bean class="org.springframework.http.converter.json.MappingJacksonHttpMessageConverter"></bean>
</util:list>

<bean name="exceptionHandlerExceptionResolver"
      class="org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver">
    <property name="order" value="0"/>
    <property name="messageConverters" ref="messageConverters"/>
</bean>

<bean name="handlerAdapter"
      class="org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter">
    <property name="webBindingInitializer">
        <bean class="org.springframework.web.bind.support.ConfigurableWebBindingInitializer">
            <property name="conversionService" ref="conversionService" />
            <property name="validator" ref="validator" />
        </bean>
    </property>
    <property name="messageConverters" ref="messageConverters"/>
</bean>

<bean id="handlerMapping"
      class="org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping">
</bean>

I implemented, in the open source project Resthub that I am part of, a set of tests on these subjects : see https://github.com/resthub/resthub-spring-stack/pull/219/files & https://github.com/resthub/resthub-spring-stack/issues/217

Buna

Forgive me I am a novice, so where do you put the bean configs? and what spring version does it apply to? – Salmonoid

@Salmonoid : You must define these beans into your "standard" Spring applicationContext.xml file(s). This applies to Spring 3.2 at least. Probably (at least partially) before – Buna

This ist the correct answer in my opinion. It seems that parameter "useRegisteredSuffixPatternMatch" was introduced exactly for the OPs problem. – Enwrap

This was only half of the solution for me. See @Paul Aerer's answer. – Regeneration

Answer 3

Update for Spring 4: since 4.0.1 you can use PathMatchConfigurer (via your WebMvcConfigurer), e.g.

@Configuration
protected static class AllResources extends WebMvcConfigurerAdapter {

    @Override
    public void configurePathMatch(PathMatchConfigurer matcher) {
        matcher.setUseRegisteredSuffixPatternMatch(true);
    }

}


@Configuration
public class WebConfig implements WebMvcConfigurer {

   @Override
   public void configurePathMatch(PathMatchConfigurer configurer) {
       configurer.setUseSuffixPatternMatch(false);
   }
}

In xml, it would be (https://github.com/spring-projects/spring-framework/issues/14796 [SPR-10163]):

<mvc:annotation-driven>
    [...]
    <mvc:path-matching registered-suffixes-only="true"/>
</mvc:annotation-driven>

Tupiguarani

this is by far the cleanest solution: turn off the feature causing it, rather than hacking around it. We're not using this feature anyway, so problem solved - perfect! – Calif

Where does the AllResources class go? – Piapiacenza

@ste_irl Add a java class in the same package as your main. – Linzer

Use matcher.setUseSuffixPatternMatch(false) to completely disable suffix match. – Amador

This was only half of the solution for me. See @Paul Aerer's answer. – Regeneration

@Dave i thought you're a hero when i saw your profile picture, but after seeing this comment i realized you're god. – Maleeny

Answer 4

In addition to Martin Frey's answer, this can also be fixed by adding a trailing slash in the RequestMapping value:

/path/{variable}/

Keep in mind that this fix does not support maintainability. It now requires all URI's to have a trailing slash - something that may not be apparent to API users / new developers. Because it's likely not all parameters may have a . in them, it may also create intermittent bugs

Cassatt

Thats even a cleaner solution. I had to find out the hard way that IE is setting accept headers according to the suffix. So i wanted to post on some .doc requestmapping and i always got a download instead of the new html page. This approach fixed that. – Equipage

this is the simplest solution for me to and solved my problem; regexp seems a bit of an overkill for many cases – Omnivorous

but it collides with AngularJS's default behavior to remove trailing slashes automagically. That can be configured in latest Angular releases but it is something to track for hours if you don't know what is going on. – Lachance

@Lachance And you just saved me hours of debugging, thanks! Nevertheless you should mention in the answer that the trailing slash will be required in the HTPP requests. – Fubsy

This is very dangerous! I certainly wouldn't recommend it as any one implementing the API would least expect it. Very non-maintainable. – Milch

@Spider You are right. The endpoint won't know whether a client adds a trailing slash or not. Dave Syers answer is better albeit not so obvious. – Linzer

Or you can add a "/path/{variable}/something" at the end (i.e. like a dummy value). Works fine there. – Vesiculate

Answer 5

In Spring Boot Rest Controller, I have resolved these by following Steps:

RestController :

@GetMapping("/statusByEmail/{email:.+}/")
public String statusByEmail(@PathVariable(value = "email") String email){
  //code
}

And From Rest Client:

Get http://mywebhook.com/statusByEmail/[email protected]/

Shawntashawwal

This answer is dependent on a trailing slash in order to work. – Regeneration

works like a charm (also without trailing slash). thank you! – Scotney

Answer 6

adding the ":.+" worked for me, but not until I removed outer curly brackets.

value = {"/username/{id:.+}"} didn't work

value = "/username/{id:.+}" works

Hope I helped someone :)

Lettered

That's because the curly brackets evaluate the RegEx and you already have some around id – Regeneration

Answer 7

/somepath/{variable:.+} works in Java requestMapping tag.

Aciniform

I prefer this answer because it doesn’t show what didn’t work. – Bickerstaff

Doesn't work for email addresses with more than one dot. – Regeneration

@Regeneration Sth like "/{code:.+}" works for many dots not one i.e. 61.12.7 it also works for i.e. [email protected] – Hiero

Same case with me. It works for multiple dots also. Spring version is 4.3.6.RELEASE – Squatness

Answer 8

Here's an approach that relies purely on java configuration:

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurationSupport;
import org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping;

@Configuration
public class MvcConfig extends WebMvcConfigurationSupport{

    @Bean
    public RequestMappingHandlerMapping requestMappingHandlerMapping() {
        RequestMappingHandlerMapping handlerMapping = super.requestMappingHandlerMapping();
        handlerMapping.setUseSuffixPatternMatch(false);
        handlerMapping.setUseTrailingSlashMatch(false);
        return handlerMapping;
    }
}

Bonnee

Thanks, solved it for me. Also, it's very clean and explicit. +1 – Rebato

Answer 9

One pretty easy way to work around this issue is to append a trailing slash ...

e.g.:

use :

/somepath/filename.jpg/

instead of:

/somepath/filename.jpg

Som

Answer 10

In Spring Boot, The Regular expression solve the problem like

@GetMapping("/path/{param1:.+}")

Sago

Note that this only works for one dot. It doesn't work for email addresses. – Regeneration

@Regeneration Sth like "/{code:.+}" works for many dots not one i.e. 61.12.7 it also works for i.e. [email protected] – Hiero

@Regeneration I have tested it with IP address. It workes very well. So, that means it works for multiple dots. – Sago

Answer 11

The complete solution including email addresses in path names for spring 4.2 is

<bean id="contentNegotiationManager"
    class="org.springframework.web.accept.ContentNegotiationManagerFactoryBean">
    <property name="favorPathExtension" value="false" />
    <property name="favorParameter" value="true" />
    <property name="mediaTypes">
        <value>
            json=application/json
            xml=application/xml
        </value>
    </property>
</bean>
<mvc:annotation-driven
    content-negotiation-manager="contentNegotiationManager">
    <mvc:path-matching suffix-pattern="false" registered-suffixes-only="true" />
</mvc:annotation-driven>

Add this to the application-xml

Wobbling

Upvote - this is the only answer here which makes clear that both the ContentNegotiationManagerFactoryBean and contentNegotiationManager configuration items are required – Regeneration

Answer 12

If you are using Spring 3.2.x and <mvc:annotation-driven />, create this little BeanPostProcessor:

package spring;

public final class DoNotTruncateMyUrls implements BeanPostProcessor {
    @Override
    public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
        if (bean instanceof RequestMappingHandlerMapping) {
            ((RequestMappingHandlerMapping)bean).setUseSuffixPatternMatch(false);
        }
        return bean;
    }
    @Override
    public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
        return bean;
    }
}

Then put this in your MVC config xml:

<bean class="spring.DoNotTruncateMyUrls" />

Holocrine

Is it related to ContentNegotiationManager ? – Altercation

My code only configures the RequestMappingHandlerMapping so that URLs will not be truncated. ContentNegotiationManager is another beast. – Holocrine

This is old, but you really don't need a BeanPostProcessor for this. If you use WebMvcConfigurationSupport you can override the requestMappingHandlerMapping @Bean method. If you use XML config, you can just declare your own RequestMappingHandlerMapping bean and declare that property. – Leddy

Thank you very much, I tried a different number of solutions for the same problem, only this one worked for me. :-) – Manner

Answer 13

For me the

@GetMapping(path = "/a/{variableName:.+}")

does work but only if you also encode the "dot" in your request url as "%2E" then it works. But requires URL's to all be that...which is not a "standard" encoding, though valid. Feels like something of a bug :|

The other work around, similar to the "trailing slash" way is to move the variable that will have the dot "inline" ex:

@GetMapping(path = "/{variableName}/a")

now all dots will be preserved, no modifications needed.

Vesiculate

Answer 14

Finally I found solution in Spring Docs:

To completely disable the use of file extensions, you must set both of the following:

 useSuffixPatternMatching(false), see PathMatchConfigurer

 favorPathExtension(false), see ContentNegotiationConfigurer

Adding this to my WebMvcConfigurerAdapter implementation solved the problem:

@Override
public void configureContentNegotiation(ContentNegotiationConfigurer configurer) {
    configurer.favorPathExtension(false);
}

@Override
public void configurePathMatch(PathMatchConfigurer matcher) {
    matcher.setUseSuffixPatternMatch(false);
}

Selfconfessed

Answer 15

If you write both back and frontend, another simple solution is to attach a "/" at the end of the URL at front. If so, you don't need to change your backend...

somepath/[email protected]/

Be happy!

Jovia

Answer 16

As of Spring 5.2.4 (Spring Boot v2.2.6.RELEASE) PathMatchConfigurer.setUseSuffixPatternMatch and ContentNegotiationConfigurer.favorPathExtension have been deprecated ( https://spring.io/blog/2020/03/24/spring-framework-5-2-5-available-now and https://github.com/spring-projects/spring-framework/issues/24179).

The real problem is that the client requests a specific media type (like .com) and Spring added all those media types by default. In most cases your REST controller will only produce JSON so it will not support the requested output format (.com). To overcome this issue you should be all good by updating your rest controller (or specific method) to support the 'ouput' format (@RequestMapping(produces = MediaType.ALL_VALUE)) and of course allow characters like a dot ({username:.+}).

Example:

@RequestMapping(value = USERNAME, consumes = MediaType.APPLICATION_JSON_VALUE, produces = MediaType.APPLICATION_JSON_VALUE)
public class UsernameAPI {

    private final UsernameService service;

    @GetMapping(value = "/{username:.+}", consumes = MediaType.APPLICATION_JSON_VALUE, produces = MediaType.ALL_VALUE)
    public ResponseEntity isUsernameAlreadyInUse(@PathVariable(value = "username") @Valid @Size(max = 255) String username) {
        log.debug("Check if username already exists");
        if (service.doesUsernameExist(username)) {
            return ResponseEntity.status(HttpStatus.NO_CONTENT).build();
        }
        return ResponseEntity.notFound().build();
    }
}

Spring 5.3 and above will only match registered suffixes (media types).

Patch

Answer 17

If you are using Spring 3.2+ then below solution will help. This will handle all urls so definitely better than applying regex pattern in the request URI mapping to allow . like /somepath/{variable:.+}

Define a bean in the xml file

<bean class="org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping">
        <property name="useSuffixPatternMatch" value="false"/>
        <property name="useRegisteredSuffixPatternMatch" value="true"/>
    </bean>

The flags usage can be found on the documentation. I am putting snipped to explain

exlanation of useRegisteredSuffixPatternMatch is said to be resolving the issue. From the java doc in the class

If enabled, a controller method mapped to "/users" also matches to "/users.json" assuming ".json" is a file extension registered with the provided {@link #setContentNegotiationManager(ContentNegotiationManager) contentNegotiationManager}. This can be useful for allowing only specific URL extensions to be used as well as in cases where a "." in the URL path can lead to ambiguous interpretation of path variable content, (e.g. given "/users/{user}" and incoming URLs such as "/users/john.j.joe" and "/users/john.j.joe.json").

Poseidon

Answer 18

Simple Solution Fix: adding a regex {q:.+} in the @RequestMapping

@RequestMapping("medici/james/Site")
public class WebSiteController {

    @RequestMapping(value = "/{site:.+}", method = RequestMethod.GET)
    public ModelAndView display(@PathVariable("site") String site) {
        return getModelAndView(site, "web site");

    }
}

Now, for input /site/jamesmedice.com, “site” will display the correct james'site

Swaggering