AWS API Gateway Integration Response - Lambda Error Regex fails matching when response contains new line

Question

I have API Gateway as a trigger to my lambda function. Whenever my lambda raises an exception, it gets matched with the pattern configured in Integration Response. The problem is if the exception contains new line, then the pattern is ignored and it returns 200 as status code by default.

To reproduce the issue:

1. Create an lambda with python 3.6 and add API Gateway as trigger.
2. In the API Gateway, configure integration response regex as "Internal Server Error.*" to custom code 500. (means whenever lambda returns a message with prefix as "Internal Server Error" it would be considered as Exception and status code will be set to 500).
3. In lambda, raise an exception message containing "\n" new line.
4. When you trigger the lambda from API Gateway, it returns 200 instead of 500 as status code.

Note: If you remove newline from the exception message it works fine and returns 500 as expected.

Did anybody face the same issue? Any Solutions for it?

Dissent

Same here. In addition the returned json is invalid in my case. Come on AWS, fix this! – Glassy

Answer 1

Until AWS comes to the party and fixes this one my only workaround is to catch thrown exceptions in the Lambda handler and throw a wrapping exception with the same message but with '\n' characters replaced by "\n". This is an imperfect solution because currently the non-proxy integration error handling does something odd with that character sequence and I end up with two backslashes and an n in the json instead of one backslash and n (which is how you specify a newline character in json).

One interesting thing about the this integration problem I notice is that the cause part of the exception (which is mapped to a json block for the causing exception) does have correct handling of newline characters (!). This means that wrapping the exception with an exception with a simple message or no message will pass the original exception message correctly formatted into the cause field of the json.

Here's an example of json returned from a non-proxy api lambda integration that uses the approach I discussed. You can see the extra backslashes in the errorMessage field and you can see correctly formatted newline characters in the cause errorMessage field.

{
   "errorMessage":"ServerException: Error occurred at position, start= 40, finish=81 with desc='aircraftRegistrationMarking', value='VH OSA?':\\nWARNING - SUSPECT NON-SPEC IN AIRCRAFT REG. MARKING\\n?1 = 100100 = Space - Right Justified\\n",
   "errorType":"com.github.davidmoten.aws.helper.ServerException",
   "stackTrace":[
      "au.gov.amsa.beacon.decoder.Handler.handleRequest(Handler.java:76)",
      "sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
      "sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
      "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
      "java.lang.reflect.Method.invoke(Method.java:498)"
   ],
   "cause":{
      "errorMessage":"Error occurred at position, start= 40, finish=81 with desc='aircraftRegistrationMarking', value='VH OSA?':\nWARNING - SUSPECT NON-SPEC IN AIRCRAFT REG. MARKING\n?1 = 100100 = Space - Right Justified\n",
      "errorType":"java.lang.RuntimeException",
      "stackTrace":[
         "au.gov.amsa.fgb.internal.DecodeAsJson.getData(DecodeAsJson.java:28)",
         "au.gov.amsa.fgb.internal.HexDecoder.decodeFullAsJson(HexDecoder.java:51)",
         "au.gov.amsa.fgb.internal.Decoder.decodeFullAsJson(Decoder.java:11)",
         "au.gov.amsa.fgb.Beacon15HexId.decodeHexToJson(Beacon15HexId.java:16)",
         "au.gov.amsa.beacon.decoder.Handler.handleRequest(Handler.java:39)",
         "sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
         "sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
         "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
         "java.lang.reflect.Method.invoke(Method.java:498)"
      ]
   }
}

For your interest (and for the attention of AWS!) if I don't replace the new lines in the primary exception message this is what I get (invalid JSON, missing lots of double quote characters):

{errorMessage=ServerException: Error occurred at position, start= 40, finish=81 with desc='aircraftRegistrationMarking', value='VH OSA?':
WARNING - SUSPECT NON-SPEC IN AIRCRAFT REG. MARKING
?1 = 100100 = Space - Right Justified
, errorType=com.github.davidmoten.aws.helper.ServerException, stackTrace=["au.gov.amsa.beacon.decoder.Handler.handleRequest(Handler.java:75)","sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)","sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)","sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)","java.lang.reflect.Method.invoke(Method.java:498)"], cause={errorMessage=Error occurred at position, start= 40, finish=81 with desc='aircraftRegistrationMarking', value='VH OSA?':
WARNING - SUSPECT NON-SPEC IN AIRCRAFT REG. MARKING
?1 = 100100 = Space - Right Justified
, errorType=java.lang.RuntimeException, stackTrace=["au.gov.amsa.fgb.internal.DecodeAsJson.getData(DecodeAsJson.java:28)","au.gov.amsa.fgb.internal.HexDecoder.decodeFullAsJson(HexDecoder.java:51)","au.gov.amsa.fgb.internal.Decoder.decodeFullAsJson(Decoder.java:11)","au.gov.amsa.fgb.Beacon15HexId.decodeHexToJson(Beacon15HexId.java:16)","au.gov.amsa.beacon.decoder.Handler.handleRequest(Handler.java:38)","sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)","sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)","sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)","java.lang.reflect.Method.invoke(Method.java:498)"]}}

Glassy