Unable to translate bytes [...] at index ... from specified code page to Unicode when adding to index

Question

I am using Newtonsoft.Json to create the JSON to update add items to an index, but I get the following error when I POST the request:

{"error":{"code":"","message":"The request is invalid.","innererror":{"message":"parameters : Unable to translate bytes [E3] at index 752 from specified code page to Unicode.\r\n","type":"","stacktrace":""}}}

I know the error occurs with some non letter characters in some of the strings in the data that I am serializing. The string data comes from SQL, so I'm guessing something is going on to do with encoding that I cannot figure out.

When I inspect the JSON string, and put it in manually construct a request with the same data in Fiddler it all works fine.

Does anyone have any idea what might be the problem, and how I can work around it?

Tola

I have done some debugging, and it seems that one of the offending strings in a the JSON is: "longName": "São Tome and Principe Dobras", So one of the characters is ASCII 198 (a with tilde), that cannot be encoded to Unicode? – Tola

Answer 1

I found my own solution after a bit more digging.

Adding "StringEscapeHandling.EscapeNonAscii" to the serialization options solves the problem:

jsonSettings = new JsonSerializerSettings
{
    Formatting = Newtonsoft.Json.Formatting.Indented,
    ContractResolver = new CamelCasePropertyNamesContractResolver(),
    DateTimeZoneHandling = DateTimeZoneHandling.Utc,
    StringEscapeHandling = StringEscapeHandling.EscapeNonAscii
};

Tola