How to disable Smart punctuation in flutter textfield ? Problem when a String contains the apostrophe sign : " s'habiller", "Brin d'herbe"

Question

Am encountering a problem when a string contains an apostrophe (single quote). Am developing a vocabulary learning app (French / English). The user must answer quizzes. The phone tells the user if what he/she types is the correct translation. Whenever there is an apostrophe, the string is not recognized.

Ex : "A blade of grass" : "un brin d'herbe" / "A beehive" : "Un nid d'abeilles".

To check the answer, I split the chain into a list : [un,nid,d'abeilles] Even if the user types "d'abeilles", it's never recognized as correct.

I noticed the same problem with the speech to text functionality : if the user says the word "s'habiller" (to get dressed), it is never found by my search function.

Anyone sees why this happens ? Would it be a "flutter" bug ?

In the app, I have a wordbank which was created with Microsoft Excel and then imported into visual studio. The wordbank is made up of WORD objects that have a series of parameters. Here is the one for blade of grass.

Word(
    id: 37,
    level: 6,
    main: "blade of grass",
    mainFr: "brin d'herbe",
    theme: [
      T(
          e: "house",
          f: "maison",
          ste: ["in the garden"],
          stf: ["dans le jardin"]),
      T(
          e: "nature",
          f: "nature",
          ste: ["in the meadow"],
          stf: ["dans la prairie"])
    ],
    articleEng: "a",
    articleFr: "un",
    nature: "noun",
    plur: "blades of grass ",
    ortho2: "",
    phon: "ˈbleɪd əv ˈgrɑːs",
    son: "",
    remPhon: "",
    french: [],
    syn: [],
    ant: [],
    wDef: "a single thin flat piece of grass.",
    wPhrase: "There was no wind at all. Not a leaf or blade of grass moved.",
    wPhraseFr:
        "Il n'y avait pas de vent du tour. Pas une feuille ni un brin d'herbe ne bougeait.",
    past: [],
    idiomsEn: [],
    idiomsFr: [],
    marqueur: "",
    remarque: "",
  ),

Patronizing

Please add some code, your bug has a lot more chance to come from your search method than being a "flutter bug", I've also encountered issues when working with strings containing ' but it was because of my own code. – Bologna

Are you perhaps comparing straight apostrophes (U+0027 ') against "smart" right single quotation marks (U+2019 ’) (or perhaps to other, similar Unicode characters)? I suggest printing the runes for the strings to see a list of their Unicode code points to see where they aren't identical. – Predecease

I edited my question and added the way my wordbank is constructed. The " ' " was typed in Microsoft Excel. How can I know if it's the correct one... ? When user types " ' " on their phone keyboard, do you know which one it is ? – Patronizing

Indeed you were right : in my wordbank, I think the apostrophes are the straight ones and when people type on their phone, it is the U+2019.... So I guess I will have to transform all of them in U+2019.... I hope all phones work the same way as far as " ' " is concerned when user types it... – Patronizing

Apparently it has to do with Apple's implementation of SMART PUNCTUATION. (developer.apple.com/forums/thread/89706) Would you know if there is a way in flutter to disable SMART PUNCTUATION in a textfield widget ? – Patronizing

I suggest processing your strings to perform some form of normalization on them (e.g. converting smart quotes to straight quotes, collapsing and/or stripping whitespace, changing capitalization) before performing string comparisons. – Predecease

Yes I had thought of that, but found a better solution. Cf. Answer below – Patronizing

Answer 1

I finally found the solution for this problem. It was indeed created with Apple's implementation of SMART PUNCTUATION. It is easy to disable it when using a textfield :

TextFormField(
     smartQuotesType: SmartQuotesType.disabled,

Patronizing

I had the same problem typing an apostrophe into a TextField. The TextField would show a right single quotation, but the saved controller text would be garbled. I didn't want to mess around with encodings and I prefer straight apostrophes so disabling smart quotes worked perfectly for my app. Thanks! – Anatto