Delphi Rest.JSON JsonToObject only works with f variables

Question

I'm using Delphi XE8.

I was just looking at the REST.Json ObjectToJsonString() and JsonToObject() calls.

Mainly attempting to do something like this:

How to convert an object to JSON and back with a single line of code

I noticed that I could only get variables to work when they started with an F character. I couldn't find any documentation on this. Is this expected behavior? Should I be naming all variables inside of my classes with an F at the start? If Yes, can someone explain why?

I created a class TTestJSON and defined two member variables and set them to 'WORKS' and 'FAILS'.

Then I created a JSON string value from the object:

{
  "varThatWorksBeacuseItStartsWithF":"WORKS",
  "sVarThatFailsBecauseItStartsWithS":"FAILS"
} 

When going back from a JSON string to object, only the fVarThatWorksBeacuseItStartsWithF variable is being reset correctly. In the code below, test := TJson.JsonToObject<TTestJSON>(JsonStr); using the above JSON, notice that the sVarThatFailsBecauseItStartsWithS is "" and not "FAILS".

procedure TForm3.btn1Click(Sender: TObject);
var
  test : TTestJSON;
  JsonStr : String;
begin
  m1.Clear;
  test := TTestJSON.Create;
  try
    test.fVarThatWorksBeacuseItStartsWithF := 'WORKS';
    test.sVarThatFailsBecauseItStartsWithS := 'FAILS';
    JsonStr := TJson.ObjectToJsonString( test );
  finally
    test.Free;
  end;
  m1.Lines.Add(  '** JSONStr Value START **' + #13#10 + JsonStr + '** JSONStr Value END **' + #13#10 );

  test := TJson.JsonToObject<TTestJSON>(JsonStr);
  try
    m1.Lines.Add('** Obj loaded from JSON String Start **' + #13#10 + TJson.ObjectToJsonString( test ) + #13#10 + '** Obj loaded from JSON String End **');
  finally
    test.Free;
  end;
end;

From the results, the var that starts with f has the f stripped out of the JSON string, and the one that starts with s still has it in there. I would have expected that the second result would have looked like this:

{
  "varThatWorksBeacuseItStartsWithF":"WORKS",
  "sVarThatFailsBecauseItStartsWithS":"FAILS"
}

Here is the full code to reproduce - just has a button and a memo on a vcl form - also uses REST.Json:

unit Main;

interface

uses
  Winapi.Windows, Winapi.Messages, System.SysUtils, System.Variants, System.Classes, Vcl.Graphics,
  Vcl.Controls, Vcl.Forms, Vcl.Dialogs, Vcl.StdCtrls, rest.Json;

type
  TTestJSON = class
    fVarThatWorksBeacuseItStartsWithF : String;
    sVarThatFailsBecauseItStartsWithS : String;
  end;

  TForm3 = class(TForm)
    btn1: TButton;
    m1: TMemo;
    procedure btn1Click(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form3: TForm3;

implementation

{$R *.dfm}

procedure TForm3.btn1Click(Sender: TObject);
var
  test : TTestJSON;
  JsonStr : String;
begin
  m1.Clear;
  test := TTestJSON.Create;
  try
    test.fVarThatWorksBeacuseItStartsWithF := 'WORKS';
    test.sVarThatFailsBecauseItStartsWithS := 'FAILS';
    JsonStr := TJson.ObjectToJsonString( test );
  finally
    test.Free;
  end;
  m1.Lines.Add(  '** JSONStr Value START **' + #13#10 + JsonStr + '** JSONStr Value END **' + #13#10 );

  test := TJson.JsonToObject<TTestJSON>(JsonStr);
  try
    m1.Lines.Add('** Obj loaded from JSON String Start **' + #13#10 + TJson.ObjectToJsonString( test ) + #13#10 + '** Obj loaded from JSON String End **');
  finally
    test.Free;
  end;
end;

end.

Cheyney

Variable names are irrelevant, except when they cause naming conflicts across scopes. What exactly "does not work" in your example? Please describe the actual problem. – Bore

i have updated the question the problem is the variables that dont start with f, don't seem to be reset when going from TJson.ObjectToJsonString then back to an object again using TJson.JsonToObject<TTestJSON>(JsonStr) – Cheyney

@Remy Not here. In their infinite wisdom Emba decided to serialize variables whose first letter is F. So Fred comes, but George stays. – Zweig

@Danga Yes, the docs don't really exist here. Use JSONMarshalled and JSONName attributes. – Zweig

Do yourself a favor and don't use Rest.Json It is full of bugs and serious regressions can pop up between Delphi versions making your code unusable. List of JSON issues in QP – Hayrack

what would you recommend using for JSON handling instead? – Cheyney

@Dalija If we stop using classes with bugs, we will have to stop using Delphi. – Ironstone

SuperObject is a common and sound choice – Zweig

@DavidHeffernan Thanks ill have a look at it – Cheyney

@AndreiGalatyn In this case you are not just dealing with buggy class, but with class (framework) that is badly designed from the start. I doubt it will ever work properly, unless it gets complete overhaul. Rest.Json serializes fields instead of properties and that is in contradiction with how Delphi classes are commonly designed. – Hayrack

@Dalija I tend to agree with you, mostly. But we have what we have and it is still usable i think, if we know how it works, even if implementation is not perfect. – Ironstone

@AndreiGalatyn If you can tolerate handcrafting classes around Rest.Json implementation and if you tolerate possible breaking changes when you move from one Delphi version to another then, yes, Rest.Json is usable. – Hayrack

@DalijaPrasnikar not to mention it pulls the entire rest framework (including dependencies which help you write a server) into the project. The server and the json parser is not isolated at all! – Rosyrot

Answer 1

JSON serialization in Delphi is based on fields, not properties. But most of Delphi classes have friendly properties and F-prefixed fields. At same time seems Emb is trying to avoid of F-prefixed names in generated JSON. They cut off first "F" from name when serialize fields and add it back (to find correct field) when read from JSON. Seems the only (safe) way to use JSON serialization in Delphi is to keep all field names with prefix "F" (for the fields that you want to serialize):

TTestJSON = class
protected
  FName: String;
public
  property Name: String read FName write FName;
end;

UPDATE2: As David mentions, we can use attributes and then we have much better control:

uses
  REST.Json.Types, // without this unit we get warning: W1025 Unsupported language feature: 'custom attribute'
  REST.Json;

type
  // All fields of records are serialized, no control here.
  TRec = record
    RecStr: String;
  end;

  // By default all fields of class are serialized, but only F-prefixed serialized correctly.
  // We can use JSONMarshalled attribute to enable/disable serialization.
  // We can use JSonName attribute to serialize field with specific name in JSON.
  TTestJSON = class
    [JSONMarshalled(True)] [JSonName('RecField')]
    R: TRec;
  end;

procedure TForm28.FormCreate(Sender: TObject);
var
  Test: TTestJSON;
  JsonStr: string;
begin
  Test := TTestJSON.Create;
  try
    Test.R.RecStr := 'Some str';
    JsonStr := TJson.ObjectToJsonString( Test );
  finally
    FreeAndNil(Test);
  end;

  // JsonStr: {"RecField":["Some str"]}

  Test := TJson.JsonToObject<TTestJSON>(JsonStr);
  FreeAndNil(Test);
end;

Ironstone

Do you know where i can find some info about serialization. looking through the docwiki.embarcadero.com has lots of references to it, but no real explanation on what it does – Cheyney

No. You can use the attributes identified in my comment to customise serialization. – Zweig

@Dangas The docs are close to useless here. Read the source code or pick a better library. – Zweig

:) thanks for the help, just checking i wasn't missing something obvious – Cheyney

@David Just tried, seems absence of F-prefix doesn't mean that field will not be serialized, it means that it will fail to deserialize correctly. – Ironstone

Did you read my comment to the question that mentions the attributes? – Zweig

@David Sorry, i missed it. It works great with attributes! I will update the answer. – Ironstone

Answer 2

This library serializes fields. Since common practise is to prefix fields with the letter F, the designers want to remove that letter from the names used in the JSON. So they decided to make the default behaviour to be to strip off the first letter in fields whose name begins with F. Frankly that seems pretty weird to me.

This behaviour can be customized using attributes. For example:

[JSONMarshalled(False)]
FFoo: Integer; 

[JSONMarshalled(True)]
[JSONName('Blah')]
Bar: Integer;

So far as I can see, none of this is properly documented.

Zweig

I mentioned it in my answer, they do serialization of fields without "F" too, you will see all such fields in generated JSON. But they fail to deserialize them, because they are looking for field name with prefix "F". – Ironstone

"deserialize just fields that begin with the letter F" - still not correct. For example if you have fields "FA" and "A", they both will be serialized with same name "A" in JSON. And under some conditions after deserialization you get value of A loaded into FA. – Ironstone

@AndreiGalatyn All fields are serialized and deserialized, but if the name begins with F, that F is stripped from the JSON name? Right? – Zweig

Yes, correct. Such implementation has side effects, so the only safe way to use it is either name all fields starting from "F" or use attributes (as you suggested). If someone need to serialize only part of fields, then attributes is the only safe option i think. – Ironstone

And it also camelCases the field names as well, which is a pain when the endpoint requires non camelCase. – Dystopia