How to compare TFunc/TProc containing function/procedure of object?
Asked Answered
S

1

12

We use a TList<TFunc<Boolean>> with some function ... of objects in it and now want to Remove() some of the entries again. But it doesn't work because obviously you simply can not compare these reference to ... thingies reliably.

Here's some test code:

program Project1;

{$APPTYPE CONSOLE}

uses
  Generics.Defaults,
  SysUtils;

type
  TFoo = class
  strict private
    FValue: Boolean;
  public
    constructor Create();
    function Bar(): Boolean;
  end;

{ TFoo }

function TFoo.Bar: Boolean;
begin
  Result := FValue;
end;

constructor TFoo.Create;
begin
  inherited;

  FValue := Boolean(Random(1));
end;

function IsEqual(i1, i2: TFunc<Boolean>): Boolean;
begin
  Result := TEqualityComparer<TFunc<Boolean>>.Default().Equals(i1, i2);
end;

var
  s: string;
  foo: TFoo;
  Fkt1, Fkt2: TFunc<Boolean>;

begin
  try
    Foo := TFoo.Create();

    WriteLn(IsEqual(Foo.Bar, Foo.Bar));             // FALSE (1)
    WriteLn(IsEqual(Foo.Bar, TFoo.Create().Bar));   // FALSE (2)

    Fkt1 := function(): Boolean begin Result := False; end;
    Fkt2 := Fkt1;
    WriteLn(IsEqual(Fkt1, Fkt2));                   // TRUE  (3)

    Fkt2 := function(): Boolean begin Result := False; end;
    WriteLn(IsEqual(Fkt1, Fkt2));                   // FALSE (4)

    Fkt2 := function(): Boolean begin Result := True; end;
    WriteLn(IsEqual(Fkt1, Fkt2));                   // FALSE (5)

    FreeAndNil(Foo);
  except
    on E:Exception do
      Writeln(E.Classname, ': ', E.Message);
  end;
  Readln(s);
end.

We tried virtually everything, = operator, comparing pointers, etc..

We even tried some really nasty things like repeatedly casting to PPointer and dereferencing until we get equal values, but that of course didn't yield satisfying results either =).

  • Case (2), (4) and (5) are OK, as there are in fact distinct functions.
  • Case (3) is trivial and OK, too.
  • Case (1) is what we want to detect, and this is what we can't get to work.

I fear, Delphi stealthily creates two distinct anonymous functions that forward the call to Foo.Bar. In this case we'd be completely powerless, unless we wanted to wade through a morass of unknown memory... and well, we don't.

Shetler answered 1/3, 2011 at 10:53 Comment(7)
+1 because those anonymous references are weired. What's in them? I just did var F: TFunc<Boolean>; ShowMessage(IntToStr(SizeOf(F))); - it shows 1 for my Delphi 2010! how can that be?Terrieterrier
What are those cases you’re referring to?Badly
@Cosmin - it's returning the size of the type of the expression F, which in your case is Boolean, because F is a function returning Boolean.Clubby
@Martijn: I'm referring to the numbers in the comments. @Cosmin: It looks like they are 4 bytes long, at least the addresses of the parameters inside IsEqual are 4 bytes apart (could be alignment, though).Shetler
@kiw: ah, ok. Hadn’t seen those, sorry.Badly
@Cosmin The ambiguity between F as function call or function / method pointer / reference is built into Pascal; you can blame Wirth or Hejlsberg, the first for making no-arg invocation implicit (no () operator needed like C or C++), the second for not requiring disambiguation when introducing general procedure pointers into the language (Wirth Pascal only permitted downward funargs owing to the problem of passing nested procedures, which ironically is the same problem that method references fix, by using reference counting to keep data from the outer frame alive).Clubby
I dissagree. You forgot about @ operator.Tarrant
C
15

You'll have to associated a name or index with them by some other means. Anonymous methods don't have names and may capture state (so they are recreated per instance); there is no trivial way to make them comparable without breaking encapsulation.

You can get at the object behind the method reference, if there is indeed an object behind it (there's no guarantee of this - the interfaces that method references are implemented in terms of COM semantics, all they really need is a COM vtable):

function Intf2Obj(x: IInterface): TObject;
type
  TStub = array[0..3] of Byte;
const
  // ADD [ESP+$04], imm8; [ESP+$04] in stdcall is Self argument, after return address
  add_esp_04_imm8: TStub = ($83, $44, $24, $04);
  // ADD [ESP+$04], imm32
  add_esp_04_imm32: TStub = ($81, $44, $24, $04);

  function Match(L, R: PByte): Boolean;
  var
    i: Integer;
  begin
    for i := 0 to SizeOf(TStub) - 1 do
      if L[i] <> R[i] then
        Exit(False);
    Result := True;
  end;

var
  p: PByte;
begin
  p := PPointer(x)^; // get to vtable
  p := PPointer(p)^; // load QueryInterface stub address from vtable

  if Match(p, @add_esp_04_imm8) then 
  begin
    Inc(p, SizeOf(TStub));
    Result := TObject(PByte(Pointer(x)) + PShortint(p)^);
  end
  else if Match(p, @add_esp_04_imm32) then
  begin
    Inc(p, SizeOf(TStub));
    Result := TObject(PByte(Pointer(x)) + PLongint(p)^);
  end
  else
    raise Exception.Create('Not a Delphi interface implementation?');
end;

type
  TAction = reference to procedure;

procedure Go;
var
  a: TAction;
  i: IInterface;
  o: TObject;
begin
  a := procedure
    begin
      Writeln('Hey.');
    end;
  i := PUnknown(@a)^;
  o := i as TObject; // Requires Delphi 2010
  o := Intf2Obj(i); // Workaround for non-D2010
  Writeln(o.ClassName);
end;

begin
  Go;
end.

This will (currently) print Go$0$ActRec; but if you have a second anonymous method, structurally identical, it will result in a second method, because anonymous method bodies are not compared for structural equality (it would be a high-cost, low-value optimization, as it's unlikely the programmer would do such a thing, and large structural comparisons aren't cheap).

If you were using a later version of Delphi, you could use RTTI on the class of this object and try and compare fields, and implement structural comparison yourself.

Clubby answered 1/3, 2011 at 11:33 Comment(22)
@Barry When exactly is the state captured? Suppose I have f1, f2: TProc. I then define f1 which captures state. Later on in the same routine, if I assign f2 := f1, is the state captured again?Cumin
Thanks for the information. A second class for a second anonymous method will be okay. The code you printed will not compile. The as-operator is not allowed for TObject. A test using the TObject(i)-cast looks a little bit illegal. We found a code similar to yours but could not make them work. Any idea? Are there any compiler settings that we've to do?Ashburn
@David My understanding of closures tells me, that the state is captured the moment the closure is created. If you do f2 = f1, nothing is created. So it won't capture again. You even could do the assignment in an other context, so it cannot re-capture the state.Shetler
@Shetler That's what my understanding told me. However my experiments suggest otherwise. It seems that there is not captured until there needs to be, e.g. the TFunc is returned out of the method whose state it refers to. I really don't understand any of this!!Cumin
I don't have a Blog and I can't put this amount of code in a comment, so here's when and how variables get captured: #5155414Terrieterrier
@Berry We think that the code you posted does only work with delphi XE, right? We're using delphi 2009. Are there any solutions for this version? (posted by Sebastian Schumann (not able to comment but working together with kiw))Shetler
@David In the sample code I wrote no state is captured; but in general, accesses to state (variables and parameters) by anonymous methods are rewritten at compile time (to fields on a heap-allocated object), and this state is allocate at procedure entry time. When it is deallocated depends on when the last method reference goes out of scope (i.e. the reference counting).Clubby
@Shetler - the TObject cast thing, I don't recall which version it was released in - I think it may have been Delphi 2010, sorry. But again, if you don't have RTTI to compare fields, it's probably not a big help to you. There is another way to go from the interface to object instance - switch to CPU view and step through the opcodes for an interface or method reference call and you'll see it goes through a stub that jumps to the final destination after altering EAX (in register calling convention) - this alteration to EAX is the delta between the interface reference and the object-typed Self.Clubby
@Shetler - you could disassemble that stub, i.e. check that it's a JMP rel32 after a SUB EAX,imm32 by checking the opcode bytes are as expected) and extracting the imm32 32-bit immediate operand to the SUB. That immediate operand (a 32-bit integer) would tell you what to subtract from the interface reference to get to the instance reference. It would be an ugly hack, but as long as you check the stub opcode bytes, it would be safe (i.e. dynamically checked).Clubby
@Barry Thanks. I'd always imagined that each anonymous function instance (if I can call it that) carried its own copy of the captured state. Now I understand that capturing (if that's still the right term) only really occurs when an anonymous method is returned out of a method. What's more two anonymous methods defined in the same method are forever linked by their common, shared state. I must say I'm impressed you got all this to work - it seems like it would be extraordinarily hard to implement on your side of the fence!Cumin
@Shetler @Sebastian I wrote a procedure Intf2Obj that should work for almost all 32-bit versions of Delphi for interfaces implemented by Delphi objects, where the vtable was written by the compiler. It may help.Clubby
@David the fun doesn't really start until you have anonymous methods nested inside anonymous methods, and also have to deal with nested procedures, to arbitrary recursive depth, and also with generics (which makes the classes holding the state generic). Lambda lifting is a pretty aggressive transformation of the original source, but the amount of work it does also contains its power, the way it implicitly passes state along and lets you write library functions (e.g. parallel-for) that are parameterized by code rather than just data.Clubby
@Barry Now you are making it sound even more hard!! I actually came across what I think is a bug with nested anonymous functions recently? Would it be worthwhile submitting it to QC?Cumin
@Barry Unfortunately, all that disassembly stuff and your function does not really sound like something we would want to use in production code. I do like that hacky stuff, but just for having tried it out. But thanks anyway!Shetler
@Shetler - no problem; just be aware that you are using some of my hacky code already, albeit professionally tested... ;)Clubby
@Barry I've just noticed that you edited Intf2Obj in your answer and we actually gave it a try. It seems to do its job quite well, but it still doesn't help with the equality/equivalency comparison.Shetler
@Shetler Barry already answered the equality/equivalency comparison question. It can't be done. You need to use a cookie type approach.Cumin
@Barry I just submitted QC#91876. It's of no personal interest to myself, but it may be valuable to you because there may well be a wider problem lurking. Or I might be doing something wrong, it's happened many times before!Cumin
@Shetler - the only use of getting at the instance is to then try and compare instance data. Getting a match would need the data to be exactly the same, though...Clubby
@David - that code should work in XE (it does in my fulcrum_updates branch, which is the same source base as XE); it also works on the trunk, but not in 64-bit...Clubby
@Barry Thanks. I'm holding off on XE having just gone from D6 to 2010. Hoping to move straight to the version that supports 64 bit.Cumin
@Barry - Even if you can compare the instances, you still can not distinguish two different methods of that instance, can you?Shetler

© 2022 - 2024 — McMap. All rights reserved.