How are anonymous methods implemented under the hood?
Asked Answered
O

2

13

Does Delphi "instantiate" each anonymous method (like an object)?, if so when does Delphi create this instance, and most important, when does Delphi free it?

Because anonymous method also captures external variables and extends their life time, it's important to know when these variables will be "released" from the memory.

What are the possible drawbacks to declare an anonymous method inside another anonymous method. Are circular reference possible?

Ogata answered 10/10, 2016 at 9:20 Comment(3)
See also Variable Binding Mechanism.Archer
See also: blog.barrkel.com/2008/07/anonymous-method-details.htmlFlyweight
FWIW, Barry Kelly implemented anonymous methods.Flyweight
A
21

Anonymous methods are implemented as interfaces. This article has a good explanation of how it is done by the compiler: Anonymous methods in Delphi: the internals.

In essence, the compiler generated interface has a single method named Invoke, behind which is the anonymous method that you provide.

Captured variables have the same lifetime as any anonymous methods that capture them. The anonymous method is an interface and its lifetime is managed by reference counting. Therefore, the captured variables life extends as long as does the anonymous methods that capture them.

Just as it is possible for circular references to be created with interfaces, it must equally be possible for circular references to be created with anonymous methods. Here is the simplest demonstration that I can construct:

uses
  System.SysUtils;

procedure Main;
var
  proc: TProc;
begin
  proc :=
    procedure
    begin
      if Assigned(proc) then
        Beep;
    end;
end;

begin
  ReportMemoryLeaksOnShutdown := True;
  Main;
end.

Behind the scenes the compiler creates a hidden class that implements the anonymous method interface. That class contains as data members any variables that are captured. When proc is assigned to, that increases the reference count on the implementing instance. Since proc is owned by the implementing instance, that instance therefore has taken a reference to itself.

To make this a little clearer, this program presents the identical issue but re-cast in terms of interfaces:

uses
  System.SysUtils;

type
  ISetValue = interface
    procedure SetValue(const Value: IInterface);
  end;

  TMyClass = class(TInterfacedObject, ISetValue)
    FValue: IInterface;
    procedure SetValue(const Value: IInterface);
  end;

procedure TMyClass.SetValue(const Value: IInterface);
begin
  FValue := Value;
end;

procedure Main;
var
  intf: ISetValue;
begin
  intf := TMyClass.Create;
  intf.SetValue(intf);
end;

begin
  ReportMemoryLeaksOnShutdown := True;
  Main;
end.

It is possible to break the circularity by explicitly clearing the self-reference. In the anonymous method example that looks like this:

procedure Main;
var
  proc: TProc;
begin
  proc :=
    procedure
    begin
      if Assigned(proc) then
        Beep;
    end;
  proc := nil;
end;

The equivalent for the interface variant is:

procedure Main;
var
  intf: ISetValue;
begin
  intf := TMyClass.Create;
  intf.SetValue(intf);
  intf.SetValue(nil);
end;
Aiglet answered 10/10, 2016 at 9:38 Comment(4)
method 2 does not seem to capture method 1 hereGarton
@Arioch'The It doesn't need to for there to be circularity and a memory leak.Aiglet
yes, I see the last paragraph of Stefan's answer, but your answer does not mention/explain it. Your answer does not explain while linear reference graph suddenly becomes circularGarton
@Arioch'The In my edit, I have produced a much simpler example, and explained it.Aiglet
T
14

Anonymous methods are implemented as interfaces with a method called Invoke that has the same signature as the anonymous method declaration. So technically a reference to function(a: Integer): string is binary compatible to this interface:

X = interface
  function Invoke(a: Integer): string;
end;

Up to a few versions ago it was even possible to call .Invoke on anonymous methods but the compiler now prevents that.

When you are declaring an anonymous method inline the compiler creates some code in the prologue of the routine to ensure that any variable that is captured does not live on the stack but on the heap (this is also the reason why you cannot inspect any captured variable during debugging because it is unfortunately lacking that information). The compiler creates a class behind that interface with fields that have the same name as the variables that you are capturing (see this blog article for more information).

As for circular references, yes. Be aware that when you for example capture an interface (or object in case of nextgen platforms where you have ARC for objects enabled) you might cause a circular reference causing a memory leak.

Also it is interesting to know that in case you have multiple anonymous methods inside the same routine they are all backed by the same compiler generated object. This might cause another situation where a memory leak might appear because one anonymous method might also capture the other and create another circular reference.

Thorlie answered 10/10, 2016 at 9:34 Comment(6)
I thought that captured variables (locals, parameters, globals) were implemented as fields (members) of the interface? I know that was the plan, but I never looked closely enough if that is really the case.Flyweight
@RudyVelthuis since when can interfaces have fields?Thorlie
The implementing object is the anonymous method implementation. The interface is only used for lifetime management. As Barry Kelly (who implemented anonmeths) said: "Basically, the variable 'x' (or any other variable touched by an anonymous method) gets hoisted out and turned into a field on a class. An instance of that class is created when the function is entered, and anonymous methods get converted into methods on that hidden class." See his comments here: reddit.com/r/programming/comments/6tp3i/…Flyweight
@RudyVelthuis Which is what I wrote in my third paragraph.Thorlie
@StefanGlienke Whilst thinking about the issue you outline in your final paragraph, I realised that the issue can be produced with just a single anonymous method that references itself. My answer illustrates that now.Aiglet
You can find that out after reading the very first sentence imo. If an interface (or rather the object behind that interface) keeps a reference to itself (as interface) it's clear to everybody that this is a mem leak because of referencing itself. But its not obvious that two different anonymous methods are backed by the same object instance.Thorlie

© 2022 - 2024 — McMap. All rights reserved.