How to create a custom enumerator for a class derived from TDictionary?

Question

I have defined a collection derived from TDictionary, and need to define a custom enumerator that apply an additional filter.

I'm stuck as I can't access the TDictionary FItems array (it is private) so I can't define the MoveNext method

How would you proceed to redefine a filtered enumerator on a class derived from TDictionary?

Here's a simple code to illustrate what I want to do:

TMyItem = class(TObject)
public
  IsHidden:Boolean; // The enumerator should not return hidden items
end;
TMyCollection<T:TMyItem> = class(TDictionary<integer,T>)
public
   function GetEnumerator:TMyEnumerator<T>; // A value filtered enumerator
   type
     TMyEnumerator = class(TEnumerator<T>)
     private
       FDictionary: TMyCollection<integer,T>;
       FIndex: Integer;
       function GetCurrent: T;
     protected
       function DoGetCurrent: T; override;
       function DoMoveNext: Boolean; override;
     public
       constructor Create(ADictionary: TMyCollection<integer,T>);
       property Current: T read GetCurrent;
       function MoveNext: Boolean;
     end;
end;

function TMyCollection<T>.TMyEnumerator.MoveNext: Boolean;
begin
// In below code, FIndex is not accessible, so I can't move forward until my filter applies
  while FIndex < Length(FDictionary.FItems) - 1 do   
  begin
    Inc(FIndex);
    if (FDictionary.FItems[FIndex].HashCode <> 0) 
      and not(FDictionary.FItems[FIndex].IsHidden) then // my filter
      Exit(True);
  end;
  Result := False;
end;

Fucus

Answer 1

You can base your Enumerator on TDictionary's enumerator, so you don't actually need access to FItems. This works even if you write a wrapper class around TDictionary as Barry suggests. The enumerator would look like this:

TMyEnumerator = class
protected
  BaseEnumerator: TEnumerator<TPair<Integer, T>>; // using the key and value you used in your sample
public
  function MoveNext:Boolean;
  property Current:T read GetCurrent;
end;

function TMyEnumerator.MoveNext:Boolean;
begin
  Result := BaseEnumerator.MoveNext;
  while Result and (not (YourTestHere)) do // ie: the base enumerator returns everything, reject stuff you don't like
    Result := BaseEnumerator.MoveNext;
end;

function TMyEnumerator.Current: T;
begin
  Result := BaseEnumerator.Current.Value; // Based on your example, it's value you want to extract
end;

And here's a complete, 100 lines console application that demonstrates this:

program Project23;

{$APPTYPE CONSOLE}

uses
  SysUtils, Generics.Collections;

type

  TMyType = class
  public
    Int: Integer;
    constructor Create(anInteger:Integer);
  end;

  TMyCollection<T:TMyType> = class(TDictionary<integer,T>)
  strict private
    type
      TMyEnumerator = class
      protected
        BaseEnum: TEnumerator<TPair<Integer,T>>;
        function GetCurrent: T;
      public
        constructor Create(aBaseEnum: TEnumerator<TPair<Integer,T>>);
        destructor Destroy;override;

        function MoveNext:Boolean;
        property Current:T read GetCurrent;
      end;
  public
    function GetEnumerator: TMyEnumerator;
  end;

{ TMyCollection<T> }

function TMyCollection<T>.GetEnumerator: TMyEnumerator;
begin
  Result := TMyEnumerator.Create(inherited GetEnumerator);
end;

{ TMyType }

constructor TMyType.Create(anInteger: Integer);
begin
  Int := anInteger;
end;

{ TMyCollection<T>.TMyEnumerator }

constructor TMyCollection<T>.TMyEnumerator.Create(aBaseEnum: TEnumerator<TPair<Integer, T>>);
begin
  BaseEnum := aBaseEnum;
end;

function TMyCollection<T>.TMyEnumerator.GetCurrent: T;
begin
  Result := BaseEnum.Current.Value;
end;

destructor TMyCollection<T>.TMyEnumerator.Destroy;
begin
  BaseEnum.Free;
  inherited;
end;

function TMyCollection<T>.TMyEnumerator.MoveNext:Boolean;
begin
  Result := BaseEnum.MoveNext;
  while Result and ((BaseEnum.Current.Value.Int mod 2) = 1) do
    Result := BaseEnum.MoveNext;
end;

var TMC: TMyCollection<TMyTYpe>;
    V: TMyType;

begin
  try
    TMC := TMyCollection<TMyType>.Create;
    try
      // Fill TMC with some values
      TMC.Add(1, TMyType.Create(1));
      TMC.Add(2, TMyType.Create(2));
      TMC.Add(3, TMyType.Create(3));
      TMC.Add(4, TMyType.Create(4));
      TMC.Add(5, TMyType.Create(5));
      TMC.Add(6, TMyType.Create(6));
      TMC.Add(7, TMyType.Create(7));
      TMC.Add(8, TMyType.Create(8));
      // Filtered-enum
      for V in TMC do
        WriteLn(V.Int);
      ReadLn;
    finally TMC.Free;
    end;
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
end.

Onfroi

Thanks for this complete example, I have a better understanding now on how to use enumerators. I also discovered that wrapping enumerators has a huge performance penalty (in my sample application, an object query takes 2.2ms with the regular TDictionary enumerator, 3.3ms (+50%!) with the wrapped enumerator, without applying any filter). – Fucus

@user315561, what takes 2.2ms? 2.2 ms on a modern CPU is an awful long time, you probably miss-calculated something; And that renders the 3.3ms result for the wrapped enumerator equally wrong. – Onfroi

I agree that my figures, provided out of context, are not relevant. The use case is a query on a collection of 75000 objects, browsed sequentially with a simple equality filter, and 850 matching objects added to a new result collection. What I wanted to highlight here is the cost penalty of the enumerator wrapping (on the very same use case), which is relevant and good to know. – Fucus

Be aware that this approach will break if a reference of type TMyCollection<T> is assigned to a location of type TDictionary<Integer,T>, and GetEnumerator is subsequently called via that location, because it will call the TDictionary<Integer,T> version of GetEnumerator. This is just one manifestation of why it's a bad idea not to wrap the type instead. – Tenace

Answer 2

You should write a class that wraps TDictionary rather than inherits from it directly. The only reason TDictionary can be inherited from at all is so that TObjectDictionary could be defined and stay polymorphic with it. That is, the only proper support through overriding TDictionary is to customize what happens when keys and values are removed from the dictionary (so they might need to be freed).

Tenace

Thanks Barry, good recommendation. The other advantage will be to make my collection less coupled to the dictionary choice, making it easier to replace it with a higher performance dict when available – Fucus