Store array in TQueue possible?

Question

Having problem storing an array in a TQueue. Any idea where I go wrong? Code works fine in Delphi XE 5 but not in Delphi 10 Seattle.

(I can't decide if this is a bug or how it should work. Tried searching embarcadero for clues but failed.)

procedure TForm1.Button1Click(Sender: TObject);
var
  FData: TQueue<TBytes>;
  FsData: TQueue<String>;

  arr: TBytes;

begin

  FData := TQueue<TBytes>.Create;
  FsData := TQueue<String>.Create;  
  try
    setlength(arr, 3);
    arr[0] := 1;
    arr[1] := 2;
    arr[2] := 3;

    FData.Enqueue(arr);
    Memo1.Lines.Add('Count, array:' + IntToStr(FData.Count));  // 0?

    FsData.Enqueue('asada');
    Memo1.Lines.Add('Count, string:' + IntToStr(FsData.Count));  // 1
  finally
    FData.Free;
    FsData.Free;
  end;
end;

Rianon

Aside, we don't need yet another incompatible byte array type. Use TBytes. More generally use TArray<T> for element types other than Byte. – Felecia

I agree. The origin array is TidBytes (Indy) – Rianon

@RudyVelthuis The comments in the code explain what is wrong. The repro in my answer is clearer. – Felecia

Yes, I saw it afterward. Seems a little buggy, indeed. – Luzluzader

Answer 1

This is a defect introduced in XE8. Here's the simplest reproduction that I can produce.

{$APPTYPE CONSOLE}

uses
  System.Generics.Collections;

var
  Queue: TQueue<TArray<Byte>>;

begin
  Queue := TQueue<TArray<Byte>>.Create;
  Queue.Enqueue(nil);
  Writeln(Queue.Count);
end.

The output is 1 in XE7 and 0 in XE8 and Seattle.

This has already been reported to Embarcadero: RSP-13196.

---

The implementation of Enqueue looks like this:

procedure TQueue<T>.Enqueue(const Value: T);
begin
  if IsManagedType(T) then
    if (SizeOf(T) = SizeOf(Pointer)) and (GetTypeKind(T) <> tkRecord) then
      FQueueHelper.InternalEnqueueMRef(Value, GetTypeKind(T))
    else
      FQueueHelper.InternalEnqueueManaged(Value)
  else
  case SizeOf(T) of
    1: FQueueHelper.InternalEnqueue1(Value);
    2: FQueueHelper.InternalEnqueue2(Value);
    4: FQueueHelper.InternalEnqueue4(Value);
    8: FQueueHelper.InternalEnqueue8(Value);
  else
    FQueueHelper.InternalEnqueueN(Value);
  end;
end;

When T is a dynamic array, the FQueueHelper.InternalEnqueueMRef branch is chosen. This in turn looks like this:

procedure TQueueHelper.InternalEnqueueMRef(const Value; Kind: TTypeKind);
begin
  case Kind of
    TTypeKind.tkUString: InternalEnqueueString(Value);
    TTypeKind.tkInterface: InternalEnqueueInterface(Value);
{$IF not Defined(NEXTGEN)}
    TTypeKind.tkLString: InternalEnqueueAnsiString(Value);
    TTypeKind.tkWString: InternalEnqueueWideString(Value);
{$ENDIF}
{$IF Defined(AUTOREFCOUNT)}
    TTypeKind.tkClass: InternalEnqueueObject(Value);
{$ENDIF}
  end;
end;

Note that there is no entry for TTypeKind.tkDynArray. Because these two methods are inlined, the inliner manages to compress it all down to nothing. No action is performed when you Enqueue a dynamic array.

Back in the good old days of XE7 the code looked like this:

procedure TQueue<T>.Enqueue(const Value: T);
begin
  if Count = Length(FItems) then
    Grow;
  FItems[FHead] := Value;
  FHead := (FHead + 1) mod Length(FItems);
  Inc(FCount);
  Notify(Value, cnAdded);
end;

No scope for type specific defects there.

---

I don't think that there's an easy workaround for you. Perhaps the most expedient way to proceed is to take the code for the XE7 TQueue and use that in place of the broken implementation from XE8 and Seattle. For the record, I've given up on the Embarcadero generic collections and use my own classes.

---

The back story here is that in XE8, Embarcadero decided to address a deficiency in their implementation of generics. Whenever you instantiate a generic type, copies of all the methods are created. For some methods, identical code is generated for different instantiations.

So it is quite common for TGeneric<TFoo>.DoSomething and TGeneric<TBar>.DoSomething to have identical code. Other compilers for other languages, C++ templates, .net generics, etc., recognise this duplication and merge together identical generic methods. The Delphi compiler does not. The end result is a larger executable than strictly necessary.

In XE8 Embarcadero decided to tackle this in what I regard was utterly the wrong way. Instead of attacking the root cause of the issue, the compiler, they decided to change the implementation of their generic collection classes. If you look at the code in Generics.Collections, you will see that it has been completely re-written in XE8. Where previously the code from XE7 and earlier was readable, from XE8 it is now exceedingly complex and opaque. This decision had the following consequences:

1. The complex code contained many errors. Many of these were found shortly after XE8 was released and have been fixed. You have stumbled upon another defect. One thing that we have learnt is that Embarcadero's internal test suite does not exercise their collection classes sufficiently. It is manifestly clear that their tests are inadequate.
2. By changing their library rather than the compiler, they have patched up the RTL classes. The original issue with generic code bloat remains for third party classes. Had Embarcadero fixed the issue at source then not only could they have retained the simple and correct collection class code from XE7, but all third generic code would have benefited.

Felecia

Thanks for sorting it out – Rianon