Why can't I return arbitrary array of string?

Question

The compiler allows me to do the following:

procedure MyProc(const ADynData: array of string);

or

procedure MyProc(const ADynData: TStringDynArray);

and pass arbitrary data like so:

MyProc(['Data1', 'Data2']);

However, won't allow

function MyFunc: TStringDynArray;
....
function MyFunc: TStringDynArray;
begin
    Result := ['Data1', 'Data2'];
end;

or

function MyFunc: TStringDynArray;
const
    CDynData: array[0..1] of string = ('Data1', 'Data2');
begin
    Result := CDynData;
end;

Why is this? Isn't this technically the same thing?

For these particular scenarios what is the recommended (and most efficient) way of returning an arbitrary array of string?

Heartwood

It is idiomatic in Delphi to use a var-array-of parameter instead of trying to return a dynamic array. Remember that Delphi does not have the large elaborate set of value and reference semantics that C++ does, and that this imposes some limitations on what you can return from a function in Delphi. In particular, Delphi's memory management of dynamic arrays is better handled by a var array (to return). Your example above would be more directly comparable if you used "var" instead of const since returning an array and passing one in are two very different things in memory management. – Mertens

@WarrenP No so. In fact it is idiomatic to return a dynamic array. That allows the callee to decide how long the array should be. The current idiom is to return TArray<T>. Try setting the length of an open array! – Bloodsucker

Okay, in Modern Delphi versions that support generics, David is right. However, the classic Delphi curmudgeons out there still don't rely on Generic code because we don't trust it. So we use var parameters and we let the classic technique continue to operate as well as it always has. Next time we see a horrible breakage posted on QualityCentral where the whole thing dies horribly when a compiler bug gets hit... We will try not to gloat. :-) – Mertens

@WarrenP I ask again, what do you do when the callee determines the length? The answer is the same even in pre-generics versions of Delphi. – Bloodsucker

good point. There are valid reasons to avoid the old pattern. – Mertens

Well the question is tagged with delphi-xe3 so generics are an acceptable solution - so long as there isn't another solution which doesn't use it. – Cuneal

@JerryDodge What do you mean by "so long as there isn't another solution which doesn't use it"? – Bloodsucker

I was referring to your first comment about TArray<T> and Warren's response about prior versions. If this question wasn't tagged a specific version, then generics should not be even considered, but since XE3 is in the tag... – Cuneal

Answer 1

No, it's not the same thing. In

procedure MyProc(const ADynData: array of string);

the argument is an open array parameter, which is not the same thing as an 'ordinary' dynamic array. The [..] syntax can only be used to create open arrays in open array parameters of functions. (Otherwise, [..] is used to specify sets in code, such as Font.Style := [fsBold, fsItalic]. But sets can only have ordinal types as their 'base types', so there is still no such thing as 'set of string'.)

In other words, it is not possible to write a dynamic array in code like you try in your second code snippet,

function MyFunc: TStringDynArray;
begin
  result := ['Data1', 'Data2']; // Won't work.
end;

However, in new versions of Delphi, it is almost possible:

type
  TStringDynArray = array of string;

function MyFunc: TStringDynArray;
begin
  result := TStringDynArray.Create('A', 'B');
end;

Finally,

function MyFunc: TStringDynArray;
const
  CDynData: array[0..1] of string = ('Data1', 'Data2');
begin
  result := CDynData;
end;

won't work because TStringDynArray is a dynamic array, while CDynData is a static array, which are two different fundamental types.

Opine

I can also do procedure MyProc(const ADynData: TStringDynArray); and pass an arbitrary array as well. So basically you can't have open array return values? – Heartwood

@James: True. Open arrays are only for parameters. – Opine

Hmm seems like a bit of a limitation, unless there's a technical reason behind why they don't allow it. Do you think there would be any performance implications if creating TStringDynArray everytime? Would it be worth caching the result (if it's not going to change). – Heartwood

@James: Would it be worth...: Common sense and testing will answer that. – Opine

Yeah I get that... and will do. What I was getting at was whether you had an opinion on whether creating a TStringDynArray is an expensive operation. In fairness, common sense would really say "cache everything" but we know that isn't always necessary. – Heartwood

@James: That entirely depends on how big the array is, and how critical performance is. But it is sure not any more expensive that a SetLength followed by a loop assignment of elements. You should also remember that dynamic arrays are actually pointers to the actual array, and so if A is a dynamic array and B has the same type, then the assignment B := A will not copy any data; instead, B and A will now point at the same array. – Opine

Do you know of any other hidden gems like TStringDynArray.Create, for example TStringDynArray.Add or TStringDynArray.Slice and where I could find out about them? – Incarcerate

Answer 2

This construct

['string1', 'string2'] 

is known as an open array constructor. From the documentation:

Open array constructors allow you to construct arrays directly within function and procedure calls.

They can be passed only as open array parameters or variant open array parameters.

So, you cannot use an open array constructor to create a function return value.

---

If you have a fixed number of elements in the array that you need to return, you can use a dynamic array constructor:

Result := TStringDynArray.Create('string1', 'string2');

---

However, this will not work for a variable number of elements. Now, I know that the example in your question only has a constant number of elements in the array. But I'm sure you'll encounter situations where you need more flexibility than a dynamic array constructor can provide.

If you wish to create a copy of an existing dynamic array and return that, use Copy.

Result := Copy(SomeOtherDynamicArray);

This breaks down when you have an open array at hand. You cannot pass an open array to Copy. Personally I think this is rather a shame since open array parameters are so exceptionally flexible and useful that I'd like to see as much RTL support for them as possible.

So, you end up having to write helper functions for those situations. You can write a dedicated helper for each array type, but that becomes somewhat tiresome. That's where generics come in handy. I have a helper class for the purpose. Here's the relevant extract:

type
  TArray = class(Generics.Collections.TArray)
    ....
    class function Copy<T>(const Source: array of T): TArray<T>; overload; static;
    ....
  end;

class function TArray.Copy<T>(const Source: array of T): TArray<T>;
var
  i: Integer;
begin
  SetLength(Result, Length(Source));
  for i := 0 to high(Result) do begin
    Result[i] := Source[i];
  end;
end;

Now, this works with your string arrays, but also with any other type. Call it like this:

Result := TArray.Copy<string>(SomeStringOpenArray);

A critical point to make is that we are using the generic version of the dynamic array, TArray<string> rather than TStringDynArray. It's essential that you do that if you want to use generics seriously. That's because TStringDynArray is not assignment compatible with TArray<string> or indeed any other type declared as array of string. It pays dividends to change your code base to use TArray<T> throughout.

Just in case anyone is interested in the rest of this helper class, here it is:

type
  TArray = class(Generics.Collections.TArray)
  private
    class function Comparison<T>(SortType: TSortType): TComparison<T>; static;
    class function Comparer<T>(const Comparison: TComparison<T>): IComparer<T>; static;
  public
    class procedure Swap<T>(var Left, Right: T); static;
    class procedure Reverse<T>(var Values: array of T); static;
    class function Reversed<T>(const Values: array of T): TArray<T>; static;
    class function Contains<T>(const Values: array of T; const Item: T; out ItemIndex: Integer): Boolean; overload; static;
    class function Contains<T>(const Values: array of T; const Item: T): Boolean; overload; static;
    class function IndexOf<T>(const Values: array of T; const Item: T): Integer; static;
    class function Sorted<T>(var Values: array of T; SortType: TSortType; Index, Count: Integer): Boolean; overload; static;
    class function Sorted<T>(var Values: array of T; SortType: TSortType): Boolean; overload; static;
    class function Sorted<T>(var Values: array of T; const Comparison: TComparison<T>; Index, Count: Integer): Boolean; overload; static;
    class function Sorted<T>(var Values: array of T; const Comparison: TComparison<T>): Boolean; overload; static;
    class function Sorted<T>(GetValue: TFunc<Integer,T>; const Comparison: TComparison<T>; Index, Count: Integer): Boolean; overload; static;
    class procedure Sort<T>(var Values: array of T; SortType: TSortType; Index, Count: Integer); overload; static;
    class procedure Sort<T>(var Values: array of T; SortType: TSortType); overload; static;
    class procedure Sort<T>(var Values: array of T; const Comparison: TComparison<T>; Index, Count: Integer); overload; static;
    class procedure Sort<T>(var Values: array of T; const Comparison: TComparison<T>); overload; static;
    class function Copy<T>(const Source: array of T; Index, Count: Integer): TArray<T>; overload; static;
    class function Copy<T>(const Source: array of T): TArray<T>; overload; static;
    class procedure Move<T>(const Source: array of T; var Dest: array of T; Index, Count: Integer); overload; static;
    class procedure Move<T>(const Source: array of T; var Dest: array of T); overload; static;
    class function Concatenated<T>(const Source1, Source2: array of T): TArray<T>; overload; static;
    class function Concatenated<T>(const Source: array of TArray<T>): TArray<T>; overload; static;
    class procedure Initialise<T>(var Values: array of T; const Value: T); static;
    class procedure Zeroise<T>(var Values: array of T); static;
    class function GetHashCode<T>(const Values: array of T): Integer; overload; static;
    class function GetHashCode<T>(Values: Pointer; Count: Integer): Integer; overload; static;
  end;

class function TArray.Comparison<T>(SortType: TSortType): TComparison<T>;
var
  DefaultComparer: IComparer<T>;
begin
  DefaultComparer := TComparer<T>.Default;
  Result :=
    function(const Left, Right: T): Integer
    begin
      case SortType of
      stIncreasing:
        Result := DefaultComparer.Compare(Left, Right);
      stDecreasing:
        Result := -DefaultComparer.Compare(Left, Right);
      else
        RaiseAssertionFailed(Result);
      end;
    end;
end;

class function TArray.Comparer<T>(const Comparison: TComparison<T>): IComparer<T>;
begin
  Result := TComparer<T>.Construct(Comparison);
end;

class procedure TArray.Swap<T>(var Left, Right: T);
var
  temp: T;
begin
  temp := Left;
  Left := Right;
  Right := temp;
end;

class procedure TArray.Reverse<T>(var Values: array of T);
var
  bottom, top: Integer;
begin
  bottom := 0;
  top := high(Values);
  while top>bottom do begin
    Swap<T>(Values[bottom], Values[top]);
    inc(bottom);
    dec(top);
  end;
end;

class function TArray.Reversed<T>(const Values: array of T): TArray<T>;
var
  i, j, Count: Integer;
begin
  Count := Length(Values);
  SetLength(Result, Count);
  j := Count-1;
  for i := 0 to Count-1 do begin
    Result[i] := Values[j];
    dec(j);
  end;
end;

class function TArray.Contains<T>(const Values: array of T; const Item: T; out ItemIndex: Integer): Boolean;
var
  DefaultComparer: IEqualityComparer<T>;
  Index: Integer;
begin
  DefaultComparer := TEqualityComparer<T>.Default;
  for Index := 0 to high(Values) do begin
    if DefaultComparer.Equals(Values[Index], Item) then begin
      ItemIndex := Index;
      Result := True;
      exit;
    end;
  end;
  ItemIndex := -1;
  Result := False;
end;

class function TArray.Contains<T>(const Values: array of T; const Item: T): Boolean;
var
  ItemIndex: Integer;
begin
  Result := Contains<T>(Values, Item, ItemIndex);
end;

class function TArray.IndexOf<T>(const Values: array of T; const Item: T): Integer;
begin
  Contains<T>(Values, Item, Result);
end;

class function TArray.Sorted<T>(var Values: array of T; SortType: TSortType; Index, Count: Integer): Boolean;
begin
  Result := Sorted<T>(Values, Comparison<T>(SortType), Index, Count);
end;

class function TArray.Sorted<T>(var Values: array of T; SortType: TSortType): Boolean;
begin
  Result := Sorted<T>(Values, Comparison<T>(SortType));
end;

class function TArray.Sorted<T>(var Values: array of T; const Comparison: TComparison<T>; Index, Count: Integer): Boolean;
var
  i: Integer;
begin
  for i := Index+1 to Index+Count-1 do begin
    if Comparison(Values[i-1], Values[i])>0 then begin
      Result := False;
      exit;
    end;
  end;
  Result := True;
end;

class function TArray.Sorted<T>(var Values: array of T; const Comparison: TComparison<T>): Boolean;
begin
  Result := Sorted<T>(Values, Comparison, 0, Length(Values));
end;

class function TArray.Sorted<T>(GetValue: TFunc<Integer, T>; const Comparison: TComparison<T>; Index, Count: Integer): Boolean;
var
  i: Integer;
begin
  for i := Index+1 to Index+Count-1 do begin
    if Comparison(GetValue(i-1), GetValue(i))>0 then begin
      Result := False;
      exit;
    end;
  end;
  Result := True;
end;

class procedure TArray.Sort<T>(var Values: array of T; SortType: TSortType; Index, Count: Integer);
begin
  Sort<T>(Values, Comparison<T>(SortType), Index, Count);
end;

class procedure TArray.Sort<T>(var Values: array of T; SortType: TSortType);
begin
  Sort<T>(Values, SortType, 0, Length(Values));
end;

class procedure TArray.Sort<T>(var Values: array of T; const Comparison: TComparison<T>; Index, Count: Integer);
begin
  if not Sorted<T>(Values, Comparison, Index, Count) then begin
    Sort<T>(Values, Comparer<T>(Comparison), Index, Count);
  end;
end;

class procedure TArray.Sort<T>(var Values: array of T; const Comparison: TComparison<T>);
begin
  Sort<T>(Values, Comparison, 0, Length(Values));
end;

class function TArray.Copy<T>(const Source: array of T; Index, Count: Integer): TArray<T>;
var
  i: Integer;
begin
  SetLength(Result, Count);
  for i := 0 to high(Result) do begin
    Result[i] := Source[i+Index];
  end;
end;

class function TArray.Copy<T>(const Source: array of T): TArray<T>;
var
  i: Integer;
begin
  SetLength(Result, Length(Source));
  for i := 0 to high(Result) do begin
    Result[i] := Source[i];
  end;
end;

class procedure TArray.Move<T>(const Source: array of T; var Dest: array of T; Index, Count: Integer);
var
  i: Integer;
begin
  for i := 0 to Count-1 do begin
    Dest[i] := Source[i+Index];
  end;
end;

class procedure TArray.Move<T>(const Source: array of T; var Dest: array of T);
var
  i: Integer;
begin
  for i := 0 to high(Source) do begin
    Dest[i] := Source[i];
  end;
end;

class function TArray.Concatenated<T>(const Source1, Source2: array of T): TArray<T>;
var
  i, Index: Integer;
begin
  SetLength(Result, Length(Source1)+Length(Source2));
  Index := 0;
  for i := low(Source1) to high(Source1) do begin
    Result[Index] := Source1[i];
    inc(Index);
  end;
  for i := low(Source2) to high(Source2) do begin
    Result[Index] := Source2[i];
    inc(Index);
  end;
end;

class function TArray.Concatenated<T>(const Source: array of TArray<T>): TArray<T>;
var
  i, j, Index, Count: Integer;
begin
  Count := 0;
  for i := 0 to high(Source) do begin
    inc(Count, Length(Source[i]));
  end;
  SetLength(Result, Count);
  Index := 0;
  for i := 0 to high(Source) do begin
    for j := 0 to high(Source[i]) do begin
      Result[Index] := Source[i][j];
      inc(Index);
    end;
  end;
end;

class procedure TArray.Initialise<T>(var Values: array of T; const Value: T);
var
  i: Integer;
begin
  for i := 0 to high(Values) do begin
    Values[i] := Value;
  end;
end;

class procedure TArray.Zeroise<T>(var Values: array of T);
begin
  Initialise<T>(Values, Default(T));
end;

{$IFOPT Q+}
  {$DEFINE OverflowChecksEnabled}
  {$Q-}
{$ENDIF}
class function TArray.GetHashCode<T>(const Values: array of T): Integer;
// see http://stackoverflow.com/questions/1646807 and http://stackoverflow.com/questions/11294686
var
  Value: T;
  EqualityComparer: IEqualityComparer<T>;
begin
  EqualityComparer := TEqualityComparer<T>.Default;
  Result := 17;
  for Value in Values do begin
    Result := Result*37 + EqualityComparer.GetHashCode(Value);
  end;
end;

class function TArray.GetHashCode<T>(Values: Pointer; Count: Integer): Integer;
// see http://stackoverflow.com/questions/1646807 and http://stackoverflow.com/questions/11294686
var
  Value: ^T;
  EqualityComparer: IEqualityComparer<T>;
begin
  EqualityComparer := TEqualityComparer<T>.Default;
  Result := 17;
  Value := Values;
  while Count>0 do begin
    Result := Result*37 + EqualityComparer.GetHashCode(Value^);
    inc(Value);
    dec(Count);
  end;
end;
{$IFDEF OverflowChecksEnabled}
  {$Q+}
{$ENDIF}

Bloodsucker

Please don't kill me for being pedantic, but there really shouldn't be a colon on the first line IMNSHO. – Opine

@Andreas Rather than killing you, I applaud you. Pedantry is a fine and desirable trait in my view. – Bloodsucker

Answer 3

The problem with

function MyFunc: TStringDynArray;
begin
  Result := ['Data1', 'Data2'];
end;

is that ['Data1', 'Data2'] is interpreted as a set.

I sometimes use the following convenience function (but usually not in performance-critical sections):

function MakeStringArray(const Strings: array of string): TStringDynArray;
var
  i: Integer;
begin
  SetLength(Result, Length(Strings));
  for i := Low(Strings) to High(Strings) do
    Result[i] := Strings[i];
end {MakeStringArray};

Using that, you could rewrite your first example as follows:

function MyFunc: TStringDynArray;
....
function MyFunc: TStringDynArray;
begin
    Result := MakeStringArray(['Data1', 'Data2']);
end;

But since you're using XE3, you're better off using TStringDynArray.Create, like Andreas Rejbrand suggests.

Frig

Yeah point is though you would expect the compiler to detect the return type and know it's a dynamic array. Seems like a bit of a limitation in general for me. – Heartwood

Although you are right that ['Data1', 'Data2'] syntactically would be interpreted as a set, it won't compile because sets cannot have strings as their 'base type'. – Opine

@AndreasRejbrand: Exactly. The error raised by the compiler explicitly mentions that it expects an ordinal type. This problem is basically due to re-use of the [...] symbols: they can denote both sets and arrays. In this case, the compiler isn't smart enough to see that it's supposed to be an array, not a set. – Frig

+1 for mentioning how one had to do before the advent of dynamic array constructors. – Opine