Delphi XE4 Indy compatibility issue between TBytes and TidBytes

Question

Today I try to compile my XE3 project in XE4. First problem that I face is with Indy's FTCPClient.Socket.ReadBytes() method.

Before it was accepting TBytes type, now it insists on TidBytes.

Definitions: TIdBytes = array of Byte; TBytes, Im not sure I guess it is generics something like TArray which is array of Byte.

Question number 1: Why does compiler complain by saying that'[dcc32 Error] HistoricalStockData.pas(298): E2033 Types of actual and formal var parameters must be identical'. As I see they are already identical.

Question number 2: Should I modify my source code with the each new delphi version?

Thanks.

Rafaello

Answer 1

The reason TIdBytes was a simple alias for TBytes in earlier Indy 10 releases was primarily for compatibility with SysUtils.TEncoding, which uses TBytes. Indy's TIdTextEncoding type used to be a simple alias for SysUtils.TEncoding in D2009+, so TIdBytes needed to be a simple alias for TBytes to match.

However, TBytes caused quite a bit of trouble for Indy in XE3, mainly because of RTTI problems with Generics (TBytes is a simple alias for TArray<Byte> in recent Delphi releases). So, Indy 10.6 re-designed TIdTextEncoding to no longer rely on SysUtils.TEncoding at all (there were other reasons as well for doing so), which then allowed TIdBytes to change into its own array type in order to avoid the XE3 issues moving forward.

On the other hand, you were passing a TBytes where a TIdBytes was expected, so that is bad programming on your part for not following Indy's defined interface in the first place. All of Indy 10's byte-based operations, including ReadBytes(), have always operated on TIdBytes only. The fact that TIdBytes silently mapped to TBytes was an implementation detail that you should not have relied on in your code. Indy 10 expects TIdBytes, so use TIdBytes, then you would not have compiler errors about incompatible types.

Moment

Libraries that invent their own types instead of using equivalent RTL types just leads to ghettoisation. How can we write code that uses Indy and its byte array and interacts with another library using its byte array? – Damned

First tell Embarcadero to stop breaking their own products when they make RTL changes. TBytes used to be a simple dynamic array (like TIdBytes is now). It worked great with RTTI, Object Inspector, compiler, etc. Then they switched TBytes to TArray<Byte> and broke all of that (bad Generics RTTI, bad C++ codegen, etc). Also remember that Indy supports multiple languages, and TArray<T> works differently in C++ than in Delphi. So there were multiple reasons for making TIdBytes go back to a simple dynamic array. I didn't make the change lightly, and even Embarcadero recommended I do it at the time. – Moment

OK, I'm sure you had good reason to change. It feels all wrong to me that in 2013 there's still debate over how to handle byte arrays. The "right" solution, assuming everything could be made to work, would be for all code to use TArray<T> directly and so enjoy the special type compatibility rules for generic types. So in an ideal world there would be no TBytes, no TIdBytes, and libraries could happily co-exist and interact smoothly. – Damned

Answer 2

The following two declarations are not the same, even though they appear to be. They're not assignment compatible, even though they're both based on array of string.

type
  TStringArrayOne = array of string;
  TStringArrayTwo = array of string;

var
  AVar1, AVar2: TStringArrayOne;
  AVar3, AVar4: TStringArrayTwo;
begin
  AVar1 := TStringArrayOne.Create('a', 'b', 'c');   // Compiles
  AVar2 := TStringArrayTwo.Create('a', 'b', 'c');   // Won't compile

  AVar3 := TStringArrayTwo.Create('a', 'b', 'c');   // Compiles
  AVar4 := TStringArrayOne.Create('a', 'b', 'c');   // Won't compile
end;

So TBytes and TIdBytes are not the same type, even if they're both defined as being array of Byte.

With regard to your question 2: It's a common problem with some third-party code. Indy in particular is known for making changes that breaks backward compatibility because they decide to reorganize or rewrite things between versions. Indy 10 was a major change from Indy 9, IIRC, and pretty much required a rewrite of most code that used it if you updated to the later version of Indy (even without updating Delphi at the same time). If you don't want to deal with those changes, you might want to look at using a more stable IP communications package. There are several available that are also free, open source packages.

Cheney

Who said I didn't enjoy it? ;-) – Cheney

Answer 3

In Indy 10.5.9 the type TIdBytes was defined differently depending on the presence of an existing TBytes type - see unit IdGlobal:

  {$IFDEF HAS_TBytes}
  TIdBytes = TBytes;
  {$ELSE}
  TIdBytes = array of Byte;
  {$ENDIF}

In Indy 10.6 (included in XE4), the declaration changed to unconditionally

  TIdBytes = array of Byte;

which means that starting with Indy 10.6, IdGlobal.TIdBytes is different from SysUtils.TBytes.

The second question is hard to answer, it is more a question of your priorities - other libraries are not immune against changes either, for example to improve performance or type-safety. Also changes in the Delphi language can always affect existing code.

Consumption