Delphi - MemoryStream or FileStream

Question

I am downloading an EXE file from internet using Indy (idHTTP), and I can use memorystream or filestream to save it to disk, but I really do not know if there is any difference between them (maybe in the result structure of the file?). I could't find yet an answer for this.

Where, here are 2 simple functions to simulate what I am doing:

Function DownloadMS(FUrl, Dest: String): Boolean;
var
  Http: TIdHTTP;
  Strm: TMemoryStream;
Begin
  Result := False;
  Http := TIdHTTP.Create;
  Strm := TMemoryStream.Create;
  With Http, Strm Do
  Try
    Try
      Get(FUrl, Strm);
      If (Size > 0) Then
      Begin
        Position := 0;
        SaveToFile(Dest);
        Result := True;
      end;
    Except
    end;
  Finally
    Strm.Free;
    Http.Free;
  end;
end;

Function DownloadFS(FUrl, Dest: String): Boolean;
var
  Http: TIdHTTP;
  Strm: TFileStream;
Begin
  Result := False;
  Http := TIdHTTP.Create;
  Strm := TFileStream.Create(Dest, fmCreate);
  With Http, Strm Do
  Try
    Try
      Get(FUrl, Strm);
      Result := (Size > 0);
    Except
    end;
  Finally
    Strm.Free;
    Http.Free;
  end;
end;

What you experts think about using one or other type (memorystream or filestream)? Is there any difference in the structure of the EXE file when using one or other type? What type is recommended?

Thank you! Have a nice weekend!

Prescriptive

The TMemoryStream uses internally TFileStream for saving to file (for the SaveToFile method), so the answer is pretty simple - use TFileStream. – Vardhamana

I could think about 2 reasons for using TMemoryStream instead of TFileStream. Avoiding a cleanup in the filesystem in case of an exception and the need of some manipulations before saving it to a file. – Jaquelynjaquenetta

Your wanton use of with makes me scared. I'd recommend that you stop doing that. – Blasting

@bummi, I am creating an installer that download application files from my site, and I think you are right about using memorystream, because connection may fail and generate an exception. – Prescriptive

@DavidHeffernan, thank you for the advise. I will take care next time. – Prescriptive

@bummi, I'd say there are drawbacks for the approaches, temporary file mess on the one side and memory hogging on the other side. – Demesne

Answer 1

There is no difference between TMemoryStream or TFileStream from the stream point of view.

They are both streams and hold a stream of bytes and are both derived from TStream.

You can implement your function generalized like this

function DownloadToStream( const AUrl : String; ADest : TStream ): Boolean;
var
  LHttp: TIdHTTP;
begin
  LHttp := TIdHTTP.Create;
  try
    LHttp.Get( AUrl, ADest );
    Result := ADest.Size > 0;
  finally
    LHttp.Free;
  end;
end;

and call it with a TFileStream

var
  LStream : TStream;

begin
  LStream := TFileStream.Create( 'MyFile.exe', fmCreate );
  if DownloadToStream( '', LStream ) then
    ...
end;

or TMemoryStream or whatever stream instance you like

Litmus

That is indeed how the code should be re-factored. But I think the question is not so much about code, but about design. Should memory stream or file stream be used for this purpose seems to be the question. – Blasting

@DavidHeffernan "I am downloading an EXE file from internet ... Is there any difference in the structure of the EXE file when using one or other type?" OP is wondering if TMemoryStream will store the data different than TFileStream would do :o) – Litmus

I didn't read the question that way, I suppose because it seems bizarre that anyone could think that the content of a stream is changed when you store it to disk rather than memory. That's just weird! – Blasting

Answer 2

In many cases there will be no point in putting an intermediate memory stream in between the download and the file. All that will do is consume memory because you have to put the entire file in memory before you can put it to disk. Using a file stream directly avoids that issue.

The main situation where the file stream option has problems is if you want to be sure that you've downloaded the entire file successfully before saving to disk. For example, if you are overwriting a previous version of a file, you may want to download it, check a hash signature, and only then overwrite the original file. In that scenario you need to put the file to some temporary location before over-writing. You could use a memory stream, or you could use a file stream using a temporary file name.

Blasting

Yes, I am creating an installer, and maybe user will be updating the files. So I think I will use MemoryStream, because files are not so big (2 or 3 MB). Thank you! – Prescriptive

@Paruba You can still use a file stream if you wish. Just use a temporary file name. When you are happy the file downloaded, delete the old and rename the temp file. – Blasting

Using a TFileStream has another benefit - the ability to resume a broken download without starting over from scratch. If the HTTP server supports byte ranges, you can configure TIdHTTP to tell the server what portion of the remote file to send, instead of the entire file. For small files, this does not matter much, but for larger files it can, even the ability to download different sections of the same file using multiple threads at one time. – Assignation

TFilestream can be MUCH slower if saving across a complex or inefficient network and if saving or reading mostly very little chunks of data. – Annmaria

@MikeScott hard to imagine it being slower than the Internet which is the data source. But since the ultimate destination is the file there's no escaping that. An intermediate memory stream doesn't mean you don't have to go to the file later. The only pertinent point you make though is that writing very small chunks of data can be slow. So use a buffered file stream. – Blasting