C# Download big file from Server with less memory consumption

Question

I have a big file of memory size 42 mb. I want to download the file with less memory consumption.
Controller Code

public ActionResult Download()
{
    var filePath = "file path in server";
    FileInfo file = new FileInfo(filePath);
    Response.ContentType = "application/zip";                        
    Response.AppendHeader("Content-Disposition", "attachment; filename=folder.zip");                   
    Response.TransmitFile(file.FullName);
    Response.End(); 
}

alernative method tried with Stream

public ActionResult Download()
{           
    string failure = string.Empty;
    Stream stream = null;
    int bytesToRead = 10000;


    long LengthToRead;
    try
    {
        var path = "file path from server";
        FileWebRequest fileRequest = (FileWebRequest)FileWebRequest.Create(path);
        FileWebResponse fileResponse = (FileWebResponse)fileRequest.GetResponse();

        if (fileRequest.ContentLength > 0)
            fileResponse.ContentLength = fileRequest.ContentLength;

        //Get the Stream returned from the response
        stream = fileResponse.GetResponseStream();

        LengthToRead = stream.Length;

        //Indicate the type of data being sent
        Response.ContentType = "application/octet-stream";

        //Name the file 
        Response.AddHeader("Content-Disposition", "attachment; filename=SolutionWizardDesktopClient.zip");
        Response.AddHeader("Content-Length", fileResponse.ContentLength.ToString());

        int length;
        do
        {
            // Verify that the client is connected.
            if (Response.IsClientConnected)
            {
                byte[] buffer = new Byte[bytesToRead];

                // Read data into the buffer.
                length = stream.Read(buffer, 0, bytesToRead);

                // and write it out to the response's output stream
                Response.OutputStream.Write(buffer, 0, length);

                // Flush the data
                Response.Flush();

                //Clear the buffer
                LengthToRead = LengthToRead - length;
            }
            else
            {
                // cancel the download if client has disconnected
                LengthToRead = -1;
            }
        } while (LengthToRead > 0); //Repeat until no data is read

    }
    finally
    {
        if (stream != null)
        {
            //Close the input stream                   
            stream.Close();
        }
        Response.End();
        Response.Close();
    }
    return View("Failed");
}

due to size of the file, it is consumpting more memory which leads to performance issue.
After checking in iis log, the download process is taking 42 mb and 64 mb each respectively.
Thanks in advance

Ambulacrum

Stream is ur friend – Oria

Is there a reason you are not using the built in action results for just this scenario like FilePathResult / FileStreamResult and so on? – Toluol

I am using ActionResult – Ambulacrum

Are you using IIS as the web server? – Grit

i am using iis as web server. – Ambulacrum

Is the file local to the server (I mean with a path not like \\server\path or something funny)? Is impersonation configured on the server? These can cause TransmitFile (wich is the best) to revert back to standard operations – Grit

the file is place in d or e drive in server – Ambulacrum

Can you post the IIS line indicating the latency? Also you meant read the file or download? – Cooe

What's this doing exactly? LengthToRead = stream.Length; if you're reading perhaps you read it in chunks not to end – Cooe

Why don't you just post the file to some CDN and redirect the user to download it from there? – Ammunition

Can you take a look at the SO answer #3363349 – Carping

Answer 1

A better option would be to use FileResult instead of ActionResult:

Using this method means you don't have to load the file/bytes in memory before serving.

public FileResult Download()
{
     var filePath = "file path in server";
     return new FilePathResult(Server.MapPath(filePath), "application/zip");
}

Edit: For larger files FilePathResult will also fail.

Your best bet is probably Response.TransmitFile() then. I've used this on larger files (GBs) and had no issues before

public ActionResult Download()
{
   var filePath = @"file path from server";

    Response.Clear();
    Response.ContentType = "application/octet-stream";
    Response.AppendHeader("Content-Disposition", "filename=" + filePath);

    Response.TransmitFile(filePath);

    Response.End();

    return Index();
}

From MSDN:

Writes the specified file directly to an HTTP response output stream, without buffering it in memory.

Theodosia

the file will in c or d folder. So i think we can't use "Server.MapPath()" – Ambulacrum

What is the equivalent of 'Response' in ASP.NET Core 3.1? – Meritocracy

It is HttpContext.Response (available on Controller) – Baumbaugh

Answer 2

Try setting the Transfer-Encoding header to chunked, and return an HttpResponseMessage with a PushStreamContent. Transfer-Encoding of chunked means that the HTTP response will not have a Content-Length header, and so the client will have to parse the chunks of the HTTP response as a stream. Note, I've never run across a client (browser, etc) that didn't handle Transfer Encoding chunked. You can read more at the link below.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding

    [HttpGet]
    public async Task<HttpResponseMessage> Download(CancellationToken token)
    {
        var response = new HttpResponseMessage(System.Net.HttpStatusCode.OK)
        {
            Content = new PushStreamContent(async (stream, context, transportContext) =>
            {
                try
                {
                    using (var fileStream = System.IO.File.OpenRead("some path to MyBigDownload.zip"))
                    {
                        await fileStream.CopyToAsync(stream);
                    }
                }
                finally
                {
                    stream.Close();
                }
            }, "application/octet-stream"),
        };
        response.Headers.TransferEncodingChunked = true;
        response.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment")
        {
            FileName = "MyBigDownload.zip"
        };
        return response;
    }

Kofu

Answer 3

I had similar problem but I didn't have file on local disk, I had to download it from API (my MVC was like a proxy). The key thing is to set Response.Buffer=false; on your MVC Action. I think @JanusPienaar's first solution should work with this. My MVC action is:

public class HomeController : Controller
{
    public async Task<FileStreamResult> Streaming(long RecordCount)
    {
        HttpClient Client;
        System.IO.Stream Stream;

        //This is the key thing
        Response.Buffer=false;

        Client = new HttpClient() { BaseAddress=new Uri("http://MyApi", };
        Stream = await Client.GetStreamAsync("api/Streaming?RecordCount="+RecordCount);
        return new FileStreamResult(Stream, "text/csv");
    }
}

And my test WebApi (which generates the file) is:

public class StreamingController : ApiController
{
    // GET: api/Streaming/5
    public HttpResponseMessage Get(long RecordCount)
    {
        var response = Request.CreateResponse();

        response.Content=new PushStreamContent((stream, http, transport) =>
        {
            RecordsGenerator Generator = new RecordsGenerator();
            long i;

            using(var writer = new System.IO.StreamWriter(stream, System.Text.Encoding.UTF8))
            {
                for(i=0; i<RecordCount; i++)
                {
                    writer.Write(Generator.GetRecordString(i));

                    if(0==(i&0xFFFFF))
                        System.Diagnostics.Debug.WriteLine($"Record no: {i:N0}");
                    }
                }
            });

            return response;
        }

        class RecordsGenerator
        {
            const string abc = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
            char[] Chars = new char[14];//Ceiling(log26(2^63))

            public string GetRecordString(long Record)
            {
                int iLength = 0;
                long Div = Record, Mod;

                do
                {
                    iLength++;
                    Div=Math.DivRem(Div, abc.Length, out Mod);
                    //Save from backwards
                    Chars[Chars.Length-iLength]=abc[(int)Mod];
                }
                while(Div!=0);

                return $"{Record} {new string(Chars, Chars.Length-iLength, iLength)}\r\n";
            }
        }
    }
}

If RecordCount is 100000000, the file generated by TestApi is 1.56 GB. Neither WebApi nor MVC consumes so much memory.

Rego

I had a similar issue (the file was from API). I could not find a separate StackOverflow question dedicated to this issue and with solution as simple as this. Could you please share your solution as a separate question and post the answer there? – Baculiform

Answer 4

There is the Rizwan Ansari post that worked for me:

There are situation when you need to provide download option for a big file located somewhere on server or generated at runtime. Below function could be used to download files of any size. Sometimes downloading big file throws exception OutOfMemoryException showing “Insufficient memory to continue execution of the program”. So this function also handle this situation by breaking down file in 1 MB chunks (can be customized by changing bufferSize variable).

Usage:

DownloadLargeFile("A big file.pdf", "D:\\Big Files\\Big File.pdf", "application/pdf", System.Web.HttpContext.Current.Response);

You can change "application/pdf" by the right Mime type

Download Function:

public static void DownloadLargeFile(string DownloadFileName, string FilePath, string ContentType, HttpResponse response)
    {
        Stream stream = null;

        // read buffer in 1 MB chunks
        // change this if you want a different buffer size
        int bufferSize = 1048576;

        byte[] buffer = new Byte[bufferSize];

        // buffer read length
        int length;
        // Total length of file
        long lengthToRead;

        try
        {
            // Open the file in read only mode 
            stream = new FileStream(FilePath, FileMode.Open, FileAccess.Read, FileShare.Read);

            // Total length of file
            lengthToRead = stream.Length;
            response.ContentType = ContentType;
            response.AddHeader("Content-Disposition", "attachment; filename=" + HttpUtility.UrlEncode(DownloadFileName, System.Text.Encoding.UTF8));

            while (lengthToRead > 0)
            {
                // Verify that the client is connected.
                if (response.IsClientConnected)
                {
                    // Read the data in buffer
                    length = stream.Read(buffer, 0, bufferSize);

                    // Write the data to output stream.
                    response.OutputStream.Write(buffer, 0, length);

                    // Flush the data 
                    response.Flush();

                    //buffer = new Byte[10000];
                    lengthToRead = lengthToRead - length;
                }
                else
                {
                    // if user disconnects stop the loop
                    lengthToRead = -1;
                }
            }
        }
        catch (Exception exp)
        {
            // handle exception
            response.ContentType = "text/html";
            response.Write("Error : " + exp.Message);
        }
        finally
        {
            if (stream != null)
            {
                stream.Close();
            }
            response.End();
            response.Close();
        }
    }

Wartburg

Answer 5

you just have to Using IIS to Enable HTTP Downloads look at this link

and you just need to return the HTTP path of the file it will be download so fast and so easy.

Riproaring