PHP/Javascript chunked upload: IE9 corrupt file if filesize is over upload_max_filesize or post_max_size

Asked 17/1, 2015 at 11:12 Answered 25/1, 2015 at 17:57

Solved javascript php plupload chunked-encoding

I'm using Plupupload to upload files. if I try to load an exe with IE9 and the filesize it's over upload_max_filesize or post_max_size setting, the uploaded file is corrupt.

This is the PHP script that I am using:

<?php
/**
 * upload.php
 *
 * Copyright 2013, Moxiecode Systems AB
 * Released under GPL License.
 *
 * License: http://www.plupload.com/license
 * Contributing: http://www.plupload.com/contributing
 */

// Make sure file is not cached (as it happens for example on iOS devices)
header("Expires: Mon, 26 Jul 1997 05:00:00 GMT");
header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT");
header("Cache-Control: no-store, no-cache, must-revalidate");
header("Cache-Control: post-check=0, pre-check=0", false);
header("Pragma: no-cache");

// 5 minutes execution time
@set_time_limit(5 * 60);

// Settings
$targetDir  = __DIR__ . DIRECTORY_SEPARATOR . "upload";

// Create target dir
if (!file_exists($targetDir)) {
    @mkdir($targetDir);
}

// Get a file name
if (isset($_REQUEST["name"])) {
    $fileName = $_REQUEST["name"];
} elseif (!empty($_FILES)) {
    $fileName = $_FILES["file"]["name"];
} else {
    $fileName = uniqid("file_");
}

$filePath = $targetDir . DIRECTORY_SEPARATOR . $fileName;

// Chunking might be enabled
$chunk  = isset($_REQUEST["chunk"])  ? intval($_REQUEST["chunk"])  : 0;
$chunks = isset($_REQUEST["chunks"]) ? intval($_REQUEST["chunks"]) : 0;


// Open temp file
if (!$out = @fopen("{$filePath}.part", $chunks ? "ab" : "wb")) {
    die('{"jsonrpc" : "2.0", "error" : {"code": 102, "message": "Failed to open output stream."}, "id" : "id"}');
}

if (!empty($_FILES)) {
    if ($_FILES["file"]["error"] || !is_uploaded_file($_FILES["file"]["tmp_name"])) {
        die('{"jsonrpc" : "2.0", "error" : {"code": 103, "message": "Failed to move uploaded file."}, "id" : "id"}');
    }

    // Read binary input stream and append it to temp file
    if (!$in = @fopen($_FILES["file"]["tmp_name"], "rb")) {
        die('{"jsonrpc" : "2.0", "error" : {"code": 101, "message": "Failed to open input stream."}, "id" : "id"}');
    }
} else {    
    if (!$in = @fopen("php://input", "rb")) {
        die('{"jsonrpc" : "2.0", "error" : {"code": 101, "message": "Failed to open input stream."}, "id" : "id"}');
    }
}

while ($buff = fread($in, 4096)) {
    fwrite($out, $buff);
}

@fclose($out);
@fclose($in);

// Check if file has been uploaded
if (!$chunks || $chunk == $chunks - 1) {
    // Strip the temp .part suffix off 
    rename("{$filePath}.part", $filePath);
}

// Return Success JSON-RPC response
die('{"jsonrpc" : "2.0", "result" : null, "id" : "id"}');

upload occurs through the html page:

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" dir="ltr">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8"/>

<title>Plupload - Custom example</title>

<!-- production -->
<script type="text/javascript" src="../js/plupload.full.min.js"></script>

</head>
<body style="font: 13px Verdana; background: #eee; color: #333">

<h1>Custom example</h1>

<p>Shows you how to use the core plupload API.</p>

<div id="filelist">Your browser doesn't have Flash, Silverlight or HTML5 support.</div>
<br />

<div id="container">
    <a id="pickfiles" href="javascript:;">[Select files]</a> 
    <a id="uploadfiles" href="javascript:;">[Upload files]</a>
</div>

<br />
<pre id="console"></pre>


<script type="text/javascript">
// Custom example logic

var uploader = new plupload.Uploader({
    runtimes : 'html5,flash,silverlight,html4',
    browse_button : 'pickfiles', // you can pass in id...
    container: document.getElementById('container'), // ... or DOM Element itself
    url : 'upload.php',
    flash_swf_url : '../js/Moxie.swf',
    silverlight_xap_url : '../js/Moxie.xap',
    chunk_size : '2mb',

    filters : {
        max_file_size : '100mb',
        mime_types: [
            {title : "Image files", extensions : "jpg,gif,png"},
            {title : "Zip files", extensions : "zip"},
            {title : "Exe files", extensions : "exe"}
        ]
    },

    init: {
        PostInit: function() {
            document.getElementById('filelist').innerHTML = '';

            document.getElementById('uploadfiles').onclick = function() {
                uploader.start();
                return false;
            };
        },

        FilesAdded: function(up, files) {
            plupload.each(files, function(file) {
                document.getElementById('filelist').innerHTML += '<div id="' + file.id + '">' + file.name + ' (' + plupload.formatSize(file.size) + ') <b></b></div>';
            });
        },

        UploadProgress: function(up, file) {
            document.getElementById(file.id).getElementsByTagName('b')[0].innerHTML = '<span>' + file.percent + "%</span>";
        },

        Error: function(up, err) {
            document.getElementById('console').innerHTML += "\nError #" + err.code + ": " + err.message;
        }
    }
});

uploader.init();

</script>
</body>
</html>

When the exe are corrupt, if I try to open them with notepad++, I find:

enter image description here

My setting:

PHP Version 5.5.9
System          Windows NT PC-XXX 6.0 build 6002 (Windows Vista Service Pack 2) i586 
Compiler        MSVC11 (Visual C++ 2012) 
Architecture    x86 
Server API      Apache 2.0 Handler

php.ini

max_execution_time=30
max_input_time=60
memory_limit=128M
max_file_uploads=20

Additional info

All Plupupload methods (html5,flash,silverlight,html4) have the problem
Antivirus disabled
UAC is disabled

TRY ISSUE YOURSELF

I have created a package for anyone who wants to try.

Download package: http://www.sndesign.it/shared/stackoverflow/plupload-2.1.2.zip

My plupload-2.1.2.zip also contains a corrupted upload file in plupload-2.1.2/examples/upload/file_54c4c1d05c2ef folder and the file to try to upload plupload-2.1.2/examples/TryMe.exe

Prepare for the test (I use XAMPP Version 1.8.3):

Unzip plupload-2.1.2.zip in your htdocs
set php.ini upload_max_filesize=22M post_max_size=22M (less to TryMe.exe file size 23MB), restart Apache
Open IE9 (IE9 always fails), and go to: http://localhost/plupload-2.1.2/examples/custom.html
Select file in %YourHtdocs%/plupload-2.1.2/examples/TryMe.exe and upload
go in %YourHtdocs%/plupload-2.1.2/examples/upload/ and find the uploaded file
the uploaded file is corrupted.
set php.ini upload_max_filesize=24M post_max_size=24M (up to TryMe.exe file size 23MB), restart Apache
Select file in %YourHtdocs%/plupload-2.1.2/examples/TryMe.exe and upload
go in %YourHtdocs%/plupload-2.1.2/examples/upload/ and find the uploaded file
the uploaded file is ok.

Apospory answered 17/1, 2015 at 11:12 Comment(20)

@maytham is for a service that collects indie software, uploaded by developers. – Apospory 17/1, 2015 at 11:36

@maytham What kind of problems did you have ? I tried it with images and zip and works well – Apospory 17/1, 2015 at 11:41

@maytham I reported the problem even on github plupupload project . Stay tuned, maybe someone finds a solution... – Apospory 17/1, 2015 at 11:48

@maytham I started a bounty. if you are interested in this issue you can add bounty or vote up this question to give it more visibility. – Apospory 19/1, 2015 at 20:33

Since you said you can strip the headers out using notepad++, have you tried stripping them out programatically? – Unquestioned 19/1, 2015 at 20:40

@Unquestioned Yep. But I would like to solve the problem and not find a patch – Apospory 19/1, 2015 at 20:42

locate your problem by using one upload method each time (html5, flash etc.) – Marashio 19/1, 2015 at 20:53

@Marashio all methods have the problem. – Apospory 19/1, 2015 at 21:2

also try to upload renamed exe file with another extension – Marashio 19/1, 2015 at 21:12

@Marashio I tried to rename exe before upload to zip or jpg, but still does not work . thanks for your help – Apospory 19/1, 2015 at 21:16

@SimoneNigro just an idea, is it posible to find out if php are able to zip files fly and upload them? if that helps? – Dominik 19/1, 2015 at 21:26

should javascript zip on the fly :( – Apospory 19/1, 2015 at 21:27

@maytham I'm sorry but my English is very bad . What do you mean: ppl – Apospory 19/1, 2015 at 21:28

to work the exe file must be zipped before uploading. So, should the client to zip the fly, so javascript – Apospory 19/1, 2015 at 21:32

Hmmm true; I agree javascript way, i am not a ware of if there is a javascript to do this, if you find let me know? – Dominik 19/1, 2015 at 21:47

try to add ".exe" to "low risk files" like described here – Marashio 19/1, 2015 at 21:57

I think it must be some Group Policy setting. Turn off UAC in Vista. – Marashio 19/1, 2015 at 22:59

UAC is already disabled – Apospory 20/1, 2015 at 12:53

CHANGE EXTENSION FROM EXE TO ABC AND THEN UPLOAD – Corliss 25/1, 2015 at 18:18

@Corliss not work – Apospory 25/1, 2015 at 18:31

What we know is that an intact file is split up into chunks, each of which is prefixed by an HTTP multipart/form-data header and a Content-Disposition header. The first one is always stripped correctly, the second one is not.
This leaves us with 3 possibilities:

At least one of the headers is corrupted when the file is sent.
At least one of the headers is corrupted after it was sent by the browser but before it is parsed by PHP.
Something goes wrong while PHP is parsing the request.

The cause for any of the above could be destructive filtering by a firewall, antivirus or any other service that for some reason feels the need to go over your network traffic or RAM/file system activity. For 1. it could also be a bug in the browser/JavaScript/Flash/Silverlight/PlUpload engine. For 2. it could theoretically be Apache messing something up, but that is extremely unlikely since it passes the data 1:1 on to PHP. Now for 3. we can't rule out a bug in PHP, but that is too extremely unlikely since PHP is a constant here and the results vary with different browsers. But I can imagine PHP receives the file, saves it together with the second header, then the file locked because some service is filtering it, filtering takes long because the file is untrusted and big, PHP tries to remove the second header but is denied access because the filtering is still going on and at the end you are left with a file with header. The different outcomes with different browsers could be explained by different chunk sizes or simply the browser performance.

Unfortunately, that's all just speculation. Now since Microsoft did their very best to make it as hard as possible to downgrade IE, I'm currently unable to test it with IE9, all I can give you is some debugging instructions:

In your php.ini, set

enable_post_data_reading = Off

this will completely break all POST requests on that server, but it will allow you to read and dump the file upload requests.

In your upload.php, add those two lines before any other code:

file_put_contents('out.txt', print_r(getallheaders(), true).PHP_EOL.substr(file_get_contents('php://input'), 0, 1000), FILE_APPEND);
exit;

Start Apache and upload TryMe.exe with IE9. Next to your upload.php should now be an out.txt file containing all the relevant data about the file upload requests. Please upload that file somewhere and give us a link to it.

Mammary answered 25/1, 2015 at 17:57 Comment(9)

thanks for your help, you can download the output file sndesign.it/shared/stackoverflow/out.txt – Apospory 25/1, 2015 at 18:32

Well, this is the result with IE11, notice how the chunk and chunks sections are missing? Could you give me a dump when uploading a file that works with IE9? – Mammary 25/1, 2015 at 18:36

How small? Smaller than 2MB? or 8KB? – Mammary 25/1, 2015 at 18:47

fail if filesize is over upload_max_filesize or post_max_size – Apospory 25/1, 2015 at 19:27

But it happens with signed exes, unsigned exes and any other file type? – Mammary 26/1, 2015 at 0:52

I think I found the problem: $in = @fopen("php://input", "rb"). Why would you do that? If $_FILES is empty, the upload has obviously failed. Try returning an error instead, and see what then happens in IE9. – Mammary 26/1, 2015 at 15:7

First in your JavaScript, set chunk_size to the same as upload_max_file_size or post_max_size from the php.ini. If the browser is still not chunking the file, report this as a Plupload bug. – Mammary 26/1, 2015 at 19:1

IE9 not chunking the file... chunk_size: '1mb' in php error log PHP Warning: POST Content-Length of 23827605 bytes exceeds the limit of 5242880 in php.ini set upload_max_filesize=5M post_max_size=5M – Apospory 26/1, 2015 at 19:4

Then report that to the Plupload people. – Mammary 28/1, 2015 at 13:22

By default PHP max upload file size is set to 2MB.

Try updating your php settings (php.ini):

upload_max_filesize = 20M
post_max_size = 22M

More info: http://php.net/manual/en/ini.core.php#ini.upload-max-filesize and http://php.net/manual/en/ini.core.php#ini.post-max-size

Erethism answered 19/1, 2015 at 20:50 Comment(11)

Note that you need to restart your apache server before the settings take effect (if you haven't) – Erethism 19/1, 2015 at 21:1

Ah, I also noticed in your post that you haven't defined "chunk_size" when you instantiated plupload. – Erethism 19/1, 2015 at 21:5

Have you tried using a different large exe file (something safe from Microsoft maybe)? Would you also know if you have some background process (anti-virus, jobs, etc) that might affect this? I just want to get these out of the way... – Erethism 19/1, 2015 at 21:32

What I meant was to try a large Microsoft exe file, I think it is safe to assume that other file types would work except for exe files larger than 2MB – Erethism 19/1, 2015 at 21:44

Hmm, try another? Maybe larger around 5MB-10MB and up to the size of the file you're having problem with, then try larger. That way you can deduct other possibilities... – Erethism 19/1, 2015 at 21:55

Wondering what's the problem with the original exe file? Could it be running at the time when you were uploading it? Could it be that an anti-virus process is still alive and monitoring that file? – Erethism 19/1, 2015 at 22:3

Hmm, doesn't add up but glad it works for you now. Question now would be what technical/encoding/etc difference between a signed/unsigned exe file? Just wondering, are you signing using sn.exe? – Erethism 19/1, 2015 at 22:17

Ha! I'm not actually inclined to the signed/unsigned issue. I'm thinking there's something happening during the signing process that makes your exe file uploadable. – Erethism 19/1, 2015 at 22:24

It's because signed exe files are "from trusted developpers" it's a security issue – Minaret 25/1, 2015 at 11:13

Indeed if increase the values over the file size to upload, it works. However it should not work that way. I deleted my other comments because they were wrong . Only now I have found exactly the problem. – Apospory 25/1, 2015 at 19:31

This answer doesn't make sense at all. If increasing the value solved the problem, upload won't need to be in chunks. – Mailemailed 11/4, 2016 at 7:39

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Recommended topics

Hot tags