How can a Chrome extension save many files to a user-specified directory?

Question

I'm working on a Chrome extension to be used as an internal tool. Its required behavior is:

1. As a page action, enable an address bar icon when looking at certain intranet pages.
2. when the user clicks the icon, identify all files of a certain media type (say, .jpg) on the page, and
3. silently save them all to a directory on the user's local drive.

This question has been asked before, but the answer then was "use NPAPI", and NPAPI is now derelict.

So, what is the currently available way to achieve this? The ones I've looked at are:

• The chrome.FileSystem API --- but this does not save files in any user-accessible location. Instead the stored files are hidden behind obfuscated names in an undocumented directory. User requires that the files be stored under their original names in an accessible directory.
• The HTML5 download attribute, by creating a data: URL and programmatically clicking it. This pops up a "save as..." dialog for each file, which is unacceptable when there are a hundred assets on a single page. User requires that the files be downloaded without further interaction beyond the single icon click.
• The Chrome Download API, but that is only available in the beta and dev channels. User requires this extension work with mainstream Chrome.
• Use the Native Messaging API by creating a small .exe that simply saves a file to disk, and then pass the .jpg as a blob to it. This seems very cumbersome and I am not even sure how to reliably pass large blobs to EXEs like that.

Is there another approach I can try?

Asserted

For the curious, we ultimately solved this problem by abandoning Chrome and Web UIs in our internal tools altogether. – Asserted

Well that's one way of doing it .. :D – Fidel

@Crashworks, Instead of NPAPI, why not use PPAPI? – Oehsen

@Asserted - What did you move to? We are doing something similar and have many tools in chrome.... Straight up apps instead? – Oceanic

@EdWilliams Yes, we went back to building native C++ apps with Qt. – Asserted

Any update to the answers, I' curious? E.g. there's new developer.mozilla.org/en-US/docs/Web/API/FileSystem – Puissance

Answer 1

You've done quite a lot of research. Indeed, regular web pages cannot write to the user's filesystem without any plugins or extensions. Also, the HTML5 Filesystem API only provides access to a virtual filesystem, as you've observed.

However, you are confusing the chrome.fileSystem API with the HTML5 FileSystem API. Unlike the HTML FileSystem API, Chrome's fileSystem (app) API can directly write to the user's filesystem (e.g. ~/Documents or %USERPROFILE%\Documents), specified by the user.

This API is only available to Chrome apps, not extensions. This is not a problem, especially since you're developing an internal tool, because you can install the app and extension, and use message passing to communicate between the extension (page action) and app (file system access) (example).

---

About chrome.downloads: Since your extension is internal, you can probably force users to get on the beta/dev channel in order to use this API. The only limitation of this API is that the files will be saved in (a subdirectory of) the user-defined Downloads folder.

EDIT: The chrome.downloads API is now avaiable in all channels, including the stable branch (since Chrome 31).

Mcpeak

+1 Since the question mentioned Chrome Extension, I never thought about a Chrome App (inspite of how obvious it seems now) :) Good catch ! – Kynewulf

The problem with using the beta branch is that people expect it to have more bugs; therefore every time anyone in production encounters some strange Chrome behavior, they'll jump to the conclusion that it's due to using the beta branch, and the bug will land on my desk. – Asserted

@Asserted Despite its name, Beta is quite stable. I'm a daily user on the Beta channel, and have never encountered a problem. Chrome's developer cycle is typically something like this: Canary/dev -> beta -> stable. Canary is for the ones who like to get the latest features, bug fixes (and bugs). Beta is for the ones who enjoy cool features. Stable is for the ones who like the label "stable". There is almost no difference between stable and beta besides the different version string ("beta" vs no beta) and the disabled features. – Mcpeak

@RobW The problem is perception. People hear "beta", they think "bugs", and thus whenever they hit any kind of bug whatsoever, they will assume it's due to being in beta and come to me about it. – Asserted

Unless they look at the version number, they won't know that they're on the beta channel. By the way you describe your clients, I guess that they won't even notice when they're on the beta channel. You can always use the method I've explained at the top of my answer, but if you have a choice, try the downloads API. – Mcpeak

@RobW I would still have to tell them to install the beta channel; they need to do it at their end. Just today there was a bug in the Beta Chrome that broke every artist using one of our native extensions. Switching back to stable fixed it. – Asserted

@Asserted That's unfortunate. I have actually experienced the opposite: Because I'm on the beta channel, a patch against an always-reproducable crash bug was available to me before it went to the stable channel. Anyway, it's up to you whether you choose the stable channel (+app & extension) or the dev channel (extension, downloads API). – Mcpeak

@Asserted The chrome.downloads API is now available on Chrome stable (31+). – Mcpeak

@RobW The chrome.downloads API still will prompt the save file dialog to user!! – Sick

@AllanRuin Visit your settings page, click on Advanced settings and remove the tick at the "Ask where to save each file before downloading" checkbox. – Mcpeak

@AllanRuin Is there any way of using chrome.downloads silently? Ideally it wouldn't show up in the history or in the downloads bar, however having it show up in history isn't a deal breaker. – Burial

@mm865 See my comment from March, 10:24 – Mcpeak

Here is an updated link to the example github.com/GoogleChrome/chrome-app-samples/tree/master/samples/… – Endoparasite

Chrome is removing support for Chrome apps soon. Back to square one, damn. – Oehsen

Answer 2

I am afraid that you have done your homework, meaning you looked at all possible alternatives.

The best way to achieve exactly what you want, would be (as you mentioned) using a supporting native app and communicating through Native Messaging. BTW, since bandwidth is rarely a problem on intranets, you might find it simpler to pass the resources (e.g. images) URLs and have the app download and save them.
(Yes, it will be more cumbersome than simply developing an extension, but one's got to do what they've got to do, right ?)

On the other hand, if you are willing to sacrifice a little bit of user's experience over simplicity of development, I suggest combining the HTML5 goodies (that allow you to create and download a file locally) with a JS zipping library (e.g. JSZip), so the user only has to download a single zip file (and is only prompted once). BTW, if the user wishes, he/she can choose to always download the files without prompting (but you knew that already).

Kynewulf

Answer 3

Use the Native Messaging App idea.

The native app is cumbersome and a pain to write because documentation is poor, and unless you get the JSON formatting exactly correct on both ends you will not see anything in a console because stdin and stdout are taken over.

But, you will be happier when it is done because you can use standard tools (e.g., Windows Explorer, a hex editor, TeamViewer...) to view, move, and delete files, and otherwise see what is going on. Chrome's sand-boxed file system works, but seems to now be a dead-end (no other browsers have picked it up). No one is likely to develop third-party tools for it. Of course you probably don't need tools once everything is working, but until then, debugging is a nightmare because you need to write code (and quite a lot of code) just to track what files are in what directories, file versions, remaining disk space...

Bedelia

Please do not vandalize your posts. Once you've posted a question, you have licensed the content to the Stack Overflow community at large (under the CC-by-SA license). If you would like to disassociate this post from your account, see What is the proper route for a disassociation request?. – Hightower

Answer 4

Another solution for internal (or may be non-internal) usage is to connect to a websocket server, local or remote.

You can put it in both background.js or content.js (use wss:// for https://)

var ws = new WebSocket('ws://echo.websocket.org');
// var ws = new WebSocket('ws://127.0.0.1:9000');
ws.onmessage = function(res) {
    console.log('received data:', res.data);
};
ws.onopen = function() {
    ws.send('hello');
};

Nought

Would it be possible to pass file(s) from content to background page using WS? I'm searching for some modern way to upload files to the server, but all I can find are posts/threads dated around 2014, the time you could make cross-origin requests from content script environment. I'd appreciate any recommendations... – Susurration