Can I buy remote MATLAB processing time? [closed]

Question

I'm a regular user of Matlab. For a very CPU-intensive task, I need to keep my PC running for hours (or days) simulating stuff, which is a pain since I need to carry it around.

Do you know of a public website/company that allows a user to run a remote instance for as long as he wants to, and then charge the corresponding fee? I don't care where it is done, or how many cores are used or anything, I just want to run some .m files and then retrieve the output variables.

I don't think it is too difficult to implement but I just cannot find (I've googled a lot) someone to sell me this service. I know some universities have servers for this, but they are private.

Any help on this will be much appreciated.

Thanks, Germán

Serum

How about cloud computing.. Here's one paper on how to configure MATLAB on Amazon EC2: mathworks.com/programs/techkits/ec2_paper.html, though I have no experience in this – Celio

Yes, cloud computing is a very wide subject, and certainly applies in this case (thanks for the paper). However, I want to avoid the burden of having to set up a server (virtual or real), install Matlab (which means buying a license) , etc, because I won't run these simulations too often. – Serum

I'm a bit late to this question, but following on from @Celio 's comment - MathWorks themselves offer the service requested by the OP. You can install MATLAB Distributed Computing Server on Amazon EC2, and pay MathWorks for it on an hourly basis rather than by buying a permanent license. This pricing option is widely unadvertised, and you'd need to speak to your account manager to discuss it on an individual basis. Note that unless you need a pretty big cluster, for very intensive tasks, and fairly infrequently, it's not likely to be cheaper than buying a permanent license. – Andras

@SamRoberts If I understand correctly, that solution still requires the user to run MATLAB locally: "You develop your program or model on a multicore desktop computer using Parallel Computing Toolbox™ and then scale up to many computers by running it on MATLAB Distributed Computing Server." mathworks.com/products/distriben – Serum

@Serum That's right. The model is always that the user runs MATLAB locally. With PCT, computations can then be offloaded onto worker processes running on the local machine; with MDCE, those worker processes can run on a remote cluster, or in the cloud as described in my previous comment. But computations are always kicked off from a local MATLAB. – Andras

Answer 1

You can try to get an account with MATLAB on the Teragrid, operated jointly by Cornell university and Purdue university. It is an NSF funded project and provides access to researchers (both academic and industrial) for free. You just need to fill out a request form and you should get an account setup in a day or two.

I have used the Teragrid for my projects and I am entirely satisfied with it. They have a good infrastructure with 64 servers w/ 8 cores each, for a total of 512 cores of processing power and 16 GB of RAM/server.

However, do note that you will have to change your workflow drastically and switch to writing distributed jobs instead of parallel (or serial jobs), which can be a pain in the initial days (if you're not used to it). They do have helpful resources for that.

Also important when working with shared resources is the concept of "wall time", which also directly relates to how long your job stays in the queue. Wall time is basically how long a single task takes to run. In order to set a sufficient wall time, you must be intricately familiar with your code's complexity (number of operations, etc) so that you don't under estimate the time required (tasks are terminated if they exceed wall time!). On the other hand, you don't want to be too lax and choose a high wall time, because then your job will stay longer in the queue. In other words, you can't just set a wall time of 7 days just to be safe, because the scheduler will make you wait till a resource can be freed for 7 days (once you access a core, it's yours and yours only), which is close to never.

Iand

Thanks for the reference, it looks promising. However, it seems to be limited to research in universities or the sort. They ask for (a) class/research project, (b) project faculty advisor, (c) Matlab code description. I will give it a try. – Serum

Its not just for universities. The project name is just a name that you give it so that they can identify it internally; need not necessarily be the "official name" under which you got a grant or something. If you don't have one, make one up :) You don't have to enter a project advisor if you don't have one and the MATLAB code is just a brief description of the code... for e.g., "The computationally expensive part of the code does Cholesky decompositions on 2000x2000 matrices in a loop". That is descriptive enough. – Iand

Sadly, this project no longer has free access, although it lets external institutions use the "Red Cloud". The Academic Subscription (including MATLAB) for academic institutions other than Cornell University is US$1200 and includes 8585 core hours plus 50 GB of storage (with more available). Quite expensive but at least it is there should someone need it. cac.cornell.edu/RedCloud/start.aspx – Serum

That's a bummer... I did find it useful for getting some heavy stuff done (sadly, that project of mine never got anywhere) – Iand

Answer 2

As I mentioned in the comment, cloud computing seems to be what you are looking for.

In addition to the paper I already linked to, I came by another solution that doesn't involve buying additional licenses for installing MATLAB on the cloud.

Instead it compiles your program into a standalone executable (this requires the MATLAB Compiler on your existing platform). This executable along with the freely available MCR runtime are the only things transferred to the cloud.

To be exact, the author is using sockets to build a server that sits in the cloud listening for connections. This is the part that is compiled into an executable, with only the MCR as dependency. It takes the input, EVAL-uates it, and sends back the results when done.

The client side (MATLAB script on your side), connects to the server, and sends it commands to be run along with serialized data needed. This is inherently scalable as you can run servers on as many instances as you want (as many as you can afford really), and simply dividing the work among them. Obviously it requires a little bit of work to rewrite your program into breakable tasks.

I think this is really what the MATLAB Distributed Computing solution is doing but with all the gory details abstracted away... Check it out to see if it fits your needs: Using Amazon EC2 to speed up matlab optimisation

Celio