Limit memory used by cabal install?

Question

I'm limited by 1GB memory on my server hosting. When I want to compile some big program like git-annex, Cabal eats lot of memory. Is there a way to limit cabal or gcl using some option to limit memory usage?

I updated question with some details: I'm running cabal (1.22.4.0) and Ghc 7.10.2 on the Webfaction hosting (CentOS 7 - 64bit), with access to shell (non root access). Webfaction admin/robot generally tolerate some burst due to compilation. But Ghc / Cabal need too much memory and spend too many minutes for certain package compilation. So system automatically kills all processes.

I need to relaunch the compilation many times to finally obtain a successful result.

My primary objective is that compilation ends with success, however long it takes.

Mattoid

What OS are you using? Have you enabled any swap space? – Standing

I think most try to compile on their own machine and ship the binaries - maybe this is an option? – Chane

If everything else fails, I would create a VM locally on my own PC with the same(-ish) OS as in the server, compile everything locally, and upload everything to the server. Since GHC statically links most things, uploading locally compiled binaries might even work without needing a custom local VM. – Bitstock

unix.stackexchange.com/questions/44985/… has tons of suggestions on how to limit memory usage of a single process though. Perhaps, you can take a look at this as see if it helps. – Sagamore

Answer 1

@reyman64 suggestion confirmed to work on Xubuntu 18.04 with 2GB ram. Thanks! So, cabal install -v cabal-install hangs the system. The following works fine:

cabal install -v --ghc-options='+RTS -M1G -RTS' cabal-install

Muliebrity

Answer 2

EDIT: the post was unclear about the true issue, so ignore this answer as it was written with something else in mind, but has some useful comments.

I think if you externally limit Cabal's or GHC's memory consumption to 1GB, you will start getting a lot of failed builds due to early terminations by a Linux memory guard (or whatever it's called — somebody, please?). This might not be the case if you find a way to convince Cabal itself to use less memory, and convince GHC to go on a diet, too.

I ran into this just recently while building Curry KiCS2 on a 1GB virtual machine with just 1GB of swap space. (Of course my goal was not to limit memory consumption but instead get the damn thing to finish building, so all I did was increase VM memory to 4GB and swap to 8GB, but that's clearly not what you're after.)

So I think there is probably no workaround to GHC consuming a lot of memory during compilation. Just make sure parallel compilation is turned off though. And research means to reduce GHC's memory consumption, by compilation flags (of GHC itself, possibly), or turn off optimization phases that might consume a lot of memory, etc.

A brighter mind could prove me wrong though.

Mannerism

Ok, in fact when i say that my hosting provide limit memory, this is because memory guard kill any process which eat more during a certain amount of time.. VM is not possible because i'm not root. Only ways is flag for ghc, but I dont know it. Any idea on this point? – Mattoid

Nope, is it a particular .hs file that is blowing GHC up? If you compile it separately? – Mannerism

Oh, you're saying you can exceed the limit but only during short bursts and not for longer than n seconds? Then you could just suspend and resume the compilation process periodically when it hits a certain threshold. But that requires some non-trivial but not impossible scripting. If you can determine that to be the case, you might post this as another question or modify this one. – Mannerism

Yeah this is exactly what happen :-/ I see an option on the net, but i don't understand if i can use in my case, what do you think of this : cabal install ... --ghc-options='+RTS -M400m' ? I found some other information here, but it's really hard to understand : downloads.haskell.org/~ghc/7.4.1/docs/html/users_guide/… – Mattoid

RTS I think refers to the runtime. – Mannerism

@Mattoid If you do try to use +RTS, make sure to include an -RTS so that it can be interspersed with other options, as in --ghc-options='+RTS -M400m -RTS'. I don't know off-hand whether that will actually do what you want -- e.g. it depends on GHC being built with the RTS turned on -- but it sounds promising. – Latterday