Abort evaluating Haskell expression if memory limit reached

Question

I'm using QuickCheck to test automatically-generated properties (similar to QuickSpec) but one common problem I'm running into is exhausting the memory, either due to naive recursive generators or very large function outputs (e.g. one failure was caused by an exponential function for Peano numerals, which generated huge nested structures).

I'm wondering if there's a way to abandon evaluation if a (resident) memory limit is reached. It seems we can do this for timeouts, but memory seems to be trickier. This way, if we use too much memory, that test can be discarded (as if a ==> precondition had failed).

I can see how to measure the whole program's memory usage, by looking at the source of the weigh package. This would be workable, but it would be much nicer (and robust) to measure it for one particular expression (perhaps by getting the memory used by one thread, or something?).

For my purposes it would be enough to fully-normalise the expression, since I don't need to worry about recursive structures (I can apply this to the test results, which are effectively booleans).

Buyer

I'm not an expert (this sounds like a question for Simon Marlow or maybe Ben Gamari) but I'm not optimistic. Some of the necessary information can be captured by the profiler, but I don't know about being able to get the sort of access you need. Maybe. I think you're likely better off writing your QuickCheck generators more carefully. – Prevail

@Prevail I'm dealing with arbitrary, user-supplied code. The generators are auto-generated from the types and seem mostly fine; the main problem is functions with pathologically large output (or intermediate values), e.g. an exp function for Peano naturals. In most situations generating a Peano-encoded 10 would be reasonable; but exp 10 10 isn't good. I basically want a "sandbox" for user-supplied code, to prevent a few "bad" expressions from freezing/OOM-killing the whole program. (I ignore safety for now; if a user throws unsafePerformIO into QuickCheck that's their fault!) – Buyer

Of course, there's a GHC switch to limit the maximum possible heap size. (I believe it throws some kind of exception when the limit is reached, which you could conceivably catch. Unsure how easy it is to proceed once all available memory is exhausted...) – Bartonbartosch