freeze/2 goals blocking on variables that have become unreachable
Asked Answered
T

1

6

I made the following little program to determine if the memory used for goals like freeze(X,Goal) is reclaimed when X becomes unreachable:

%:- use_module(library(freeze)). % Ciao Prolog needs this

freeze_many([],[]).
freeze_many([_|Xs],[V|Vs]) :-
   freeze(V,throw(error(uninstantiation_error(V),big_freeze_test/3))),
   freeze_many(Xs,Vs).

big_freeze_test(N0,N,Zs0) :-
   (  N0 > N
   -> true
   ;  freeze_many(Zs0,Zs1),
      N1 is N0+1,
      big_freeze_test(N1,N,Zs1)
   ).

Let's run the following query...

?- statistics, length(Zs,1000), big_freeze_test(1,500,Zs), statistics.

... with different Prolog processors and look at memory consumption. What a difference!

(AMD64) SICStus Prolog 4.3.2 : global stack in use = 108   MB
(AMD64) B-Prolog       8.1   : stack+heap   in use = 145   MB
(i386)  Ciao Prolog    1.14.2: global stack in use =  36   MB (~72 MB w/AMD64)
(AMD64) SWI-Prolog     7.3.1 : global stack in use =   0.5 MB
(AMD64) YAProlog       6.2.2 : global stack in use =  16   MB

When running more iterations with ?- length(Zs,1000), big_freeze_test(1,10000,Zs)., I made the following observations:

  • Ciao Prolog reports {ERROR: Memory allocation failed [in Realloc()]} before aborting.

  • and allocate more and more until the machine freezes.

  • performs all iterations in 3.554 seconds.
  • also performs all iterations, but takes 36.910 seconds.

Any ideas why it works with SWI-Prolog and YAProlog, but not with the other ones?

Considering runtime, how come SWI-Prolog beats YAProlog by more than an order of magnitude?

My intuition is leaning towards the interaction of "attributed variables" with "garbage collection". SWI-Prolog and YAProlog have (share?) a different attributed variable API and implementation than the other Prolog processors ... and, then again, it could be something completely different. Thank you!

Tokenism answered 28/6, 2015 at 0:49 Comment(5)
@j4nbur53. Thanks for pointing out the cycle! I meant unreachable from the roots. I agree that this "use" of freeze/2 is neither typical nor exemplary, but rather a desparate measure used as a last resort: a dirty hack which in some cases makes the implementation of the frozen goal a lot easier... it should probably better have no influence on the internal implementation or the design of the suspension mechanism.Tokenism
@j4nbur53: freeze/2 is really the goto of constraints. Too crude to be used reliably.Rotgut
@j4nbur53. Are you referring to empirical space-time measurements of "when((nonvar(X),nonvar(Y)),foo(X,Y)), and both X, Y unreachable.", running / using different Prolog processors?Tokenism
If call_residue_vars/2 is in place, it will still use ca 98 MB. See also here: github.com/SWI-Prolog/issues/issues/19#issuecomment-437183547Downdraft
The way of measuring is not good. You should call garbage_collect before the statistics. SWI-Prolog is quite aggressive in calling GC while other systems tend to prefer expanding the stacks. Stack expansion is often faster, but uses more memory which is particularly painful if you have many threads. Scalable threading is a focus point for SWI-Prolog.Liscomb
R
3

TL;DR: bug in SWI no more!

You are creating 500,000 frozen goals which are subsequently unreachable. What do these goals mean? Prolog systems do not analyze a goal as to its semantic relevance (prior to actually executing it). So we have to assume that the goals may be semantically relevant. As the goals are already disconnected, the only semantic effect they might have is to be false and thus making the next answer false.

So it is sufficient to consider freeze(_,false) instead.

Semantically, the predicates p/0 and q/0 are equivalent:

p :-
   false.

q :-
   freeze(_,false).

Procedurally, however, the first goal fails whereas the second succeeds. It is in such situations key to distinguish between solutions and answers. When Prolog succeeds, it produces an answer — most commonly this is known as an answer substitution in Prolog without constraints, where answer substitutions always contain one or infinitely many solutions1. In the presence of constraints or crude coroutining, an answer may now contain frozen goals or constraints that have to be taken into account to understand which solutions are actually described.

In the case above, the number of solutions is zero. When a system now garbage collects those frozen goals, it actually changes the meaning of the program.

In SICStus this is shown as follows:

| ?- q.
prolog:freeze(_A,user:false) ? ;
no

In SWI and YAP, those goals are not shown by default and thus chances are that bugs as this one have not been discovered.


PS: In the past, there has been a comparison between various systems concerning GC and constraints with SICStus being at that time the only one that passed all tests. In the meantime some systems improved.

I first looked at the SICStus numbers: 216 bytes per freeze! That's 27 words with 13 just for the term representing the goal. So simply 14 words for the freeze. Not so bad.

PPS: the frozen goal was throw/2, it should have been throw/1


Fine print 1: Some examples: An answer substitution X = 1 contains exactly one solution, and X = [_A] contains infinitely many solutions like X = [a] and many, many more. All this gets much more complex in the context of constraints.

Rotgut answered 28/6, 2015 at 13:6 Comment(7)
Thank you for the detailed answer! Regarding the semantics I am still on the fence... What is the semantics of the goal I stated freeze(V,throw(error(uninstantiation_error(V),big_freeze_test/3)))?Tokenism
@repeat: The point is that the goal is never examined prior to V being instantiated. Therefore is could be tantamount to false. I thought I addressed this above.Rotgut
I think I get the picture. The implementor must assume that the frozen goal can have an impact on the logical semantics. What I was looking for does not fit freeze/2. Should I work with attributed variables directly? Are attributed variables (and the auxiliary data) collected when they become unreachable?Tokenism
No, for building subsumes_term_t/3. These "hopefully never used" frozen goals help us stay logically sound. (I got it working with SICStus and yap. Yay! Leaking memory, though.)Tokenism
?? You can answer also your own questions!Rotgut
Referring to github.com/SWI-Prolog/issues/issues/19 ... does that basically mean that these variables remain referenced when using call_residue_vars/2, and they become unreachable otherwise?Tokenism
@repeat: Always assume that call_residue_vars/2 is present.Rotgut

© 2022 - 2024 — McMap. All rights reserved.