Prolog DCG set_prolog_flag double_quotes source code directive location matters; documentation?

Question

I learned the hard way that with SWI-Prolog the location for the Prolog directive set_prolog_flag matters in a source code file.

The only documentation I found of value about loading source code files with directives was in Loading Prolog source files

A directive is an instruction to the compiler. Directives are used to set (predicate) properties (see section 4.15), set flags (see set_prolog_flag/2) and load files (this section). Directives are terms of the form :- <term>.

Is there documentation for SWI-Prolog that covers loading of source code that notes if a directive applies to the entire file or depends on the location in the source code file?

Or is it that all lines loaded from a source code file are just a simple playing of statements into the top-level and location always matters?

Supplement / TL;DR

Default

When using Definitive Clause Grammars (DCG) in Prolog it is known that DCG requires the input to be a list of character codes, e.g.

?- string_codes("abc123",Cs).
Cs = [97, 98, 99, 49, 50, 51].

and with the following DCG rule in a source code file and loaded into the top-level

digit(0) --> "0".

the DCG can be used with

?- string_codes("0",Cs),phrase(digit(D),Cs,R).
Cs = [48],
D = 0,
R = []

set_prolog_flag

Now to make it easier to use DCG instead of having to use string_codes the Prolog directive

:- set_prolog_flag(double_quotes, chars).

can be used in a source code file and with the following DCG rule in a source code file and loaded into the top-level

digit(0) --> "0".

the DCG can be used with

?- phrase(digit(D),"0",R).
D = 0,
R = [].

That left out something important

It turns out that the if set_prolog_flag appears before the DCG rule then skipping string_codes works, but if set_prolog_flag appears after the DCG rule then skipping string_codes fails.

:- set_prolog_flag(double_quotes, chars).
digit(0) --> "0".

?- phrase(digit(D),"0",R).
D = 0,
R = [].

vs

digit(0) --> "0".
:- set_prolog_flag(double_quotes, chars).

?- phrase(digit(D),"0",R).
false.

The reasoning that led me afoul

While I am aware that a lot of programming with Prolog can be done in just the top-level, I tend to rely on source code files and consult/1.
In writing lots of code I started to use modules. With modules I found out that the Prolog flags are independent for each module.

?- current_prolog_flag(double_quotes,V).
V = string.

?- current_prolog_flag(symbolic:double_quotes,V).
V = string.

?- set_prolog_flag(symbolic:double_quotes,chars).
true.

?- current_prolog_flag(double_quotes,V).
V = string.

?- current_prolog_flag(symbolic:double_quotes,V).
V = chars.

and that the default top-level module is user

?- current_prolog_flag(double_quotes,V).
V = string.

?- current_prolog_flag(user:double_quotes,V).
V = string.

?- set_prolog_flag(double_quotes,chars).
true.

?- current_prolog_flag(double_quotes,V).
V = chars.

?- current_prolog_flag(user:double_quotes,V).
V = chars.

?- set_prolog_flag(user:double_quotes,codes).
true.

?- current_prolog_flag(double_quotes,V).
V = codes.

?- current_prolog_flag(user:double_quotes,V).
V = codes.

which lulled me into the false belief that the Prolog directive set_prlog_flag applied to the entire module no matter where it was written.

What broke the mold

In writing lots of example code it was easier to keep all of the little examples in one file and associated with each little example was set_prolog_flag. For an identifier example it needed two little example DCG rules, one for digit and one for letters. The digit rules were above the letter rules and working, but the letter rules had the set_prolog_flag directive because I was working on them at the time. Remember I am thinking that the directive applies to the whole file at this point. Then in testing ident the DCG rules for letters were working but the DCG rules for digits were failing.

digit(0) --> "0", !.
digit(1) --> "1", !.
digit(2) --> "2", !.

:- set_prolog_flag(double_quotes, chars).

ident(Id) --> letter(C), identr(Cs), { name(Id, [C|Cs]) }.

identr([C|Cs]) --> letter(C), !, identr(Cs).
identr([C|Cs]) --> digit(C), !, identr(Cs).
identr([])     --> [].

letter(a) --> "a", !.
letter(b) --> "b", !.
letter(c) --> "c", !.

?- phrase(ident(Id),"ab12",R).
Id = ab,
R = ['1', '2'].

Root cause

So using listing/1

?- listing(digit).
digit(0, [48|B], A) :- !,
        A=B.
digit(1, [49|B], A) :- !,
        A=B.
digit(2, [50|B], A) :- !,
        A=B.

?- listing(ident).
ident(C, A, F) :-
        letter(D, A, B),
        identr(E, B, G),
        name(C, [D|E]),
        F=G.

?- listing(identr).
identr([A|D], B, F) :-
        letter(A, B, C), !,
        E=C,
        identr(D, E, F).
identr([A|D], B, F) :-
        digit(A, B, C), !,
        E=C,
        identr(D, E, F).
identr([], A, A).

?- listing(letter).
letter(a, [a|B], A) :- !,
        A=B.
letter(b, [b|B], A) :- !,
        A=B.
letter(c, [c|B], A) :- !,
        A=B.

the problem was apparent

digit(0, [48|B], A) :- !,
    A=B.

letter(a, [a|B], A) :- !,
        A=B.

that digit was converted to use character codes 48 and letter was converted to use characters a. That's when I asked myself if the location of set_prolog_flag in source mattered.

Confirming root cause

To test this I created a little source code file

digit_before(0) --> "0".

:- set_prolog_flag(double_quotes, chars).

digit_after(0) --> "0".

and in top-level

?- current_prolog_flag(double_quotes,V).
V = string.

?- current_prolog_flag(symbolic:double_quotes,V).
V = string.

?- consult("C:/Users/Eric/Documents/Projects/Calculus Project/test.pl").
true.

?- current_prolog_flag(double_quotes,V).
V = chars.

?- current_prolog_flag(symbolic:double_quotes,V).
V = string.

?- listing(digit_before).
digit_before(0, [48|A], A).

true.

?- listing(digit_after).
digit_after(0, ['0'|A], A).

true

which confirmed that the Prolog directive set_prolog_flag does not apply to an entire file. Notice that digit_before is converted to 48 and digit_after is converted to '0'.

Notes

Note: The directive set_prolog_flag(F,V) can also be used in the top-level and does not require the preceding :- .

Note: The example used :- set_prolog_flag(double_quotes, chars). but :- set_prolog_flag(double_quotes, codes). also works. Using chars value is preferred because it makes the values easier to read when debugging, etc.

Fearful

Of interest: Command line option --traditional - double quoted text is represented by a list of character codes. – Fearful

You can always write letter(a) in place of letter('a'). No need for quoting here, not even in SWI. – Dakar

The SO-format is very well suited for many questions in particular because answering them is so nicely possible. But indeed it demands a lot from the asker. I really found this question much too lengthy, you really should put more effort in making them more precise. – Dakar

@Dakar That is why I put most of it into the supplement section. I don't expect people to read that section if they don't find the question interesting. I use to use TL;DR instead of supplement, but that was not always befitting. – Fearful

For the record: You have here identified a weakness of the standard, too. Thank you! And please be patient until this is fixed :-). – Dakar

fyi I'll try to contact with swi-proog people as well as logtalk one's – Avion

@AntonDanilov Why the comment? – Fearful

@Guy Coder just to inform you that the work is in progress – Avion

@GuyCoder I said: Here is the ref to s/o discussion. if you want to say something about that I'm ready to talk. Though this is about swi-prolog nevertheless it could be relevant for logtalk too. ... So, Paulo,do you have any interest to the problem discovered in StackOverflow by Guy Coder? Paulo Moura: Busy with work. Will look into it later. Thanks for the link. – Avion

Answer 1

In SWI-Prolog, directives and clauses are processed in order. Prolog flags are complicated. The overall rule is that they are thread scoped, where child threads share the flags from their creator using copy-on-write semantics, which effectively means the same as when all flags would be copied except for performance and memory usage. But then, some flags are scoped to the source file in which they appear. This means that load_files/2 saves the state of the flag before the load and restores it afterwards. Some other flags are module scoped, which means the flag API is merely a proxy to changing a module attribute. Such flags are not thread-specific because modules are global. Also note that some flags affect reading (e.g., double_quotes), while others affect the compiler (optimise) and most affect runtime behaviour.

Ideally, the documentation with current_prolog_flag/2 should document these aspects. Not sure this documentation is accurate. For double_quotes it says maintained for each module.

Sanctimonious

Thanks. I appreciate your response here. Since false noted You have here identified a weakness of the standard, too. Thank you! And please be patient until this is fixed :-). and Anton Danilov noted fyi I'll try to contact with swi-prolog people as well as logtalk one's and as I don't know all of the details of what either is working on, I will wait before accepting an answer. – Fearful

For those unaware of who Jan Wielemaker is, just take a look at the SWI-Prolog repository on GitHub and look to see who does most of the commits. – Fearful

Answer 2

I may say that you can make for sure that set_prolog_flag(double_quotes, chars) directive has the desired behavior (applicability to an entire file).

This can be done by using initialization/2. directive with the option after_load, or by using initialization/1.

digit_before(0) --> "0".

:- initialization( set_prolog_flag(double_quotes, chars),  after_load ).

digit_after(0) --> "0".

SWI-Prolog initializаtion/2 directive

SWI-Prolog initializаtion/1 directive

Regarding the problem how to suggest your ideas to the SWI-Prolog community I hope the (initial) solution is the presence of the second answer.

Useful links:

Research papers by Ulrich Neumerkel and Fred Mesnard

Home Page of Markus Triska

Contains a large number of diverse materials dedicated to the programming language Prolog.

Avion

I appreciate your answer but it is just pointing me back to the same pages I referenced already, e.g. Loading Prolog source files. I don't see how if someone else reads those pages it will keep them from making the same mistake I made. – Fearful

@Guy Coder You may want to suggest your ideas to swi-prolog community. Also, I slightly confused - looks like you appreciate my answer ,but don't agree. I thought I gave the way not to worry about the directives execution order. it's quite untypical to put the declarations all over the file imho – Avion

It is not that I agree or don't agree with your answer, I don't understand the parts that reference initialization. Also I found the part With modules I found out that the Prolog flags are independent for each module. odd as it was not a question and yet you responded to it like a question. – Fearful

You are right on the mark by saying You may want to suggest your ideas to swi-prolog community. Part of the reason I asked the question was to see if this was documented somewhere that I was not aware. If no one pointed out any documentation in a few weeks then I would suggest it to the SWI-Prolog site. – Fearful

If in a few weeks if no one can identify any posted documentation that explains the problem I encountered and how to avoid it, even for someone new to Prolog, and if you add to your answer that it is not documented anywhere other than this question and also add the idea of your suggestion You may want to suggest your ideas to swi-prolog community. I would find that an acceptable answer. – Fearful

I know the SWI-Prolog site is the primary place for SWI-Prolog documentation, but also SO Q&A are sometimes relevant and when I found the Home Page of Markus Triska or research papers by Ulrich Neumerkel and Fred Mesnard e.g. Localizing and explaining reasons for non-terminating logic programs with failure-slices I know there is more out there but you have to look for it or be pointed to it. – Fearful

@GuyCoder I said to Paulo Moura: Here is the ref to s/o discussion. if you want to say something about that I'm ready to talk. Though this is about swi-prolog nevertheless it could be relevant for logtalk too. ... So, Paulo, do you have some interest to the problem discovered in StackOverflow by Guy Coder? Paulo Moura: Busy with work. Will look into it later. Thanks for the link. – Avion

@GuyCoder I've searched periodically for a third-paty docs about SWI-Prolog. IIRC, I found only docs related to XPCE and other little stuff. – Avion

Thanks for the effort. No sense in working on this for a while. I suspect that either Jan or false will have something related to a weakness of the standard in a few months or more, so as false said, please be patient until this is fixed. – Fearful