Finding the Pharo documentation for the compile & evaluate methods, etc. in the compiler class

Question

I've got an embarrassingly simple question here. I'm a smalltalk newbie (I attempt to dabble with it every 5 years or so), and I've got Pharo 6.1 running. How do I go about finding the official standard library documentation? Especially for the compiler class? Things like the compile and evaluate methods? I don't see how to perform a search with the Help Browser, and the method comments in the compiler class are fairly terse and cryptic. I also don't see an obvious link to the standard library API documentation at: http://pharo.org/documentation. The books "Pharo by Example" and "Deep into Pharo" don't appear to cover that class either. I imagine the class is probably similar for Squeak and other smalltalks, so a link to their documentation for the compiler class could be helpful as well.

Thanks!

Monmouthshire

It’s weird that a newbie would want to know how the compiler works, but question would be more apt to be answered if you asked on the Pharo developers list (forum.world.st/Pharo-Smalltalk-Developers-f1294837.html is a web interface to it) – Mutazilite

Smalltalk newbie, not a programming newbie. There is a difference between "how the compiler works" and "how to use the compiler". FWIW, I'm interested in evaluating strings. Maybe I'm looking in the wrong class? I'm trying to find the moral equivalent of 'eval' found in other languages (en.wikipedia.org/wiki/Eval#Smalltalk). I suppose there is a small chance that there is no general smalltalk API documentation publicly available, since it is all pay-walled by the ANSI standard? docs.python.org/3/library/index.html ruby-doc.org/stdlib-2.5.0 – Monmouthshire

Answer 1

There are several classes that collaborate in the compilation of a method (or expression) and, given your interest in the subject, I'm tempted to stimulate you even further in their study and understanding.

Generally speaking, the main classes are the Scanner, the Parser, the Compiler and the Encoder. Depending on the dialect these may have slightly different names and implementations but the central idea remains the same.

The Scanner parses the stream of characters of the source code and produces a stream of tokens. These tokens are then parsed by the Parser, which transforms them into the nodes of the AST (Abstract Syntax Tree). Then the Compiler visits the nodes of the AST to analyze them semantically. Here all variable nodes are classified: method arguments, method temporaries, shared, block arguments, block temporaries, etc. It is during this analysis where all variables get bound in their corresponding scope. At this point the AST is no longer "abstract" as it has been annotated with binding information. Finally, the nodes are revisited to generate the literal frame and bytecodes of the compiled method.

Of course, there are lots of things I'm omitting from this summary (pragmas, block closures, etc.) but with these basic ideas in mind you should now be ready to debug a very simple example. For instance, start with

Object compile: 'm ^3'

to internalize the process.

After some stepping into and over, you will reach the first interesting piece of code which is the method OpalCompiler >> #compile. If we remove the error handling blocks this methods speaks for itself:

compile
  | cm |
  ast := self parse.
  self doSemanticAnalysis. 
  self callPlugins.  
  cm := ast generate: self compilationContext compiledMethodTrailer 
  ^cm

First we have the #parse message where the parse nodes are created. Then we have the semantic analysis I mentioned above and finally #generate: produces the encoding. You should debug each of these methods to understand the compilation process in depth. Given that you are dealing with a tree be prepared to navigate thru a lot of visitors.

Once you become familiar with the main ideas you may want to try more elaborated -yet simple- examples to see other objects entering the scene.

Fagen

Thanks for your long and detailed response. My apologies for not being clear in my original question. I'm looking for the documentation describing the use of the compile class methods, so I can learn to create and appropriately use things like the "aContext" argument (for setting up the lexical environment) of the #evaluate:in: method (among other things). Documentation similar in spirit to what you find for other programming languages: docs.racket-lang.org/guide/eval.html Thanks! – Monmouthshire

Answer 2

Here are some simple facts:

1. Evaluation in Smalltalk is available everywhere: in workspaces, in the Transcript, in Browsers, inspectors, the debugger, etc. Basically, if you are allowed to edit text, most likely you will also be allowed to evaluate it.

2. There are 4 evaluation commands

   • Do it (evaluates without showing the answer)
   • Print it (evaluates and prints the answer next to the expression)
   • Inspect it (evaluates and opens an inspector on the result)
   • Debug it (opens a debugger so you can evaluate your expression step by step).

3. Your expression can contain any literal (numbers, arrays, strings, characters, etc.)

   17 "valid expression"
   

4. Your expression can contain any message.

   3 + 4.
   'Hello world' size.
   1 bitShift: 28
   

5. Your expression can use any Global variable

   Object new.
   Smalltalk compiler
   

6. Your expression can reference self, super, true, nil, false.

   SharedRandom globalGenerator next < 0.2 ifTrue: [nil] ifFalse: [self]
   

7. Your expression can use any variables declared in the context of the pane where you are writing. For example:

   • If you are writing in a class browser, self will be bound to the current class
   • If you are writing in an inspector, self is bound to the object under inspection. You can also use its instances variables in the expression.
   • If you are in the debugger, your expression can reference self, the instance variables, message arguments, temporaries, etc.

8. Finally, if you are in a workspace (a.k.a. Playground), you can use any temporaries there, which will be automatically created and remembered, without you having to declare them.

Fagen

Answer 3

As near as I can tell, there is no API documentation for the Pharo standard library, like you find with other programming languages. This seems to be confirmed on the Pharo User's mailing list: http://forum.world.st/Essential-Documentation-td4916861.html

...there is a draft version of the ANSI standard available: http://wiki.squeak.org/squeak/uploads/172/standard_v1_9-indexed.pdf

...but that doesn't seem to cover the compiler class.

Monmouthshire