Extending simple ANTLR grammar to support input variables

Question

I'm still on my quest for a really simple language and I know now that there are none. So I'm writing one myself using ANTLR3.

I found a really great example in this answer:

Exp.g:

grammar Exp;

eval returns [double value]
    :    exp=additionExp {$value = $exp.value;}
    ;

additionExp returns [double value]
    :    m1=multiplyExp       {$value =  $m1.value;} 
         ( '+' m2=multiplyExp {$value += $m2.value;} 
         | '-' m2=multiplyExp {$value -= $m2.value;}
         )* 
    ;

multiplyExp returns [double value]
    :    a1=atomExp       {$value =  $a1.value;}
         ( '*' a2=atomExp {$value *= $a2.value;} 
         | '/' a2=atomExp {$value /= $a2.value;}
         )* 
    ;

atomExp returns [double value]
    :    n=Number                {$value = Double.parseDouble($n.text);}
    |    '(' exp=additionExp ')' {$value = $exp.value;}
    ;

Number
    :    ('0'..'9')+ ('.' ('0'..'9')+)?
    ;

WS  
    :   (' ' | '\t' | '\r'| '\n') {$channel=HIDDEN;}
    ;

Java Code:

public Double evaluate(String string, Map<String, Double> input) throws RecognitionException {
    ANTLRStringStream in = new ANTLRStringStream(string);
    ExpLexer lexer = new ExpLexer(in);
    CommonTokenStream tokens = new CommonTokenStream(lexer);
    return new ExpParser(tokens).eval();
}

Using this ANTLR grammer I can evaluate expressions like

(12+14)/2

and get 13 as a result.

Now the only thing missing for my use-case is a way to inject simple double variables into this, so that I can evaluate the following by supplying {"A": 12.0, "B":14.0} as the input map:

(A+B)/2

Any ideas?

Boswell

Answer 1

You could create a Map<String, Double> memory in your parser and introduce a Identifier in your grammar:

Identifier
  :  ('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z' | '_' | '0'..'9')*
  ;

Then your atomExp parser rule would look like this:

atomExp returns [double value]
    :    n=Number                {$value = Double.parseDouble($n.text);}
    |    i=Identifier            {$value = memory.get($i.text);} // <- added!
    |    '(' exp=additionExp ')' {$value = $exp.value;}
    ;

Here's a small (complete) demo:

grammar Exp;

@parser::members {

  private java.util.HashMap<String, Double> memory = new java.util.HashMap<String, Double>();

  public static Double eval(String expression) throws Exception {
    return eval(expression, new java.util.HashMap<String, Double>()); 
  }

  public static Double eval(String expression, java.util.Map<String, Double> vars) throws Exception {
    ANTLRStringStream in = new ANTLRStringStream(expression);
    ExpLexer lexer = new ExpLexer(in);
    CommonTokenStream tokens = new CommonTokenStream(lexer);
    ExpParser parser = new ExpParser(tokens);
    parser.memory.putAll(vars);
    return parser.parse(); 
  }
}

parse returns [double value]
    :    exp=additionExp {$value = $exp.value;}
    ;

additionExp returns [double value]
    :    m1=multiplyExp      {$value =  $m1.value;} 
        ( '+' m2=multiplyExp {$value += $m2.value;} 
        | '-' m2=multiplyExp {$value -= $m2.value;}
        )*  
    ;

multiplyExp returns [double value]
    :   a1=atomExp       {$value =  $a1.value;}
        ( '*' a2=atomExp {$value *= $a2.value;} 
        | '/' a2=atomExp {$value /= $a2.value;}
        )*  
    ;

atomExp returns [double value]
    :    n=Number                {$value = Double.parseDouble($n.text);}
    |    i=Identifier            {$value = memory.get($i.text);}
    |    '(' exp=additionExp ')' {$value = $exp.value;}
    ;

Identifier
    :    ('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z' | '_' | '0'..'9')*
    ;

Number
    :    ('0'..'9')+ ('.' ('0'..'9')+)?
    ;

WS  
    :   (' ' | '\t' | '\r'| '\n') {$channel=HIDDEN;}
    ;

And now theres no need to instantiate the parser/lexer yourself, you can simply do:

import org.antlr.runtime.*;
import java.util.*;

public class ANTLRDemo {
    public static void main(String[] args) throws Exception {
        Map<String, Double> vars = new HashMap<String, Double>();
        vars.put("two", 2.0);
        vars.put("pi", Math.PI);
        System.out.println(ExpParser.eval("two * pi", vars));
    }
}

which will produce:

6.283185307179586

Good luck!

Participate

Answer 2

Bah, took the time to implement this so might as well post it, even though I was beat to the punch :)

In the grammar below I have implemented the format of variable assigning you were looking to do.

grammar Exp;



eval returns [double value]
scope
{
    java.util.Hashtable varMap; 
}
@init
{
    $eval::varMap = new java.util.Hashtable();
}
:    exp=additionExp {$value = $exp.value;}
    | varList
;

additionExp returns [double value]
    :    m1=multiplyExp       {$value =  $m1.value;} 
         ( '+' m2=multiplyExp {$value += $m2.value;} 
         | '-' m2=multiplyExp {$value -= $m2.value;}
         )* 
    ;

multiplyExp returns [double value]
    :    a1=atomExp       {$value =  $a1.value;}
         ( '*' a2=atomExp {$value *= $a2.value;} 
         | '/' a2=atomExp {$value /= $a2.value;}
         )* 
    ;

atomExp returns [double value]
    :    n=Number                {$value = Double.parseDouble($n.text);}
    |    v=ID            {$value = $eval::varMap.get($v);}
    |    '(' exp=additionExp ')' {$value = $exp.value;}
    ;

varList
    :   
    OPEN_BRACE assignVar (COMMA assignVar)+ CLOSE_BRACE
    ;

assignVar
    :QUOTE var=ID n=Number QUOTE COLON { $eval::varMap.put($var, $n); }
    ;


Number
    :    ('0'..'9')+ ('.' ('0'..'9')+)?
    ;

WS  
    :   (' ' | '\t' | '\r'| '\n') {$channel=HIDDEN;}
    ;



fragment LETTER: LOWER | UPPER;
fragment LOWER: 'a'..'z';
fragment UPPER: 'A'..'Z';

OPEN_BRACE
    :   '{'
    ;

CLOSE_BRACE
    :   '}'
    ;

COLON   : ';';
COMMA   :   ',';

QUOTE   :   '"';

ID
: LETTER*;

Knowles

Sorry about that Darien :). Well, I wrote the grammar arturh posted in the first place, so I had a bit of a head-start. +1 from me nevertheless! – Participate

No worries, it's part of the perils of contributing to SO :) – Knowles