Expr eval within Julia module gives UndefVarError

Question

I'm able to run the following code just fine and it provides the expected result:

julia> using DataFrames, DataFramesMeta

julia> expr = "2*:a+:b"
"2*:a+:b"

julia> df = DataFrame(a=[1,2],b=[3,4])
2×2 DataFrame
 Row │ a      b     
     │ Int64  Int64 
─────┼──────────────
   1 │     1      3
   2 │     2      4

julia> eval(Meta.parse("@transform(df, " * join(collect(":res" => expr), " = ") * ")"))
2×3 DataFrame
 Row │ a      b      res   
     │ Int64  Int64  Int64 
─────┼─────────────────────
   1 │     1      3      5
   2 │     2      4      8

However, this fails when done inside a module.

julia> module foo
           using DataFrames, DataFramesMeta
           function bar()
               expr = "2*:a+:b"
               df = DataFrame(a=[1,2],b=[3,4])
               eval(Meta.parse("@transform(df, " * join(collect(":res" => expr), " = ") * ")"))
           end
       end
Main.foo

julia> foo.bar()
ERROR: UndefVarError: df not defined
Stacktrace:
 [1] top-level scope
   @ ~/.julia/packages/DataFramesMeta/BkRtJ/src/macros.jl:1363
 [2] eval
   @ ./boot.jl:368 [inlined]
 [3] eval
   @ ./REPL[9]:1 [inlined]
 [4] bar()
   @ Main.foo ./REPL[9]:6
 [5] top-level scope
   @ REPL[10]:1

I imagine this is possible scope issue, and tried to be explicit by using foo.df instead of df in the function call, but without success.

Would anyone know how what is preventing df from being recognized as a defined variable here?

Edik

Answer 1

Like Bogumil said eval totally does not look like a recommended way to write your code.

However if your goal is learning macros this is the correct code:

module Foo2
    using DataFrames, DataFramesMeta
    function bar()
        expr = "2*:a+:b"
        df = DataFrame(a=[1,2],b=[3,4])
        code = quote
            @transform($df, :res = $(Meta.parse(expr)))
        end
        eval(code)
    end
end

Now you can run:

julia> Foo2.bar()
2×3 DataFrame
 Row │ a      b      res
     │ Int64  Int64  Int64
─────┼─────────────────────
   1 │     1      3      5
   2 │     2      4      8

Seminary

Answer 2

The reason is that eval is evaluated in global scope, as you can read in its docstring:

Evaluate an expression in the global scope of the containing module.

In general using Meta.parse and eval is highly not recommended. Can you please explain the problem you are trying to solve as there is probably another way to handle it.

Tramel

Thank you. I'm creating an application where the user inputs a formula to perform simple operations on data residing in a dataframe. The user (singular) is trusted to not try to inject code to be evaluated, which is the obvious danger. Otherwise yes you'd want to sanitize inputs. I used to have this code in python, where I was using a modified pyparsing/fourfn.py to evaluate simple arithmetic operations and apply to dataframe columns. Ideally I'd want to do the same in Julia (ie parsing a restricted grammar) but there doesnt seem to be a simple and mature package for that. Meta does the job. – Edik

Then you can consider generating a string with the whole definition of the function and doing Meta.parse and eval on it. I think it would be simplest (apart from Przemysław proposed). – Attainment

Thanks for the suggestion. This seems more complex than say Przemysław's solution, and doesn't alleviate the injection concern. I will probably go with Przemysław's solution until I can find a way to parse/eval a restricted and simple grammar. – Edik