How do I reload a module in an active Julia session after an edit?
Asked Answered
M

6

69

2018 Update: Be sure to check all the responses, as the answer to this question has changed multiple times over the years. At the time of this update, the Revise.jl answer is probably the best solution.

I have a file "/SomeAbsolutePath/ctbTestModule.jl", the contents of which are:

module ctbTestModule
export f1
f1(x) = x + 1
end

I fire up Julia in a terminal, which runs "~/.juliarc.jl". The startup code includes the line:

push!(LOAD_PATH, "/SomeAbsolutePath/")

Hence I can immediately type into the Julia console:

using ctbTestModule

to load my module. As expected f1(1) returns 2. Now I suddenly decide I want to edit f1. I open up "/SomeAbsolutePath/ctbTestModule.jl" in an editor, and change the contents to:

module ctbTestModule
export f1
f1(x) = x + 2
end

I now try to reload the module in my active Julia session. I try

using ctbTestModule

but f1(1) still returns 2. Next I try:

reload("ctbTestModule")

as suggested here, but f1(1) still returns 2. Finally, I try:

include("/SomeAbsolutePath/ctbTestModule.jl")

as suggested here, which is not ideal since I have to type out the full absolute path since the current directory might not be "/SomeAbsolutePath". I get the warning message Warning: replacing module ctbTestModule which sounds promising, but f1(1) still returns 2.

If I close the current Julia session, start a new one, and type in using ctbTestModule, I now get the desired behaviour, i.e. f1(1) returns 3. But obviously I want to do this without re-starting Julia.

So, what am I doing wrong?

Other details: Julia v0.2 on Ubuntu 14.04.

Moncada answered 30/7, 2014 at 4:39 Comment(2)
Thank you for including the 2018 update. Simply because it's been so long since workspace() existed in Julia, I'd go so far as to accept miguelmorin's answer now instead of the previously accepted answer that has since become deprecated.Irrawaddy
@Irrawaddy Honestly, I'm not sure what the appropriate etiquette is in this situation. The accepted answer does not necessarily need to be the right one. Rather the FAQ states that it is the one the asker found most helpful. For me, this is the currently accepted answer. Given my 2018 update at the top of my post directs readers in the right direction, I think it is better to just leave things be.Moncada
B
52

The basis of this problem is the confluence of reloading a module, but not being able to redefine a thing in the module Main (see the documentation here) -- that is at least until the new function workspace() was made available on July 13 2014. Recent versions of the 0.3 pre-release should have it.

Before workspace()

Consider the following simplistic module

module TstMod
export f

function f()
   return 1
end

end

Then use it....

julia> using TstMod

julia> f()
1

If the function f() is changed to return 2 and the module is reloaded, f is in fact updated. But not redefined in module Main.

julia> reload("TstMod")
Warning: replacing module TstMod

julia> TstMod.f()
2

julia> f()
1

The following warnings make the problem clear

julia> using TstMod
Warning: using TstMod.f in module Main conflicts with an existing identifier.

julia> using TstMod.f
Warning: ignoring conflicting import of TstMod.f into Main

Using workspace()

However, the new function workspace() clears Main preparing it for reloading TstMod

julia> workspace()

julia> reload("TstMod")

julia> using TstMod

julia> f()
2

Also, the previous Main is stored as LastMain

julia> whos()
Base                          Module
Core                          Module
LastMain                      Module
Main                          Module
TstMod                        Module
ans                           Nothing

julia> LastMain.f()
1
Banjermasin answered 1/8, 2014 at 10:4 Comment(5)
+1+Tick Many thanks. Will come back and award bounty once StackOverflow lets me :-)Moncada
clear!() and clear!(:TstMod) fail for me with ERROR: invalid redefinition of constant TstMod. I could not figure out how to use the documentation of Base.Distributed.clear!. @PatrickT, how would you use it?Gasket
What is the solution to this in 2020? None of the posted answers seem to work (reload function is not defined, neither is workspace in 1.5.3). Seems like quite a basic and important feature of an IDE, to clear the environment or reload a specific module.Austral
Someone who knows the answer to @Austral question?Dirham
The package Revise did the trick for meDirham
G
29

Use the package Revise, e.g.

Pkg.add("Revise") # do this only once

include("src/my_module.jl")
using Revise
import my_module

You may need to start this in a new REPL session. Notice the use of import instead of using, because using does not redefine the function in the Main module (as explained by @Maciek Leks and @waTeim).

Other solutions: Two advantages of Revise.jl compared to workspace() are that (1) it is much faster, and (2) it is future-proof, as workspace() was deprecated in 0.7, as discussed in this GitHub issue:

julia> VERSION
v"0.7.0-DEV.3089"

julia> workspace()
ERROR: UndefVarError: workspace not defined

and a GitHub contributor recommended Revise.jl:

Should we add some mesage like "workspace is deprecated, check out Revise.jl instead"?

Even in Julia 0.6.3, the three previous solutions of workspace(), import, and reload fail when a module called other modules, such as DataFrames. With all three methods, I got the same error when I called that module the second time in the same REPL:

ERROR: LoadError: MethodError: all(::DataFrames.##58#59, ::Array{Any,1}) is ambiguous. Candidates: ...

I also got many warning messages such as:

WARNING: Method definition macroexpand(Module, ANY) in module Compat at /Users/mmorin/.julia/v0.6/Compat/src/Compat.jl:87 overwritten in module Compat at /Users/mmorin/.julia/v0.6/Compat/src/Compat.jl:87.

Restarting the Julia session worked, but it was cumbersome. I found this issue in the Reexport package, with a similar error message:

MethodError: all(::Reexport.##2#6, ::Array{Any,1}) is ambiguous.

and followed the suggestion of one contributor:

Does this happen without using workspace()? That function is notorious for interacting poorly with packages, which is partly why it was deprecated in 0.7.

Gasket answered 12/6, 2018 at 11:40 Comment(1)
Thanks for this. I've added a banner at the top of the question suggesting that readers look at all the answers (especially this one), since the best solution has changed multiple times over the past few years.Moncada
P
13

In my humble opinion, the better way is to use import from the very beginning instead of using for the reported issue.

Consider the module:

module ModuleX1
  export produce_text
  produce_text() = begin
    println("v1.0") 
  end
  println("v1.0 loaded")
end

Then in REPL:

julia> import ModuleX1
v1.0 loaded

julia> ModuleX1.produce_text()
v1.0

Update the code of the module and save it:

module ModuleX1
  export produce_text
  produce_text() = begin
    println("v2.0")  
  end
  println("v2.0 loaded")
end

Next, in the REPL:

julia> reload("ModuleX1")
Warning: replacing module ModuleX1
v2.0 loaded

julia> ModuleX1.produce_text()
v2.0

Advantages of using import over using:

  • avoiding ambiguity in function calls (What to call: ModuleX1.produce_text() or produce_text() after reloading?)
  • do not have to call workspace() in order to get rid of ambiguity

Disadvantages of using import over using:

  • a fully qualified name in every call for every exported name is needed

Edited: Discarded "full access to the module, even to the not-exported names" from "Disadvantages..." according to the conversation below.

Phobia answered 12/8, 2015 at 13:27 Comment(1)
This is interesting. One quick question: I thought I could get full access to the non-exported names in a module via using as long as I fully qualified the call. Is this not correct?Moncada
H
9

workspace() has been deprecated.

You can reload("MyModule") in an active REPL session, and it works as expected: changes made to the source file that contains MyModule are reflected in the active REPL session.

This applies to modules that have been brought into scope by either import MyModule or using MyModule

Hertzfeld answered 26/8, 2017 at 8:41 Comment(3)
You need to use reload("MyModule") (i.e. with quotes) because reload() takes a string as an argument not a module.Chimera
I have a source file in src/my_module.jl. Calling reload("src/my_module.jl") gives the ERROR: LoadError: use 'include' instead of 'reload' to load source files. That is because of the slash, given that the source code suggests to include the source file if it has a slash. Without the slash, it says ERROR: LoadError: ArgumentError: Module my_module not found in current path. @bizzy, how do you use reload() with a source file in a sub-directory?Gasket
@bizzy: Found it, I had to add src/ to the LOAD_PATH: push!(LOAD_PATH, "src/"); reload("my_module").Gasket
T
2

I wanted to create a new module from scratch, and tried the different answers with 1.0 and didn’t get a satisfactory result, but I found the following worked for me:

From the Julia REPL in the directory I want to use for my project I run

pkg> generate MyModule

This creates a subdirectory like the following structure:

MyModule
├── Project.toml
└── src
    └── MyModule.jl

I put my module code in MyModule.jl. I change to the directory MyModule (or open it in my IDE) and add a file Scratch.jl with the following code:

using Pkg
Pkg.activate(".")
using Revise
import MyModule # or using MyModule

Then I can add my code to test below and everything updates without reloading the REPL.

Tegantegmen answered 9/5, 2019 at 0:1 Comment(1)
Doesn't this mean that MyModule (more like MyPackage) will now depend on Revise.jl (so installing your package will automatically install Revise), even though the latter is only used for development?Dissected
W
2

I battled to get Revise.jl to work for me, probably because I also use code generation with OOPMacro @class . I had to also call revise(...) See https://timholy.github.io/Revise.jl/stable/limitations/#Limitations-1.

I have been struggling with this problem for up to 5 years and finally got something that works for me that don't involve manually running my main module in the IDE repl after EVERY SMALL CHANGE :'( Here is some of my code I use to force a revise and exit early when running tests when there were revise errors:

# Common code I include into my test files now:
using Pkg
using Revise

# force include module
Pkg.activate("MyModule")
include("../src/MyModule.jl")


Pkg.activate("MyModule/test")
using Revise

# if there were any revise errors which means some compilation error or
# some change that requires a manual rerun or restart.
# then force you to fix it, rather that running lying tests..
for (k, e) in Revise.queue_errors
    if !isnothing(e)
        warn(logger, "Something went wrong while revising, you probably have a compile error in this file:")
        throw(e)
    end
end


module TestSomethingModule  
    using Revise
    using MyModule
    revise(MyModule)
    ... 
    # then you can call MyModule.doSomithing in a test and it actually updates
end
Waadt answered 12/2, 2023 at 20:0 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.