Speeding up package load in Julia

Question

I wrote a program to solve a linear program in Julia using GLPKMathProgInterface and JuMP. The Julia code is being called by python program which runs multiple instances of the Juila code through multiple command line calls. While I'm extremely happy with the performance of the actual solver the initialization is extremely slow. I was wondering if there were approaches to speed this up.

For example if I just save the following to a file

@time using DataFrames, CSV, GLPKMathProgInterface, JuMP, ArgParse

and run it

mylabtop:~ me$ julia test.jl 
12.270137 seconds (6.54 M allocations: 364.537 MiB, 3.05% gc time)

This seems extremely slow, is there some good way to speed up using modules like a precompile step I could do once?

Fleecy

May I ask why you use multiple command-line calls to Julia? Is there a specific need for "command-line calls"? Because what I would do instead is to have a single Julia subprocess and call "my solver" within this subprocess. – Lenis

Answer 1

Since you haven't gotten any answers yet, let me give you the general first order answers - although I hope someone more qualified will answer your question in more detail (and correct me if I'm wrong).

1) Loading packages in Julia is sometimes rather slow up to the time of this writing. It has been discussed many times and you can expect improvements in the future. AFAIK this will happen in early 1.x releases after 1.0 is out. Have a look at this thread.

2) Since you typically only have to pay the loading time cost once per Julia session one approach is to keep the session running for as long as possible. You can execute your script with include("test.jl") from within the session. Let me also mention the amazing Revise.jl - it's hardly possible to overemphasize this package!

3) (I have no experience with this more difficult approach.) There is PackageCompiler.jl which allows you to compile a package into your system image. Read this blog post by Simon.

4) (Not recommended) There has also been the highly experimental static-julia which statically compiles your script into an shared library and executable.

Hope that helps.

Pulchia