Distribute a standalone Python software with Julia dependencies

Question

I have a software that is mostly written in Python and, for now, I'm using PyInstaller to bundle and distribute the software in a user-friendly way (it's part of my CI pipeline, for Linux and Windows).

However, my performance is terrible and I want to rewrite some heavy parts in Julia while keeping the front-end in Python. I can use PyJulia to do this, but it means that the user has to install Julia manually in order to use my program.

Julia does have the equivalent of PyInstaller, which is PackageCompiler.jl, but I don't know how to call something compiled with PackageCompiler.jl from the Python side.

How can I make this work, so I can bundle and distribute an executable that has Python, Julia and everything it needs to run?

---

A little more details

My end user is someone (chemists and pharmacists) that have no idea what programming is. They don't have Python, Julia or Docker (and they don't even want to install it).

In my current approach, the software bundled with PyInstaller consists of a single executable with everything inside it (Python and everything it needs). What I really want is to keep the same user experience, but also with Julia running on the backstage.

I'll implement several functions in the Julia side, and I want (almost) the same level of integration as I get with PyJulia.

Maybe I'll go to Rust and just use the C interface, but I really would like to use Julia.

Thank y'all for your time.

Durware

Would Docker be an option for you? PackageCompiler is usually only needed if startup time for the Julia part is an issue for you, otherwise it is easier to rely on Pkg for installing the Julia side. – Fulton

@Fulton I can't rely on Pkg because it won't exist on the user's machine. For the same reason, Docker isn't helpful because, ideally, my end-user has no idea what it is. – Durware

If you didn't need to support Windows, Nix would be well-suited to providing a single tool that acts a bit like Python's virtualenv, but able to install and manage software written in any combination of C, Rust, Python, Julia, etc. On the other hand, if you don't have IT support capable of installing Docker for your users, installing Nix (or troubleshooting cases where an OS update breaks it) is probably not a reasonable ask. – Envision

Answer 1

Per here: https://julialang.github.io/PackageCompiler.jl/stable/apps.html#Creating-an-app-1 you can get what is basically an executable file and then you can just follow this post: Python Script execute commands in Terminal to execute the file you create.

Howerton

Yeah, I'm aware of these solutions, but in this case I can only have the julia_main() entrypoint, which isn't so good. I'm looking for something more like PyJulia (but compiled). – Durware

@HélitonMartins instead of building an "app", build a "library", and then call that directly from Python using ctypes. But you will have to wrap the Julia package functions with Base.@ccallable functions. This could be done programmatically with metaprogramming. – Toddle

Answer 2

You could try using JuliaWin. This provides a standalone Julia runtime that works fully self-contained.

We deployed a tool using PyInstaller with the JuliaWin included as one of the datas as PyInstaller calls them. Iirc it was not PyJulia but JuliaCall+PythonCall doing the interfacing in our case.

Unfortunately the startup was ridiculous (order of minutes). This is in part due to Julia's startup time, but largely due to the generated .exe first having to unpack the JuliaWin. So we are currently investigating using PackageCompiler, possibly also switching to providing the user with 2 files instead of 1.

Barricade