How to pack a python to exe while keeping .py source code editable?

Question

I am creating a python script that should modify itself and be portable.

I can achieve each one of those goals separately, but not together.

I use cx_freeze or pyinstaller to pack my .py to exe, so it's portable; but then I have a lot of .pyc compiled files and I can't edit my .py file from the software itself.

Is there a way to keep a script portable and lightweight (so a 70mb portable python environment is not an option) but still editable?

The idea is to have a sort of exe "interpreter" like python.exe but with all the libraries linked, as pyinstaller allows, that runs the .py file, so the .py script can edit itself or be edited by other scripts and still be executed with the interpreter.

Authority

Do you expect your users to already have a python environment available? If so you can simply ship you py files, along with a bash script (or even an executable) that will simply launch python to run your scripts. – Komsa

@BigBro no, I expect my users to just double-click and run (it's ok to have a .bat file that runs "hypotetical_python.exe my_script.exe") without any download or previous installation. Just like pyinstaller does, but keeping an editable .py for further automatic editing / patching. – Authority

You could try excluding the "patch" file and bundling it using the data option. In your script, you load the contents from the given data location. To patch, you update the contents of the data directory. – Putsch

Isn't data for including resource files or dlls or other things like that? I mean, it's possible to include in the data folder a .py file and let the executable run it? – Authority

Just tested. It works. I'll post an example as an answer. – Putsch

Answer 1

First define your main script (cannot be changed) main_script.py. In a subfolder (e.g. named data) create patch_script.py

main_script.py:

import sys
sys.path.append('./data')
import patch_script

inside the subfolder:

data\patch_script.py:

print('This is the original file')

In the root folder create a spec file e.g. by running pyinstaller main_script.py. Inside the spec file, add the patch script as a data resource:

     ...
     datas=[('./data/patch_script.py', 'data' ) ],
     ...

Run pyinstaller main_sript.spec. Execute the exe file, it should print

This is the original file

Edit the patch script to e.g. say:

print('This is the patched file')

Rerun the exe file, it should print

This is the patched file

Note: As this is a PoC, this works but is prone to security issues, as the python file inside the data directory can be used for injection of arbitrary code (which you don't have any control of). You might want to consider using proper packages and update scripts as used by PIP etc.

Putsch

Really cool, that's exactly what I was looking for! Sure, this leads to potential disasters in the security, but that's something I "don't care" right now 'cause this will be part of a bigger project including cryptography and other things to protect the code :) Thank you very much for your help! Marked as an answer! – Authority

You have been warned ;) – Putsch

You might want to have a look on how to do this properly: pypi.org/project/PyUpdater – Putsch

Thank you for the link, the problem there seems that the code couldn't self modify itself. Not a big problem right now but I'd like to keep this possibility alive. I'll read the documentation btw, maybe I'm missing something – Authority

@WyattGillette that depends on what you mean. Do you want the local app copy on the user's computer to arbitrarily edit itself? If so, then no, PyUpdater isn't really for that... but it sounds like a bad idea anyway, just use settings files like JSON or YAML for something like that. If you want to be able to send updates that alter/replace what the user's app does, then yes, that's exactly what PyUpdater is made for. – Sadfaced

Answer 2

With PyInstaller, you can achieve this simply by setting the option:

module_collection_mode = 'py'

in a hook file named "hook-{my_full_name_package}.py".

Then, add the argument to pyinstaller command:

pyinstaller --additional-hooks-dir=PATH_WHERE_HOOK_FILE_IS

Make sure to use also the argument --onedir, to be able to locate the package folder in the subfolder _internal, and modify all the python files you need.

Full documentation about this feature here: pyinstaller hooks.

Yearling