Compile Python Scripts into Executables
Introduction #
If you have problems with deploying your python scripts, you can use the following method to compile your python script.
This method involves turning your python file into a C file and compiling the C file into an executable in windows.
Cythonizing the python file #
First, you need to get cython from pip by:
pip install cython
The official cython website says:
Cython is an optimizing static compiler for both the Python programming language
and the extended Cython programming language (based on Pyrex). It makes writing
C extensions for Python as easy as Python itself.
...
The Cython language is a superset of the Python language that additionally supports
calling C functions and declaring C types on variables and class attributes. This
allows the compiler to generate very efficient C code from Cython code. The C code
is generated once and then compiles with all major C/C++ compilers...
But we can also use cython to turn a python program into a C file which uses the Python SDK. Let’s say we have a python file:
def func():
print("Hello, world")
We can use cython to `convert’ this file into a C file using:
cython file.py -o file.c -3
We also can use the -a option to see what C code corresponds to your python code, for instance, using the command:
cython file.py -o file.c -3 -a
puts a html file in your folder with the same basename of the python script (file.html in this case), which contains several sections of C code mapped to a python line:
for exanple: def func(): converts to:
static PyObject *__pyx_pw_4file_1func(PyObject *__pyx_self, CYTHON_UNUSED PyObject *unused); /*proto*/
static PyMethodDef __pyx_mdef_4file_1func = {"func", (PyCFunction)__pyx_pw_4file_1func, METH_NOARGS, 0};
static PyObject *__pyx_pw_4file_1func(PyObject *__pyx_self, CYTHON_UNUSED PyObject *unused) {
PyObject *__pyx_r = 0;
__Pyx_RefNannyDeclarations
__Pyx_RefNannySetupContext("func (wrapper)", 0);
__pyx_r = __pyx_pf_4file_func(__pyx_self);
/* function exit code */
__Pyx_RefNannyFinishContext();
return __pyx_r;
}
static PyObject *__pyx_pf_4file_func(CYTHON_UNUSED PyObject *__pyx_self) {
PyObject *__pyx_r = NULL;
__Pyx_RefNannyDeclarations
__Pyx_RefNannySetupContext("func", 0);
/* … */
/* function exit code */
__pyx_r = Py_None; __Pyx_INCREF(Py_None);
goto __pyx_L0;
__pyx_L1_error:;
__Pyx_XDECREF(__pyx_t_1);
__Pyx_AddTraceback("file.func", __pyx_clineno, __pyx_lineno, __pyx_filename);
__pyx_r = NULL;
__pyx_L0:;
__Pyx_XGIVEREF(__pyx_r);
__Pyx_RefNannyFinishContext();
return __pyx_r;
}
/* … */
__pyx_t_1 = __Pyx_CyFunction_New(&__pyx_mdef_4file_1func, 0, __pyx_n_s_func, NULL, __pyx_n_s_file, __pyx_d, ((PyObject *)__pyx_codeobj__2)); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 1, __pyx_L1_error)
__Pyx_GOTREF(__pyx_t_1);
if (PyDict_SetItem(__pyx_d, __pyx_n_s_func, __pyx_t_1) < 0) __PYX_ERR(0, 1, __pyx_L1_error)
__Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
and, print("Hello, world!") converts to:
__pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_print, __pyx_tuple_, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error)
__Pyx_GOTREF(__pyx_t_1);
__Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
/* … */
__pyx_tuple_ = PyTuple_Pack(1, __pyx_kp_u_Hello_world); if (unlikely(!__pyx_tuple_)) __PYX_ERR(0, 2, __pyx_L1_error)
__Pyx_GOTREF(__pyx_tuple_);
__Pyx_GIVEREF(__pyx_tuple_);
For now we are just compiling, we might not wanna look at the HTML file, so we omit the -a flag.
Also we notice that, the file file.c doesn’t have a main function and this is because, cython treats the file as a module and doesn’t place the interpreter in the output code.
As cython treats the input file as a module, the filename should be a valid python module name or we also can use the --module-name flag.
If we want the main function, we need to use the --embed flag.
cython file.py -o file.c -3 --embed
Now, in file.c, we see code similar to this:
/* MainFunction */
#if PY_MAJOR_VERSION < 3
int main(int argc, char** argv) {
#elif defined(WIN32) || defined(MS_WINDOWS)
int wmain(int argc, wchar_t **argv) {
#else
static int __Pyx_main(int argc, wchar_t **argv) {
#endif
#ifdef __FreeBSD__
fp_except_t m;
m = fpgetmask();
fpsetmask(m & ~FP_X_OFL);
#endif
if (argc && argv)
Py_SetProgramName(argv[0]);
Py_Initialize();
if (argc && argv)
PySys_SetArgv(argc, argv);
{
PyObject* m = NULL;
__pyx_module_is_main_file = 1;
#if PY_MAJOR_VERSION < 3
initfile();
#elif CYTHON_PEP489_MULTI_PHASE_INIT
m = PyInit_file();
if (!PyModule_Check(m)) {
PyModuleDef *mdef = (PyModuleDef *) m;
PyObject *modname = PyUnicode_FromString("__main__");
m = NULL;
if (modname) {
m = PyModule_NewObject(modname);
Py_DECREF(modname);
if (m) PyModule_ExecDef(m, mdef);
}
}
#else
m = PyInit_file();
#endif
if (PyErr_Occurred()) {
PyErr_Print();
#if PY_MAJOR_VERSION < 3
if (Py_FlushLine()) PyErr_Clear();
#endif
return 1;
}
Py_XDECREF(m);
}
#if PY_VERSION_HEX < 0x03060000
Py_Finalize();
#else
if (Py_FinalizeEx() < 0)
return 2;
#endif
return 0;
}
The function PyModule_ExecDef calls the __main__ module object.
If we are in windows and don’t need a console window, we can add #include <Windows.h> to the beginning of the C file and add
SetForegroundWindow(GetConsoleWindow(), SW_HIDE);
just after the wmain(...) function definition between #elif defined(WIN32) || defined(MS_WINDOWS) and #else directives.
This can also be done using the winapi or ctypes modules.
Now that we have the C file, we have successfully “cythonized” the python file.
Compiling the cythonized file #
First, you will have to locate your python installation folder. You can use the following command and get the folder.
where.exe python.exe
For me, python.exe was in D:\Tools\Scoop\apps\python\current\.
Maybe store it in a environment variable like:
$PYROOT = 'D:\Tools\Scoop\apps\python\current\' # Powershell
In the $PYROOT folder, you will notice the folders include which contains the headers and libs which contains the python3XX.lib library.
The XX in the library name is to be replaced with the version of your installation, or you can list the folder and just note the filename.
python -V # Python 3.11.3
# python3XX -> python311 (Omit the .3)
We can use meson to build the C file:
project('C Build', 'c')
cc = meson.get_compiler('c')
depPython = declare_dependency(
include_directories: ['Path\\to\\python\\root\\include'],
dependencies: [cc.find_library('python3XX', dirs: ['Path\\to\\python\\root\\libs'])] # CORRECT YOUR PYTHON VERSION
)
sources = ['file.c'] # UPDATE THE SOURCE FILE
executable('file', sources, dependencies: [depPython])
You can get an example meson.build file here.
To setup the meson project, get meson from pip and run:
meson setup builddir --buildtype release
meson setup builddir --buildtype debugoptimized # For -Og build
and to compile the project, run:
meson compile -C builddir
The output file is stored in ./builddir/file.exe.
Or if you happen to use gcc from msys2, you can use the following command:
gcc -I $PYROOT/include file.c -l:python3XX.lib -o file.exe -L $PYROOT/libs
You can also add a custom_target to meson.build to automate the build process.
You are free to add optimization flags like -O3 and -march=native and also strip the output file to remove all the symbols.
strip file.exe -s file_stripped.exe # If you also have binutils
The above commands work well with clang with the mingw toolchain.
Distribution #
In an empty folder, copy the output file (file.exe), python3.dll and copy the python.zip file (located in the $PYROOT folder) from the portable zip for your python version.
For my python version 3.11 Releases.
The portable zip download is named as Windows embeddable package (64-bit).
If you remember using in-built python modules like ctypes or you get errors like ModuleNotFoundError: _ctypes was not found, copy the _ctypes.pyd file from the portable zip file.
Remmeber, you will need the Visual Studio C++ Redistributable libraries on the target machine or ship the needed VCRedist libraries with the executable.