Use Pycache Prefix to Move pyc Files out of Your Source Code Starting in Python 3.8

Specify a parallel filesystem cache for compiled bytecode
Tyler Tyler (258)
0

If you've been working with Python for a while, you've probably noticed .pyc files in your source code. In fact, you've probably got a line like *.pyc in your .gitignore file to prevent checking them in.

These pyc files are called "compiled bytecode", and they are generated by the Python interpreter any time a .py file is imported. This happens because Python libraries/modules are often imported more than once by a program, and there's no reason to re-compile the file every time.

In some environments there are issues related to having pyc files in the source code, and one proposed solution involved designating a separate, parallel filesystem, outside of the source code that can be used to cache compiled bytecode. This feature was included in Python 3.8, and in this guide we'll learn how to use it.

One way to configure the pycache prefix is to set the PYTHONPYCACHEPREFIX environment variable.

As an example, I'll create an example project that looks like this:

.
├── bytecodes
└── codes
    ├── main.py
    └── my_module.py

As you might guess, I want my source code in codes and my bytecode in bytecodes. Also, main.py imports a function from my_module.py.

Now, in order to move the pyc files to the bytecodes directory, I'll set an environment variable, and run the program:

PYTHONPYCACHEPREFIX=$(pwd)/bytecodes python3 codes/main.py

Now, let's see what our file tree looks like:

.
├── bytecodes
|      |── Library
...
│      └── Users
│          └── tyler
│              └── tmp
│                  └── test_code
│                      └── codes
│                          └── my_module.cpython-38.pyc
└── codes
    ├── main.py
    └── my_module.py

As expected, Python has created a parallel file structure inside of the bytecodes directory where it stores the compiled bytecode.

Similar to the previous example, we can use the -X option and specify our path.

python3 -X pycache_prefix=bytecodes codes/main.py

This is equivalent to the example in step 1. All of the compiled bytecode is now located in our bytecodes directory.

Series: Python 3.8 Features
Positional-Only Parameters
Show all in this series