Why pytest -n auto can be slower in WSL
TIL: Why pytest -n auto Can Be Slower in WSL
While trying to speed up tests with pytest-xdist, I noticed something unexpected: running pytest -n auto inside WSL actually made the tests slower.
The issue turned out to be where the repository was stored.
If the project lives on a Windows-mounted path like:
/mnt/c/Users/...
filesystem operations have to cross the Windows - Linux boundary. This adds noticeable overhead for things Python tools do frequently, such as:
- module imports
- directory scanning
- plugin discovery
- test collection
Since pytest performs a lot of this work during startup, the slowdown becomes very visible.
Why xdist can make it worse
Running:
pytest -n auto
starts multiple worker processes.
Each worker independently performs:
- test discovery
- module imports
- plugin loading
- directory scanning
If filesystem access is already slow, parallel workers end up multiplying the overhead, which can make the run slower instead of faster.
What usually fixes it
Move the repository into the WSL filesystem
Instead of:
/mnt/c/dev/project
use something like:
~/src/project
Keeping Linux workloads inside the WSL filesystem typically reduces filesystem overhead significantly.
You can also experiment with fewer workers:
pytest -n 2
Simple rule
- Linux tools - keep code in the WSL filesystem
- Windows tools - keep code in the Windows filesystem
Mixing them works, but it often introduces unnecessary performance overhead.