Skip to content

Commit 962648e

Browse files
committed
Troubleshooting guide: add dynamic patcher problem handling
- add basic description of the module cleanup hooks
1 parent 2551f7b commit 962648e

File tree

2 files changed

+56
-6
lines changed

2 files changed

+56
-6
lines changed

docs/troubleshooting.rst

+53
Original file line numberDiff line numberDiff line change
@@ -325,6 +325,59 @@ There are some situations where that may happen, probably without you noticing:
325325
* The same is true, if you use ``setUpPyfakefs`` or ``setUpClassPyfakefs`` in a unittest context, or if you use
326326
the ``patchfs`` decorator. ``Patcher`` instances created in the tests will be ignored likewise.
327327

328+
.. _failing_dyn_patcher:
329+
330+
Tests failing after a test using pyfakefs
331+
-----------------------------------------
332+
If previously passing tests fail after a test using ``pyfakefs``, something may be wrong with reverting the
333+
patches. The most likely cause is a problem with the dynamic patcher, which is invoked if modules are loaded
334+
dynamically during the tests. These modules are removed after the test, and reloaded the next time they are
335+
imported, to avoid any remaining patched functions or variables. Sometimes, there is a problem with that reload.
336+
337+
If you want to know if your problem is indeed with the dynamic patcher, you can switch it off by setting
338+
:ref:`use_dynamic_patch` to `False` (here an example with pytest):
339+
340+
.. code:: python
341+
342+
@pytest.fixture
343+
def fs_no_dyn_patch():
344+
with Patcher(use_dynamic_patch=False):
345+
yield
346+
347+
348+
def test_something(fs_no_dyn_patch):
349+
... # do the testing
350+
351+
If in this case the following tests pass as expected, the dynamic patcher is indeed the problem.
352+
If your ``pyfakefs`` test also works with that setting, you may just use this. Otherwise,
353+
the dynamic patcher is needed, and the concrete problem has to be fixed. There is the possibility
354+
to add a hook for the cleanup of a specific module, which allows to change the process of unloading
355+
the module. This is currently used in ``pyfakefs`` for two cases: to reload ``django`` views instead of
356+
just unloading them (needed due to some django internals), and for the reload of a specific module
357+
in ``pandas``, which does not work out of the box.
358+
359+
A cleanup handler takes the module name as an argument, and returns a Boolean that indicates if the
360+
cleanup was handled (by returning `True`), or if the module should still be unloaded. This handler has to
361+
be added to the patcher:
362+
363+
.. code:: python
364+
365+
def handler_no_cleanup(_name):
366+
# This is the simplest case: no cleanup is done at all.
367+
# This makes only sense if you are sure that no file system functions are called.
368+
return True
369+
370+
371+
@pytest.fixture
372+
def my_fs():
373+
with Patcher():
374+
patcher["modulename"] = handler_no_cleanup
375+
yield
376+
377+
As this may not be trivial, we recommend to write an issue in ``pyfakefs`` with a reproducible example.
378+
We will analyze the problem, and if we find a solution we will either get this fixed in ``pyfakefs``
379+
(if it is related to a commonly used module), or help you to resolve it.
380+
328381

329382
.. _`multiprocessing`: https://docs.python.org/3/library/multiprocessing.html
330383
.. _`subprocess`: https://docs.python.org/3/library/subprocess.html

docs/usage.rst

+3-6
Original file line numberDiff line numberDiff line change
@@ -671,19 +671,16 @@ If you want to clear the cache just for a specific test instead, you can call
671671
fs.clear_cache()
672672
...
673673
674+
.. _use_dynamic_patch:
675+
674676
use_dynamic_patch
675677
~~~~~~~~~~~~~~~~~
676678
If ``True`` (the default), dynamic patching after setup is used (for example
677679
for modules loaded locally inside of functions).
678680
Can be switched off if it causes unwanted side effects, though that would mean that
679681
dynamically loaded modules are no longer patched, if they use file system functions.
682+
See also :ref:`failing_dyn_patcher` in the troubleshooting guide for more information.
680683

681-
A possible problem with dynamically loaded modules is their unloading, for example if
682-
reloading the module fails after unloading due to a problem in the module.
683-
This kind of problem may be solved more specifically by registering a handler
684-
for specific modules (using `Patcher.register_cleanup_handler`),
685-
that will be called during the cleanup process. This is used internally
686-
to handle known problems with the `django` and `pandas` packages.
687684

688685
.. _convenience_methods:
689686

0 commit comments

Comments
 (0)