Fix the stubgen test flake.

[msullivan]

The core problem is that importlib.invalidate_caches() is full of
lies and does not actually invalidate all of the relevant
caches. Python maintains sys.path_importer_cache, which maps from
directory names on the sys.path to "importer" objects for that path
entry. importlib.invalidate_caches() invalidates the caches inside
each of the per-directory importer objects, but there is an additional
negative cache that is not cleared: sys.path entries whose directory
doesn't exist have their importer set to None, which prevents that
path entry from ever being searched unless it is manually cleared.

This failure comes up rarely, because it only occurs if the following
events occur in sequence:

1. 'stubgen-test-path' is added to sys.path, but no new import is
   done while there is a 'stubgen-test-path' subdirectory in the working
   directory. This is done by all the stubgen tests that don't end with
   _import.
2. Some non-stubgen test does an import of a previously unimported
   module. This will cause
   sys.path_importer_cache['stubgen-test-path'] to be set to None
   when the directory doesn't exist during the module search.
   This can happen the first time that a _python2 test is run
   since parse.parse() dynamically imports mypy.fastparse2
   when asked to parse python 2.
3. A stubgen test tries to use importlib to import a module in
   'stubgen-test-path'. All of the _import tests do this.

We fix this by clearing out the relevant entry from
sys.path_importer_cache ourselves.

Fixes #4635

[msullivan]

Testing this was somewhat annoying, since pytest doesn't make it easy to control ordering of tests. What I ended up doing was creating a _teststubgenstupid.py with a test_name_suffix:

import mypy.test.teststubgen
class StubgenPythonLateSuite(mypy.test.teststubgen.StubgenPythonSuite):
    test_name_suffix = '_early'

and then running py.test --capture=no -v -n0 mypy/test/_teststubgenstupid.py mypy/test/testcheck.py mypy/test/teststubgen.py -k '(testEmptyFile_early or testFunctionalEnum_python2 or testAllAndClass_import) and not testAllAndClass_import_early'

[JukkaL]

Wow, impressive analysis! It will be great to finally get rid of this test flake. Thanks for fixing this!

I guess another way to work around the issue would be to only add randomly generated path names to sys.path. Maybe mention this in a comment somewhere in case we'd need to change the approach for some reason?

Also left one note about a comment.

[JukkaL]

There is something missing in this sentence. Maybe this should be "... clear cache entries that indicate ..."

[gvanrossum]

Great analysis. I filed a CPython bug at https://bugs.python.org/issue33169.