Folders named 'py' result in "Test result not found"

[botsman]

Environment data

• VS Code version: 1.60.0
• Extension version (available under the Extensions sidebar): v2021.9.1230869389
• OS and version: MacOS 11.6
• Python version (& distribution if applicable, e.g. Anaconda): 3.9.1
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): pyenv
• Relevant/affected Python packages and their versions: pytest==6.2.5
• Value of the python.languageServer setting: Pylance

Expected behaviour

This simple test should appear succeeded in UI :

def test_passing():
   assert 42 == 42

Actual behaviour

The test is failed (red circle) with this reason :

Steps to reproduce:

The issue is very similar to #17270 which is closed by #17275

The only difference that in my case folder name is not starting with "py", but the whole folder name is "py":

1. Have pytest installed and selected as a default test framework
2. Put tests inside folder named "py". For example, "./tests/py/"
3. Try to execute test
4. The test is executed correctly, but test UI shows error.

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

> ~/.pyenv/versions/3.9.1/envs/vscode_python_pytest/bin/python -m pytest --rootdir ~/projects/vscode_python_pytest --override-ini junit_family=xunit1 --junit-xml=/var/folders/0_/6w94lk1171vc_fpwbjqx02nw0000gn/T/tmp-14897brlMlEUuYALH.xml ./tests/py/test_something.py::test_passing
cwd: ~/projects/vscode_python_pytest

[karthiknadig]

This is a very specific case with folders named py. This is a problem due to the way the test runner generates test ids. We plan on addressing this in the test adapter re-write.

[botsman]

One more thing. It looks like parameter --rootdir in the python.testing.pytestArgs also affects behavior of test execution from the test explorer UI.

Just want remind that it should be taken into account.

1. If I use

"python.testing.pytestArgs": [
        "./tests/py"
    ]

then tests are discovered, but if I try to execute those, then I am getting a message in Python Test Log: ERROR: file or directory not found: ... and the test I clicked continues to be displayed as gray (not executed)

2. If I use

"python.testing.pytestArgs": [
        "--rootdir",
        "./tests/py"
    ]

then test are discovered, but I am getting an error described in the first message in this issue: Test result not found for: ... and the test I clicked is turning red (as on the screenshot in the first post)

[hmrd-forpeople]

Is there a workaround for this (other than renaming directories) until the fix/rewrite is released? Or are we stuck with a broken test runner until then?

[karthiknadig]

@hszw-forpeople I recommend using an older version of the extension 2021.8.*, if you cannot change the name.

[botsman]

Just a note, probably related to #17242

[ianjirka]

Can confirm - renaming my directory from 'py' to '_py' works around this issue.