Problem Statement
I want to run and debug my own C++ extensions for python in "hybrid mode" in VSCode. Since defining your own python wrappers can be quite tedious, I want to use pybind11 to link C++ and python. I love the debugging tools of vscode, so I would like to debug both my python scripts as well as the C++ functions in vscode.
Fortunately, debugging python and C++ files simultaneously is possible by first starting the python debugger and then attach a gdb debugger to that process as described in detail in nadiah's blog post (Windows users, please note this question). This works fine for me. Unfortunately, they define the C++ -- python bindings manually. I would like to use pybind11 instead.
I created a simplified example that is aligned with nadiah's example using pybind11. Debugging the python file works but the gdb debugger doesn't stop in the .cpp file. According to this github question it should be theoretically possible but there are no details on how to achieve this.
Steps to reproduce
Here I try to follow nadiahs example as closely as possible but include pybind11 wrappers.
Setting up the package
Create a virtual environment (also works with anaconda, as described below)
virtualenv --python=python3.8 myadd
cd myadd/
. bin/activate
Create file myadd.cpp
#include <pybind11/pybind11.h>
float method_myadd(float arg1, float arg2) {
float return_val = arg1 + arg2;
return return_val;
}
PYBIND11_MODULE(myadd, handle) {
handle.doc() = "This is documentation";
handle.def("myadd", &method_myadd);
}
, myscript.py
import myadd
print("going to ADD SOME NUMBERS")
x = myadd.myadd(5,6)
print(x)
and setup.py
from glob import glob
from distutils.core import setup, Extension
from pybind11.setup_helpers import Pybind11Extension
def main():
setup(name="myadd",
version="1.0.0",
description="Python interface for the myadd C library function",
author="Nadiah",
author_email="[email protected]",
ext_modules=[Pybind11Extension("myadd",["myadd.cpp"])],
)
if __name__ == "__main__":
main()
Clone the pybind11 repo
git clone [email protected]:pybind/pybind11.git
and install the python package
pip install pybind11
Run the setup script
python3 setup.py install
Now, we can already run the python script
python myscript.py
Setting up vscode
Open vscode
code .
Select the python interpreter with Ctrl+Shift+p -> Select python interpreter -> ./bin/python, now in the lower bar, you should see virtualenv myadd.
Create the launch.json file by clicking the debug symbol and 'Create new launch configuration'.
This is my launch.json (This might be the problem)
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Python",
"type": "python",
"request": "launch",
"program": "myscript.py",
"console": "integratedTerminal"
},
{
"name": "(gdb) Attach",
"type": "cppdbg",
"request": "attach",
"program": "${workspaceFolder}/bin/python", /* My virtual env */
"processId": "${command:pickProcess}",
"MIMode": "gdb",
"setupCommands": [
{
"description": "Enable pretty-printing for gdb",
"text": "-enable-pretty-printing",
"ignoreFailures": true
}
],
"additionalSOLibSearchPath": "${workspaceFolder}/build/lib.linux-x86_64-3.8;${workspaceFolder}/lib;${workspaceFolder}/lib/python3.8/site-packages/myadd-1.0.0-py3.8-linux-x86_64.egg/"
}
]
}
Note that I added the "additionalSOLibSearchPath" option in accordance to the github question but it did not change anything.
Debugging
In vscode, add breakpoints in myscript.py in line 5 and 7, and in myadd.cpp in line 5.
Now, first start the python debugger and let it stop on the breakpoint in line 5.
Then go to a terminal and get the correct process id of the running python script.
ps aux | grep python
The second to last process is the correct one in my case. E.g.
username **65715** 3.0 0.0 485496 29812 pts/3 Sl+ 10:37 0:00 /home/username/myadd/bin/python /home/username/.vscode/extensions/ms-python.python-2022.0.1814523869/pythonFiles/lib/python/debugpy --connect 127.0.0.1:38473 --configure-qt none --adapter-access-token ... myscript.py
In this example, 65715 would be the correct process id.
Now, in vscode start the (gdb) Attach debugger and type in the process id in the search bar. Hit enter, now you need to type y in the console to allow the attaching and type in your sudo password.
If you are following nadiah's example, you can now press continue on the python debug bar and the script will stop on the C++ breakpoint.
For this pybind11 example, the script does not stop on the C++ breakpoint.
Project Structure
Your project structure now should look like this
myadd
| bin/
| build/
| dist/
| lib/
| myadd.cpp
| myadd.egg-info/
| myscript.py
| pybind11/
| setup.py
Things I also tried
As stated in the github post, one has to ensure that the debug flag is set. Therefore, I added a setup.cfg file
[build_ext]
debug=1
[aliases]
debug_install = build_ext --debug install
And ran
python setup.py debug_install
but this did not help as well.
Using anaconda instead of virtualenv
Using conda instead of virtualenv is quite easy. Just create your env as usual and then type in
which python
to get the path to the python executable. Replace the "program" in the (gdb) Attach debug configuration of your launch.json with this path.
Software versions
I run
- Ubuntu 20.04
- Vscode 1.64.2 x64
- GNU gdb 9.2
- gcc 9.3.0
- python 3.8 as defined in the virtualenv.
ps aux | grep pythonthat I described above works for me. Thank you! – Skyler