Support `restart` in terminated event in debug adapter

[nturinski]

It would be really handy if Python launch configurations had a similar property to Node.js's restart property. For reference: https://code.visualstudio.com/docs/nodejs/nodejs-debugging#_restarting-debug-sessions-automatically-when-source-is-edited.

The use case is whenever a python file is edited and saved, it causes the debugger to detach so it'd be nice to have an option to automatically re-attach the debugger in those instances.

[karthiknadig]

Thank you for the suggestion! We have marked this issue as "needs decision" to make sure we have a conversation about your idea. We plan to leave this feature request open for at least a month to see how many 👍 votes the opening comment gets to help us make our decision.

[stefanushinardi]

Hello, just to give a few data points/verbatim from other github threads(microsoft/vscode-azurefunctions#781 & Azure/azure-functions-host#3543) regarding customer pain points that this change might alleviate

Thanks!

[anirudhgarg]

Adding +1 on this - we have many customers in Azure Functions Python VSCode experience

[stefanushinardi]

Hello @karthiknadig, do you think there is already enough data points for this work item? Thanks!

[karthiknadig]

@stefanushinardi Thanks for the data points. When python files are edited, which process is doing the reload? is there a managing process that handles terminating and reloading the project. If yes, then is that process running on python?

/cc @fabioz @int19h

[stefanushinardi]

Pinging @nturinski and @Hazhzeng who are the experts on this

[nturinski]

When python files are edited, which process is doing the reload? is there a managing process that handles terminating and reloading the project. If yes, then is that process running on python?

I honestly rarely work with Python so I really don't know for sure, but I think VS Code is using this repo to do its debugging: https://github.com/microsoft/debugpy. So I imagine that there should be a way to watch the files via the debugpy process while its debugging and then have the Python extension reattach after a change.

[karthiknadig]

@nturinski The issue is we don't know which process it is if the restarted python process was not started by a parent that was already running under the debugger.

I am one of the authors of debugpy, and we support this for projects like django which handles re-load on save. They way it works there is, the main django process is run under the debugger. django then starts multiple worker processes which render the webpage, since these processes were started by a process with debugger already in it, the debugger monitors all the processes under the main process. When a developer makes changes to python code, django detects it terminates the sub-process that has that file loaded, and start an new sub-process with the updated code. They key here is that the django main process is not killed, so the debugger is still able to handle the newly spawned sub-processes. If there is a main root process that has the debugger running in it then this should work if the debugger is injected into that root process.

If there is no main process that serves as a root (with debugger running in it), then it becomes difficult to know when the new processes gets spawned. There is no way from the debugger to tell this from outside of the newly spawned process. It could be a unrelated python process. Another issue is, how do we know that a debugging session is over? how long does it take from process terminating to a new process start? The only direction that could potentially work in this case is the reverse direction, that is the launched process some how starts debug session (this is not supported and not a small work item). I am not sure how that scenario will work with vscode server.

[karthiknadig]

My original question is still un-answered. Is there a process that you have that restarts the server? if yes then is that process a python process? if yes then can that process itself be run under the debugger?
if no then we need to understand the architecture in greater detail to provide a solution.

[int19h]

Note that the original Node.js scenario that is referenced is an attach scenario - the user explicitly runs some code under nodemon, and attaches to that code. When some file in the monitored directories change, nodemon restarts the whole process.

So if I'm understanding the feature request correctly, it's a debug configuration property that would cause the debugger to immediately try to re-attach if the debug session ends, using the same exact debug config.

Note however that this is not idiomatic in the Python ecosystem. That is, there's no generic nodemon-like tool that is idiomatically used in such circumstances. Instead, individual frameworks, such as Django and Flask, implement their own file system watchers with code reloading, as @karthiknadig described above. So, for the Python debugger, our support for such reloading amounts simply to supporting multiprocess Python applications, and letting the parent process restart the child processes as needed. If you implement your nodemon equivalent for Azure Functions in this way, then we will automatically support it.

[ejizba]

Is there a process that you have that restarts the server? if yes then is that process a python process? if yes then can that process itself be run under the debugger? if no then we need to understand the architecture in greater detail to provide a solution.

Our scenario is for debugging Azure Functions, which supports several languages (Python, C#, JavaScript, etc.). The core runtime is built in .NET Core and that is what handles managing and/or restarting the various language-specific processes (what they call "workers"). So no, I don't think we can just attach to a "root" process because it is not implemented in Python.

So if I'm understanding the feature request correctly, it's a debug configuration property that would cause the debugger to immediately try to re-attach if the debug session ends, using the same exact debug config.

Yes, but when you say "if the debug session ends" the Node.js debugger seems to know the difference between "the debug port unexpectedly closed" vs "the user manually detached" and will not try to reattach in the latter scenario.

If there is no main process that serves as a root (with debugger running in it), then it becomes difficult to know when the new processes gets spawned.

As @int19h mentioned, we're focused specifically on the attach scenario. I expect the same timeout/waiting logic would apply for the first attach vs. the second attach.

[int19h]

I had a look at how the Node debug adapter does this. It seems to boil down to using "restart" in the "terminated" event, the presence of which causes VSCode to restart the session and pass the value of "restart" in the new debug config as "__restart". And the adapter only does it if the session ends by itself, and not if it receives a "terminate" request.

It sounds like something we could implement fairly easily in debugpy following the same pattern.

[karthiknadig]

Moving this to debugpy, since this is can be entirely handled there.