… fork to avoid joining parent process' threads (pythonGH-126098)

Threads are gone after fork, so clear the queues too. Otherwise the
child process (here created via multiprocessing.Process) crashes on
interpreter exit.

(cherry picked from commit 1848ce6)

Co-authored-by: Andrei Bodrov <Drino@users.noreply.github.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

… fork to avoid joining parent process' threads (pythonGH-126098)

Threads are gone after fork, so clear the queues too. Otherwise the
child process (here created via multiprocessing.Process) crashes on
interpreter exit.

(cherry picked from commit 1848ce6)

Co-authored-by: Andrei Bodrov <Drino@users.noreply.github.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…r fork to avoid joining parent process' threads (GH-126098) (GH-127163)

Threads are gone after fork, so clear the queues too. Otherwise the
child process (here created via multiprocessing.Process) crashes on
interpreter exit.

(cherry picked from commit 1848ce6)

Co-authored-by: Andrei Bodrov <Drino@users.noreply.github.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…r fork to avoid joining parent process' threads (GH-126098) (GH-127164)

Threads are gone after fork, so clear the queues too. Otherwise the
child process (here created via multiprocessing.Process) crashes on
interpreter exit.

(cherry picked from commit 1848ce6)

Co-authored-by: Andrei Bodrov <Drino@users.noreply.github.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

… fork to avoid joining parent process' threads (pythonGH-126098)

Threads are gone after fork, so clear the queues too. Otherwise the
child process (here created via multiprocessing.Process) crashes on
interpreter exit.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

What is the motivation for this PR?
To fix the ansible worker dead issue observed in sonic-mgmt test.
This issue is that the ansible worker is detected dead when calling ansible from thread worker in thread pool.
This is same as python/cpython#88110.

The root cause is that, concurrent.futures.thread thread pool registers a callback to poll the threads when python interpreter exits, and those thread workers are stored in the dictionary concurrent.futures.thread._threads_queues. The ansible forked child worker process will inherit this dictionary, which contains those orphaned threads. And when the ansible child worker process tries to exit, the callback polls those orphaned threads, causing the worker process returns 1 and ansible complains the worker dead.

How did you do it?
Backport the cpython fix python/cpython#126098 if the sonic-mgmt python version is < 3.12.8.

Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

What is the motivation for this PR?
To fix the ansible worker dead issue observed in sonic-mgmt test.
This issue is that the ansible worker is detected dead when calling ansible from thread worker in thread pool.
This is same as python/cpython#88110.

The root cause is that, concurrent.futures.thread thread pool registers a callback to poll the threads when python interpreter exits, and those thread workers are stored in the dictionary concurrent.futures.thread._threads_queues. The ansible forked child worker process will inherit this dictionary, which contains those orphaned threads. And when the ansible child worker process tries to exit, the callback polls those orphaned threads, causing the worker process returns 1 and ansible complains the worker dead.

How did you do it?
Backport the cpython fix python/cpython#126098 if the sonic-mgmt python version is < 3.12.8.

Signed-off-by: Longxiang Lyu <lolv@microsoft.com>
Signed-off-by: nnelluri-cisco <nnelluri@cisco.com>

What is the motivation for this PR?
To fix the ansible worker dead issue observed in sonic-mgmt test.
This issue is that the ansible worker is detected dead when calling ansible from thread worker in thread pool.
This is same as python/cpython#88110.

The root cause is that, concurrent.futures.thread thread pool registers a callback to poll the threads when python interpreter exits, and those thread workers are stored in the dictionary concurrent.futures.thread._threads_queues. The ansible forked child worker process will inherit this dictionary, which contains those orphaned threads. And when the ansible child worker process tries to exit, the callback polls those orphaned threads, causing the worker process returns 1 and ansible complains the worker dead.

How did you do it?
Backport the cpython fix python/cpython#126098 if the sonic-mgmt python version is < 3.12.8.

Signed-off-by: Longxiang Lyu <lolv@microsoft.com>
Signed-off-by: Zhuohui Tan <zhuohui.tan@amd.com>

What is the motivation for this PR?
To fix the ansible worker dead issue observed in sonic-mgmt test.
This issue is that the ansible worker is detected dead when calling ansible from thread worker in thread pool.
This is same as python/cpython#88110.

The root cause is that, concurrent.futures.thread thread pool registers a callback to poll the threads when python interpreter exits, and those thread workers are stored in the dictionary concurrent.futures.thread._threads_queues. The ansible forked child worker process will inherit this dictionary, which contains those orphaned threads. And when the ansible child worker process tries to exit, the callback polls those orphaned threads, causing the worker process returns 1 and ansible complains the worker dead.

How did you do it?
Backport the cpython fix python/cpython#126098 if the sonic-mgmt python version is < 3.12.8.

Signed-off-by: Longxiang Lyu <lolv@microsoft.com>
Signed-off-by: mssonicbld <sonicbld@microsoft.com>

What is the motivation for this PR?
To fix the ansible worker dead issue observed in sonic-mgmt test.
This issue is that the ansible worker is detected dead when calling ansible from thread worker in thread pool.
This is same as python/cpython#88110.

The root cause is that, concurrent.futures.thread thread pool registers a callback to poll the threads when python interpreter exits, and those thread workers are stored in the dictionary concurrent.futures.thread._threads_queues. The ansible forked child worker process will inherit this dictionary, which contains those orphaned threads. And when the ansible child worker process tries to exit, the callback polls those orphaned threads, causing the worker process returns 1 and ansible complains the worker dead.

How did you do it?
Backport the cpython fix python/cpython#126098 if the sonic-mgmt python version is < 3.12.8.

Signed-off-by: Longxiang Lyu <lolv@microsoft.com>
Signed-off-by: Mihut Aronovici <aronovic@cisco.com>

What is the motivation for this PR?
To fix the ansible worker dead issue observed in sonic-mgmt test.
This issue is that the ansible worker is detected dead when calling ansible from thread worker in thread pool.
This is same as python/cpython#88110.

The root cause is that, concurrent.futures.thread thread pool registers a callback to poll the threads when python interpreter exits, and those thread workers are stored in the dictionary concurrent.futures.thread._threads_queues. The ansible forked child worker process will inherit this dictionary, which contains those orphaned threads. And when the ansible child worker process tries to exit, the callback polls those orphaned threads, causing the worker process returns 1 and ansible complains the worker dead.

How did you do it?
Backport the cpython fix python/cpython#126098 if the sonic-mgmt python version is < 3.12.8.

Signed-off-by: Longxiang Lyu <lolv@microsoft.com>
Signed-off-by: Ravali Yeluri (WIPRO LIMITED) <v-ryeluri@microsoft.com>

What is the motivation for this PR?
To fix the ansible worker dead issue observed in sonic-mgmt test.
This issue is that the ansible worker is detected dead when calling ansible from thread worker in thread pool.
This is same as python/cpython#88110.

The root cause is that, concurrent.futures.thread thread pool registers a callback to poll the threads when python interpreter exits, and those thread workers are stored in the dictionary concurrent.futures.thread._threads_queues. The ansible forked child worker process will inherit this dictionary, which contains those orphaned threads. And when the ansible child worker process tries to exit, the callback polls those orphaned threads, causing the worker process returns 1 and ansible complains the worker dead.

How did you do it?
Backport the cpython fix python/cpython#126098 if the sonic-mgmt python version is < 3.12.8.

Signed-off-by: Longxiang Lyu <lolv@microsoft.com>
Signed-off-by: Abhishek <abhishek@nexthop.ai>