[ZEPPELIN-6555] Avoid deadlock in ManagedInterpreterGroup.close() by not holding the group lock while closing session interpreters

[hyunw9]

What is this PR for?

ManagedInterpreterGroup.close() can deadlock with a concurrent RemoteInterpreter.open() because the two paths take the same two monitors in the opposite order.

Two monitors are involved: the interpreter-group monitor and an individual interpreter's monitor.

• open() takes the interpreter monitor first (synchronized(this)) and then the group monitor (via getOrCreateSession() and the angular-registry push). Order: interpreter → group.
• close(String) is a synchronized method, so it holds the group monitor while it spawns the per-interpreter close threads and join()s them. Each close thread runs interpreter.close(), which takes the interpreter monitor. Order: group → interpreter.

So when an interpreter is opened while its session is concurrently closed (for example, restarting or shutting down an interpreter while a paragraph on that session is still starting up), the two orders form a circular wait.

What type of PR is it?

Bug Fix

Todos

• Stop holding the group monitor while closing session interpreters in close(String)
• Keep only the session-map removal and last-session teardown under the lock
• Add regression test ManagedInterpreterGroupTest#close_doesNotDeadlockWithConcurrentOpen

What is the Jira issue?

[ZEPPELIN-6555]

How should this be tested?

• Added a deterministic regression test,ManagedInterpreterGroupTest#close_doesNotDeadlockWithConcurrentOpen.

Screenshots (if appropriate)

Questions:

• Does the license files need to update? - No
• Is there breaking changes for older versions? - No
• Does this needs documentation? - No