fix: release reference to initial interpreter shell after it is shut down#3031
Open
He-Pin wants to merge 1 commit into
Open
fix: release reference to initial interpreter shell after it is shut down#3031He-Pin wants to merge 1 commit into
He-Pin wants to merge 1 commit into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Releases the ActorGraphInterpreter's reference to its initial GraphInterpreterShell after preStart finishes initializing it, so that when the actor outlives the initial shell (e.g. while hosting other subfused interpreters), the initial shell and its logics can be garbage collected. Also fixes a debug log line that was printing the initial shell instead of the shell being registered.
Changes:
- Convert
_initialfrom a constructorvalto aprivate varand null it out at the end ofpreStartaftertryInit. - Update the
tryInitdebug println to reference theshellparameter rather than_initial. - Add two tests in
ActorGraphInterpreterSpecintended to cover the initial-shell-release behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| stream/src/main/scala/org/apache/pekko/stream/impl/fusing/ActorGraphInterpreter.scala | Makes _initial a mutable field, clears it after init, and fixes the registration debug log to use the registered shell. |
| stream-tests/src/test/scala/org/apache/pekko/stream/impl/fusing/ActorGraphInterpreterSpec.scala | Adds two specs aiming to verify the initial-shell reference release and continued operation with additional interpreters. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+439
to
+480
| "release reference to initial interpreter shell after it completes" in { | ||
| val mat = Materializer(system) | ||
| val initialShellCompleted = TestLatch(1) | ||
|
|
||
| // Create a stage that tracks when it completes | ||
| val trackingStage = new SimpleLinearGraphStage[String] { | ||
| override def createLogic(inheritedAttributes: Attributes): GraphStageLogic = new GraphStageLogic(shape) { | ||
| setHandler(in, | ||
| new InHandler { | ||
| override def onPush(): Unit = push(out, grab(in)) | ||
| override def onUpstreamFinish(): Unit = { | ||
| complete(out) | ||
| initialShellCompleted.countDown() | ||
| } | ||
| }) | ||
| setHandler(out, | ||
| new OutHandler { | ||
| override def onPull(): Unit = pull(in) | ||
| }) | ||
| } | ||
| } | ||
|
|
||
| val upstream = TestPublisher.probe[String]() | ||
| val downstream = TestSubscriber.probe[String]() | ||
|
|
||
| Source.fromPublisher(upstream).via(trackingStage).to(Sink.fromSubscriber(downstream)).run()(mat) | ||
|
|
||
| downstream.request(1) | ||
| upstream.sendNext("test") | ||
| downstream.expectNext("test") | ||
|
|
||
| // Complete the upstream to trigger stage completion | ||
| upstream.sendComplete() | ||
| downstream.expectComplete() | ||
|
|
||
| // Wait for the initial shell to complete | ||
| Await.ready(initialShellCompleted, remainingOrDefault) | ||
|
|
||
| // The actor should have stopped since there's only one interpreter shell | ||
| // and it has completed | ||
| // This verifies that the initial shell reference is released and the actor can stop | ||
| } |
Comment on lines
+482
to
+509
| "not keep initial shell alive when actor hosts additional subfused interpreters" in { | ||
| val mat = Materializer(system) | ||
| val firstStreamDone = Promise[Done]() | ||
| val secondStreamDone = Promise[Done]() | ||
|
|
||
| // Create a flow that we'll use twice through the same materializer | ||
| val identityFlow = Flow[Int].map(identity) | ||
|
|
||
| // Run first stream and complete it | ||
| val sub1 = TestSubscriber.probe[Int]() | ||
| Source(1 to 10).via(identityFlow).to(Sink.fromSubscriber(sub1)).run()(mat) | ||
| sub1.request(10) | ||
| sub1.expectNextN(1 to 10) | ||
| sub1.expectComplete() | ||
| firstStreamDone.success(Done) | ||
|
|
||
| // Run second stream - this should work even though the first stream's | ||
| // initial shell has been completed | ||
| val sub2 = TestSubscriber.probe[Int]() | ||
| Source(20 to 30).via(identityFlow).to(Sink.fromSubscriber(sub2)).run()(mat) | ||
| sub2.request(11) | ||
| sub2.expectNextN(20 to 30) | ||
| sub2.expectComplete() | ||
| secondStreamDone.success(Done) | ||
|
|
||
| Await.result(firstStreamDone.future, remainingOrDefault) | ||
| Await.result(secondStreamDone.future, remainingOrDefault) | ||
| } |
…down Motivation: ActorGraphInterpreter retains a strong reference to its initial GraphInterpreterShell via the _initial constructor val. When the actor outlives the initial shell (e.g., while hosting subfused interpreters registered via registerShell), the initial shell and all its logics cannot be garbage collected for the lifetime of the actor. Modification: - Convert _initial from a constructor val to a private var - Set _initial = null at the end of preStart() after tryInit completes - Fix debug println in tryInit to reference the shell parameter instead of _initial (which may be null after preStart) - Add behavioral test using flatMapConcat which triggers subfusing via SubFusingActorMaterializerImpl, verifying the actor continues processing subfused shells after the initial shell is released Result: The initial GraphInterpreterShell and its stage logics become eligible for garbage collection once the initial shell shuts down, reducing memory retention in long-lived ActorGraphInterpreter actors. Tests: - stream-tests/testOnly ActorGraphInterpreterSpec: 12/12 passed References: None - memory leak fix
He-Pin
force-pushed
the
fix/actor-graph-interpreter-initial-shell-leak
branch
from
bc40a16 to
6bc0142
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
An ActorGraphInterpreter starts with an initial shell but can host additional ones created using the subfusing materializer. The lifetime of ActorGraphInterpreter is not tied to the lifetime of the initial interpreter. However, a reference to the initial GraphInterpreterShell is kept around in the _initial constructor parameter, which can lead to unexpected memory usage because the initial GraphInterpreter and all its logics might be kept referenced much longer than expected.
This issue was originally reported in akka/akka-core#23535.
Modification
Result
ActorGraphInterpreter no longer keeps a reference to the initial interpreter shell after it has been shut down, allowing it and its logics to be garbage collected even when the actor is still alive hosting other subfused interpreters.
Tests
References