Skip to content

tcp: transition to FIN_WAIT1 when FIN is queued#13401

Merged
copybara-service[bot] merged 1 commit into
google:masterfrom
tanyifeng:fin_state_trans
Jun 15, 2026
Merged

tcp: transition to FIN_WAIT1 when FIN is queued#13401
copybara-service[bot] merged 1 commit into
google:masterfrom
tanyifeng:fin_state_trans

Conversation

@tanyifeng

@tanyifeng tanyifeng commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Fix #13393

gVisor performs the close-state transition (ESTABLISHED -> FIN_WAIT1, CLOSE_WAIT -> LAST_ACK) in the sender, at the moment the FIN segment is actually transmitted. Because the FIN is queued as a separate segment behind any pending data, a full send window keeps it from ever being sent: the transition never happens and the endpoint stays in ESTABLISHED forever, so a peer that actively closes the connection (e.g. a slow consumer hit by a write deadline) is never observed as closed.

Linux instead transitions at enqueue time: tcp_close_state() moves the socket to FIN_WAIT1 before tcp_send_fin(), and the FIN may remain queued until the window opens.

Move the transition out of the sender and into shutdownLocked(), performed the moment the FIN is enqueued. The queued FIN still flushes normally once the window opens (no data loss), and the connection advances through the closing state machine even when the FIN cannot be sent yet.

@tanyifeng tanyifeng force-pushed the fin_state_trans branch 2 times, most recently from 9d98248 to 75d607d Compare June 10, 2026 02:16
@ayushr2 ayushr2 requested a review from nybidari June 10, 2026 02:19
copybara-service Bot pushed a commit that referenced this pull request Jun 10, 2026
Fix #13393

gVisor performs the close-state transition (ESTABLISHED -> FIN_WAIT1, CLOSE_WAIT -> LAST_ACK) in the sender, at the moment the FIN segment is actually transmitted. Because the FIN is queued as a separate segment behind any pending data, a full send window keeps it from ever being sent: the transition never happens and the endpoint stays in ESTABLISHED forever, so a peer that actively closes the connection (e.g. a slow consumer hit by a write deadline) is never observed as closed.

Linux instead transitions at enqueue time: tcp_close_state() moves the socket to FIN_WAIT1 before tcp_send_fin(), and the FIN may remain queued until the window opens.

Move the transition out of the sender and into shutdownLocked(), performed the moment the FIN is enqueued. The queued FIN still flushes normally once the window opens (no data loss), and the connection advances through the closing state machine even when the FIN cannot be sent yet.

FUTURE_COPYBARA_INTEGRATE_REVIEW=#13401 from tanyifeng:fin_state_trans 75d607d
PiperOrigin-RevId: 929909275
copybara-service Bot pushed a commit that referenced this pull request Jun 10, 2026
Fix #13393

gVisor performs the close-state transition (ESTABLISHED -> FIN_WAIT1, CLOSE_WAIT -> LAST_ACK) in the sender, at the moment the FIN segment is actually transmitted. Because the FIN is queued as a separate segment behind any pending data, a full send window keeps it from ever being sent: the transition never happens and the endpoint stays in ESTABLISHED forever, so a peer that actively closes the connection (e.g. a slow consumer hit by a write deadline) is never observed as closed.

Linux instead transitions at enqueue time: tcp_close_state() moves the socket to FIN_WAIT1 before tcp_send_fin(), and the FIN may remain queued until the window opens.

Move the transition out of the sender and into shutdownLocked(), performed the moment the FIN is enqueued. The queued FIN still flushes normally once the window opens (no data loss), and the connection advances through the closing state machine even when the FIN cannot be sent yet.

FUTURE_COPYBARA_INTEGRATE_REVIEW=#13401 from tanyifeng:fin_state_trans 75d607d
PiperOrigin-RevId: 929909275
Fix google#13393

gVisor performs the close-state transition (ESTABLISHED -> FIN_WAIT1,
CLOSE_WAIT -> LAST_ACK) in the sender, at the moment the FIN segment is
actually transmitted. Because the FIN is queued as a separate segment
behind any pending data, a full send window keeps it from ever being
sent: the transition never happens and the endpoint stays in
ESTABLISHED forever, so a peer that actively closes the connection
(e.g. a slow consumer hit by a write deadline) is never observed as
closed.

Linux instead transitions at enqueue time: tcp_close_state() moves the
socket to FIN_WAIT1 before tcp_send_fin(), and the FIN may remain queued
until the window opens.

Move the transition out of the sender and into shutdownLocked(),
performed the moment the FIN is enqueued. The queued FIN still flushes
normally once the window opens (no data loss), and the connection
advances through the closing state machine even when the FIN cannot be
sent yet.

Signed-off-by: Tan Yifeng <yiftan@tencent.com>
@tanyifeng

Copy link
Copy Markdown
Contributor Author

I noticed two CI failures on pipeline_build_43234node-nodejs-runtime-tests and java-java-runtime-tests.

Turns out the state-transition move broke an assumption in the receiver. consumeSegment checks ackNumber == SND.NXT to tell if our FIN was acknowledged, which only works because the old code guaranteed that by FIN_WAIT1/LAST_ACK the FIN had already been sent. Now the FIN can sit queued behind blocked data while the endpoint is already in the closing state — so a normal data ACK (matching SND.NXT without the FIN byte) gets mistaken for a FIN ACK.

That triggers two things:

  • LAST_ACK → CLOSE with the FIN still queued → peer never sees EOF → node connection-leak test times out.
  • FIN_WAIT1 → FIN_WAIT2/TIME_WAIT with the FIN unsent → peer gets ECONNRESET.

The fix is small: track whether the FIN was actually transmitted with a finSent flag on the sender, set it in maybeSendSegment where we used to do the state transition. Then guard both those ACK checks with r.ep.snd.finSent &&. Both CI cases pass in my local tests now.

@nybidari Could you please take another look and confirm the approach looks reasonable? Thanks!

copybara-service Bot pushed a commit that referenced this pull request Jun 11, 2026
Fix #13393

gVisor performs the close-state transition (ESTABLISHED -> FIN_WAIT1, CLOSE_WAIT -> LAST_ACK) in the sender, at the moment the FIN segment is actually transmitted. Because the FIN is queued as a separate segment behind any pending data, a full send window keeps it from ever being sent: the transition never happens and the endpoint stays in ESTABLISHED forever, so a peer that actively closes the connection (e.g. a slow consumer hit by a write deadline) is never observed as closed.

Linux instead transitions at enqueue time: tcp_close_state() moves the socket to FIN_WAIT1 before tcp_send_fin(), and the FIN may remain queued until the window opens.

Move the transition out of the sender and into shutdownLocked(), performed the moment the FIN is enqueued. The queued FIN still flushes normally once the window opens (no data loss), and the connection advances through the closing state machine even when the FIN cannot be sent yet.

FUTURE_COPYBARA_INTEGRATE_REVIEW=#13401 from tanyifeng:fin_state_trans bcb259a
PiperOrigin-RevId: 929909275
@ayushr2

ayushr2 commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the due diligence @tanyifeng! I have kicked a new run with your latest changes in https://buildkite.com/gvisor/pipeline/builds/43252.

copybara-service Bot pushed a commit that referenced this pull request Jun 11, 2026
Fix #13393

gVisor performs the close-state transition (ESTABLISHED -> FIN_WAIT1, CLOSE_WAIT -> LAST_ACK) in the sender, at the moment the FIN segment is actually transmitted. Because the FIN is queued as a separate segment behind any pending data, a full send window keeps it from ever being sent: the transition never happens and the endpoint stays in ESTABLISHED forever, so a peer that actively closes the connection (e.g. a slow consumer hit by a write deadline) is never observed as closed.

Linux instead transitions at enqueue time: tcp_close_state() moves the socket to FIN_WAIT1 before tcp_send_fin(), and the FIN may remain queued until the window opens.

Move the transition out of the sender and into shutdownLocked(), performed the moment the FIN is enqueued. The queued FIN still flushes normally once the window opens (no data loss), and the connection advances through the closing state machine even when the FIN cannot be sent yet.

FUTURE_COPYBARA_INTEGRATE_REVIEW=#13401 from tanyifeng:fin_state_trans bcb259a
PiperOrigin-RevId: 929909275
copybara-service Bot pushed a commit that referenced this pull request Jun 11, 2026
Fix #13393

gVisor performs the close-state transition (ESTABLISHED -> FIN_WAIT1, CLOSE_WAIT -> LAST_ACK) in the sender, at the moment the FIN segment is actually transmitted. Because the FIN is queued as a separate segment behind any pending data, a full send window keeps it from ever being sent: the transition never happens and the endpoint stays in ESTABLISHED forever, so a peer that actively closes the connection (e.g. a slow consumer hit by a write deadline) is never observed as closed.

Linux instead transitions at enqueue time: tcp_close_state() moves the socket to FIN_WAIT1 before tcp_send_fin(), and the FIN may remain queued until the window opens.

Move the transition out of the sender and into shutdownLocked(), performed the moment the FIN is enqueued. The queued FIN still flushes normally once the window opens (no data loss), and the connection advances through the closing state machine even when the FIN cannot be sent yet.

FUTURE_COPYBARA_INTEGRATE_REVIEW=#13401 from tanyifeng:fin_state_trans bcb259a
PiperOrigin-RevId: 929909275
copybara-service Bot pushed a commit that referenced this pull request Jun 15, 2026
Fix #13393

gVisor performs the close-state transition (ESTABLISHED -> FIN_WAIT1, CLOSE_WAIT -> LAST_ACK) in the sender, at the moment the FIN segment is actually transmitted. Because the FIN is queued as a separate segment behind any pending data, a full send window keeps it from ever being sent: the transition never happens and the endpoint stays in ESTABLISHED forever, so a peer that actively closes the connection (e.g. a slow consumer hit by a write deadline) is never observed as closed.

Linux instead transitions at enqueue time: tcp_close_state() moves the socket to FIN_WAIT1 before tcp_send_fin(), and the FIN may remain queued until the window opens.

Move the transition out of the sender and into shutdownLocked(), performed the moment the FIN is enqueued. The queued FIN still flushes normally once the window opens (no data loss), and the connection advances through the closing state machine even when the FIN cannot be sent yet.

FUTURE_COPYBARA_INTEGRATE_REVIEW=#13401 from tanyifeng:fin_state_trans bcb259a
PiperOrigin-RevId: 929909275
@copybara-service copybara-service Bot merged commit f6e2c99 into google:master Jun 15, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TCP connection stuck in ESTABLISHED after close() when FIN is blocked by pending data

3 participants