Famedly release v1.153#260
Merged
Merged
Conversation
…nt and profile updates are disabled (#19398) Currently synapse returns `M_FORBIDDEN` when trying to use the account deactivation API, if the server admin disabled displayname changes. This is undesirable, since it prevents GDPR erasure without admin interaction. The admin API seems to work fine though. This also only seems to affect the deactivate API, when the erase flag is true. Relevant endpoint: https://spec.matrix.org/latest/client-server-api/#post_matrixclientv3accountdeactivate This change only removes the checked for condition that the displayname and profile avatar are allowed to be changed per the configuration setting. If a user is deleting themselves, why is that denied? There did not seem to be a basic test for this endpoint that checks the `erase` usage, so that was added as well as checking the above mentioned behavior.
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Andrew Morgan <andrew@amorgan.xyz>
…re new results to sync (#19714) This fixes the bug described in #19713 (and double-checked against the SDK integration test, which now passes with this change). A sync response must be returned immediately if a room subscription configuration change caused a new non-empty response (checked with `if response` in the code) to be produced. Fixes #19713. Fixes #18844. --------- Co-authored-by: Erik Johnston <erik@matrix.org>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
MSC3266 is merged in v1.15, let's stabilize it as part of #18731 1. Add support for the stable `/_matrix/client/v1/room_summary/` endpoint, keeping both unstable endpoints for compat 2. Remove the experimental `msc3266_enabled` flag
…19710) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
When upgrading a room to v12, we accidentally ended up mutating the content of the old power level. Since we cache events, this meant any future usage of the old power level event would see the wrong content (until it dropped from the cache). This meant that the creator of the new room would not be able to perform admin actions on the old room. Any federation requests for the event would fail the hash checks, since the content had been changed. All in all, quite a nasty bug.
…(#19711) Known problems: element-hq/synapse#18117 As a follow-up, we should consider removing this config option altogether. It's "expensive" and claims to "prevent bugs" but actually introduces a whole new class of bugs. It could be re-introduced with a more holistic solution to the typing. Or a completely new approach (safe mode that blows up when someone mutates the event content, always make deep clones when handing out references, etc) The `use_frozen_dict` config option was there [since inception](element-hq/synapse@a7b65bd) but was only recently [documented](element-hq/synapse#18122) for completeness sake.
Implements: [MSC4163: Make ACLs apply to EDUs](matrix-org/matrix-spec-proposals#4163) Part of #18118 to declare support for Matrix v1.13 Complement PR: ~~matrix-org/complement#783 -> matrix-org/complement#862 --------- Co-authored-by: Eric Eastwood <erice@element.io> Co-authored-by: Quentin Gliech <quenting@element.io>
Part of MSC4311: invites and knocks should contain the create event (stripped state for the client API) Part of element-hq/synapse#19414
This comes from https://github.com/erikjohnston/rust-signed-json/blob/main/src/json.rs. We need to be able to serialise canonical JSON in Rust to be able to calculate event IDs once we port the event class to Rust. We could instead make the above a properly published crate, but feels easier to pull it into Synapse utils.
The original commit should only have changed the lockfile. This reverts commit bdb1cf7 (from element-hq/synapse#19703). --------- Co-authored-by: Olivier 'reivilibre <oliverw@matrix.org>
…le. (#19743) See: - element-hq/synapse#19742 - element-hq/synapse#19686 (etc) Documentation https://docs.github.com/en/code-security/reference/supply-chain-security/dependabot-options-reference#versioning-strategy-- We were considering `lockfile-only` but it sounds like `increase-if-necessary` would increase the upper bound for us, if we had one. Let's try it. --------- Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
Exposes `tombstoned` and `replacement_room` in room details on admin API endpoint `GET /_synapse/admin/v1/rooms/<room_id>`. Resolves #18347
…19736) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
The juicy details and explanation are in the diff itself. Split out from element-hq/synapse#18873 in order to fix paginating from [MSC3871](matrix-org/matrix-spec-proposals#3871) gap tokens actually backfilling history. To be clear, this is a good change to make outside of the [MSC3871](matrix-org/matrix-spec-proposals#3871) use case. For example (as the new Complement test shows), fixes a problem where if you try to paginate `/messages` from tokens returned by `/context`, we could fail to backfill anything new and hide away history. Also fixes matrix-org/complement#853
Fixes the symptoms of element-hq/synapse#19315 / element-hq/synapse#19588 but not the underlying reason causing the number to grow so large in the first place. ``` ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit ``` Copied from the original pull request on [Famedly's Synapse repo](#221) (with some edits): Basing the time interval around a 5 seconds leaves a big window of waiting especially as this window is doubled each retry, when another worker could be making progress but can not. Right now, the retry interval in seconds looks like `[0.2, 5, 10, 20, 40, 80, 160, 320, (continues to double)]` after which logging should start about excessive times and (relatively quickly) end up with an extremely large retry interval with an unrealistic expectation past the heat death of the universe. 1 year in seconds = 31,536,000. With this change, retry intervals in seconds should look more like: ``` [ 0.2, 0.4, 0.8, 1.6, 3.2, 6.4, 12.8, 25.6, 51.2, 60, < never goes higher than this ] ``` Logging about excessive wait times will start at 10 minutes. <details> <summary>Previous breakdown when we were using 15 minutes</summary> ``` [ 0.2, 0.4, 0.8, 1.6, 3.2, 6.4, 12.8, 25.6, 51.2, 102.4, # 1.7 minutes 204.8, # 3.41 minutes 409.6, # 6.83 minutes 819.2, # 13.65 minutes < logging about excessive times will start here, 13th iteration 900, # 15 minutes < never goes higher than this ] ``` </details> Further suggested work in this area could be to define the cap, the retry interval starting point and the multiplier depending on how frequently this lock should be checked. See data below for reasons why. Increasing the jitter range may also be a good idea --------- Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
This is another stepping stone in porting the event class fully to Rust. The new `Signatures` class is relatively simple, as we actually don't interact with it that much in the code. It does *not* implement `Mapping` or `MutableMapping` as that takes quite a lot of effort that we don't need, even though it would be more ergonomic.
Similar to #19706, let's port the `unsigned` field into a Rust class. This does change things a bit in that we now define exactly what unsigned fields that are allowed to be added to an event, and what actually gets persisted. This should be a noop though, as we carefully filter out what unsigned fields we allow in from federation, for example As a side effect of this cleanup, I think this fixes handling `unsigned.age` on events received over federation.
Better to retry more quickly than have workers wait around. 5 seconds is still a reasonable gap in time to not overwhelm anything. This matters most in cross-worker scenarios. When locks are on the same worker, when the lock holder releases, we signal to other locks (with the same name/key) that they should try reacquiring the lock immediately. But locks on other workers only re-check based on their retry `_timeout_interval`. Updating to 5 seconds to match the previous intentions based on the [flawed code](https://github.com/element-hq/synapse/blob/6100f6e4f7fb0c72f1ae2802683ebc811c0e3a77/synapse/handlers/worker_lock.py#L278). We can assume they were trying to have 5 seconds as the max value to retry. Spawning from element-hq/synapse#19394 (comment)
So people have to specify which time unit they want to use. Spawning from element-hq/synapse#19394 (comment)
…son.rs`) (#19763) (introduced in element-hq/synapse#19739) Seems like some automatic change from `poetry run ./scripts-dev/lint.sh`
Handle arbitrary sized integers in `unsigned` (and other Rust objects that use `serde_json::Value`)
Since this is just a change log update, I've removed the entire checklist. Please tell me if this is incorrect.
… there are new results to sync (#19714)" (#19784) Reverts: #19714 Opens: #19783 Closes: element-hq/backend-internal#242 Related: #18880 (the performance problem that is aggravated by #19714) This reverts commit 2691d0b. --------- Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
FrenchGithubUser
force-pushed
the
famedly-release/v1.153
branch
from
f6fc8ca to
6a47a53
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #260 +/- ##
==========================================
+ Coverage 80.39% 80.41% +0.01%
==========================================
Files 501 501
Lines 72351 72403 +52
Branches 10906 10912 +6
==========================================
+ Hits 58168 58221 +53
- Misses 10903 10906 +3
+ Partials 3280 3276 -4
... and 2 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the Synapse fork for the v1.153.0 release, bringing in upstream changes around Rust-backed event structures, federation/backfill behavior, stabilized room summary API, and assorted config/docs/test updates.
Changes:
- Port
Event.signaturesandEvent.unsignedto Rust and adjust persistence/serialization paths accordingly. - Improve federation behavior (EDU ACL enforcement, more robust PDU parsing with
age→age_ts, and backfill-point selection near pagination tokens). - Release/admin/config updates (v1.153.0 version bumps, stabilized room summary endpoint, admin room details expose tombstone info, docs/tests refreshed).
Reviewed changes
Copilot reviewed 61 out of 64 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/synapse_rust/test_unsigned.py | Adds tests for Rust Unsigned handling (prev_content + large integers). |
| tests/rest/client/test_upgrade_room.py | Adds regression test for MSC4289 power-level handling during room upgrade. |
| tests/rest/client/test_sync.py | Adds tests ensuring m.room.create appears in invite/knock stripped state (MSC4311). |
| tests/rest/client/test_account.py | Adjusts account deactivation/erasure tests and adds helper docstring. |
| tests/rest/admin/test_room.py | Extends admin room details test to assert tombstone fields. |
| tests/handlers/test_profile.py | Renames tests for clarity around disabled profile endpoints. |
| tests/handlers/test_federation.py | Removes a backfill regression test and updates signatures mutation style. |
| tests/federation/test_federation_server.py | Updates tests for age handling (age_ts with received_time). |
| tests/events/test_utils.py | Updates unsigned-field tests to align with Rust Unsigned field set. |
| tests/config/test_api.py | Updates prejoin-state expectations to always include m.room.create (MSC4311). |
| synapse/util/duration.py | Makes Duration keyword-only via __new__. |
| synapse/synapse_rust/events.pyi | Adds stubs for Rust Signatures and Unsigned. |
| synapse/storage/databases/main/events.py | Persists events using get_dict_for_persistence. |
| synapse/storage/databases/main/events_worker.py | Avoids re-adding prev_* unsigned fields; mutates cached event unsigned when needed. |
| synapse/storage/databases/main/events_bg_updates.py | Uses Rust Signatures.get_signature API. |
| synapse/storage/databases/main/event_federation.py | Adjusts backfill point selection to use “nearby_depth” heuristic. |
| synapse/rest/client/room.py | Stabilizes room summary endpoint and registers servlet unconditionally. |
| synapse/rest/admin/rooms.py | Adds tombstone state-derived fields to admin room details response. |
| synapse/handlers/worker_lock.py | Reduces worker lock retry interval to 5s and documents behavior. |
| synapse/handlers/room.py | Avoids mutating old PL content when upgrading; copies/removes creators for MSC4289. |
| synapse/handlers/room_policy.py | Updates signature handling to Rust Signatures.update. |
| synapse/handlers/message.py | Updates invite signature merging to use as_dict(). |
| synapse/handlers/federation.py | Introduces backfill constant and uses nearby-depth padding. |
| synapse/federation/federation_server.py | Improves PDU parsing robustness; applies ACL checks to EDUs; deep-copies EDU content. |
| synapse/federation/federation_client.py | Passes received_time into federation event parsing. |
| synapse/federation/federation_base.py | Adds received_time plumbing and converts unsigned.age into age_ts. |
| synapse/events/utils.py | Types maybe_upsert_event_field to accept Rust Unsigned. |
| synapse/events/init.py | Wraps unsigned/signatures as Rust types; splits serialization vs persistence dict creation. |
| synapse/event_auth.py | Updates signature presence checks for Rust Signatures. |
| synapse/crypto/keyring.py | Raises early if a server has no signatures; uses Rust Signatures indexing. |
| synapse/crypto/event_signing.py | Uses Rust Signatures.get_signature helper. |
| synapse/config/experimental.py | Removes MSC3266 experimental flag (feature stabilized). |
| synapse/config/api.py | Ensures m.room.create is always included in prejoin state entries (MSC4311). |
| schema/synapse-config.schema.yaml | Bumps schema $id to v1.153 and documents frozen-dicts warning. |
| rust/src/room_versions.rs | Adjusts PyO3 annotations and Python interop for room version types. |
| rust/src/push/mod.rs | Updates PyO3 annotations for push rule types. |
| rust/src/lib.rs | Registers new Rust module for canonical JSON. |
| rust/src/events/unsigned.rs | Implements Rust-backed Unsigned with persisted vs in-memory fields. |
| rust/src/events/signatures.rs | Implements Rust-backed Signatures container and Python API. |
| rust/src/events/mod.rs | Registers new Rust event classes with the Python module. |
| rust/src/events/internal_metadata.rs | Adjusts PyO3 annotations for internal metadata. |
| rust/src/canonical_json.rs | Adds canonical JSON serializer with strict/relaxed integer handling. |
| rust/src/acl/mod.rs | Adjusts PyO3 annotations for ACL evaluator. |
| rust/Cargo.toml | Bumps PyO3/pythonize deps; enables serde_json arbitrary precision; adds itertools. |
| pyproject.toml | Bumps project version to 1.153.0 and adjusts some Python dependency constraints. |
| poetry.lock | Updates locked Python dependencies (attrs, pillow, multipart, etc.). |
| docs/usage/configuration/config_documentation.md | Adds warning about use_frozen_dicts comparison issues. |
| docs/admin_api/rooms.md | Documents new admin room tombstone fields and example response. |
| docker/complement/conf/workers-shared-extra.yaml.j2 | Removes MSC3266 flag from complement worker config. |
| debian/changelog | Adds 1.153.0 (and rc) changelog entries. |
| complement/tests/room_messages_test.go | Adds Complement test for pagination/backfill near backward extremities. |
| complement/go.sum | Updates Go dependency checksums. |
| complement/go.mod | Adds/updates Go dependencies used by complement tests. |
| CHANGES.md | Adds 1.153.0 release notes (features/bugfixes/internal changes). |
| Cargo.lock | Updates Rust lockfile for dependency changes (PyO3, pythonize, itertools, etc.). |
| .github/workflows/twisted_trunk.yml | Bumps GitHub Actions artifact upload action version. |
| .github/workflows/tests.yml | Bumps cache/artifact actions versions. |
| .github/workflows/release-artifacts.yml | Bumps cache/artifact actions versions. |
| .github/workflows/push_complement_image.yml | Bumps docker login action version. |
| .github/workflows/latest_deps.yml | Bumps artifact upload action version. |
| .github/workflows/docs-pr.yaml | Bumps artifact upload action version. |
| .github/workflows/docker.yml | Bumps docker login/build/signing actions versions. |
| .github/dependabot.yml | Adds Dependabot config including cooldowns and grouping strategies. |
| .ci/scripts/calculate_builds.py | Updates mod-pack build matrix to mod028/mod029. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Member
|
The Synapse Invite Checker tests failure is a known entity. The testing infrastructure needed an update to accommodate a new rust class. Update for this is pending: famedly/synapse-invite-checker#134 The fix was directed towards this release to verify it works, but is not used other than by testing code. It will be released at a future date and is not a road-block for this Synapse release. |
jason-famedly
approved these changes
May 27, 2026
jason-famedly
left a comment
jason-famedly
left a comment
Member
There was a problem hiding this comment.
Pretty happy with this. Assuming tests pass(except the invite checker which is a known failure, see above) let's ship it!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Famedly Synapse release v1.153.0_1
Docker image tags available:
v1.153.0_1-mod028<- TIM 1.1v1.153.0_1-mod029<- TIM Pro(and TIM 1.2)No Famedly additions for v1.153.0_1
Nothing note worthy for Famedly
Requires: famedly/complement#16