Add CROSS_ORIGIN_STORAGE link flag#27066
Conversation
sbc100
left a comment
sbc100
left a comment
There was a problem hiding this comment.
I'm a little concerned about adding more Module properties here.
For the Module['wasmSHA256'], how about we just don't initially support custom instantiateWasm here? Avoid the need to document this or add this new property?
Regarding the 3 new callback.. are they really needed by users? Or is it mostly useful for testing?
|
Thanks for the great review feedback, will address tomorrow! Just a quick reaction to the below:
Custom
It was mostly for testing, though may be useful in general as well for module owners to easily report on COS stats in their analytics?! Is there a way to have those just for testing? |
You could put them behind One issue we have historically had with these module properties is they unconditionally took up space in the output file, even for users who didn't use them. So we created In this case (and for all new settings), I think its best not to enable them by default but add them to |
tomayac
left a comment
tomayac
left a comment
There was a problem hiding this comment.
Dropped =1 from all -sCROSS_ORIGIN_STORAGE references throughout the guide.
|
Alright, this was intense, but I hope to have addressed your review comments satisfyingly, @sbc100! The module properties are all opt-in now, except for the one with the hash, which makes it more straight-forward to roll your own instantiation. I've also managed to add real browser tests that install the extension for testing, so we can actually be more confident that the flag works. PTAL! Thanks |
sbc100
left a comment
sbc100
left a comment
There was a problem hiding this comment.
Looking much better!
The amount of testing in test_other.py is perhaps still a little excessive. I'm not normally in the position of asking folks to add fewer tests but... I guess I am. @kripken WDYT? Perhaps they could be condenses / reduced somehow?
sbc100
left a comment
sbc100
left a comment
There was a problem hiding this comment.
lgtm!
I do have a qestion around when we might be able to remove the polyfil, and a slight concert about shipping features that are supported in zero browser.
But since these are tagged as experimental it should be easy enough to rib them out if that feature doesn't pan out I guess?
|
As this is an opt-in and marked as experimental, most developers will likely never touch it. At this stage, this is meant for "white glove" partners we may want to work with directly, like SQLite Wasm (and also see ffmpegwasm/ffmpeg.wasm#940), and of course interested power users. For availability and a feature bug to track, keep an eye on ChromeStatus for current developer interest. (You're in really good company.) We know that we will have a chicken and egg problem rolling this out, so the sooner we start, the higher the chance of cache hits once it lands, and developers will need lead time. For early adopters, there's the Cross-Origin Storage extension that accurately implements the proposed API today. (We use a similar approach with WebMCP, where the Model Context Tool Inspector extension mimics an actual agent's interactions.) Thanks for the reviews again! Really appreciate you taking the time! I was really out of my comfort zone most of the time, so thanks for being my patient guard rails! |
|
To get the codesize tests passing you will want to do |
|
Alright, hope this did the trick. CI running. |
|
Ah, it looks like main is actually out-of-date now. I will update that now, then you will need to rebaseline once again I'm afraid. Sorry this part is a little annoying. |
|
Branch updated, CI running, fingers crossed. |
|
Sorry, we had some CI issues. See #27110. If you rebase I think we are good to land. |
SIDE_MODULE is now a hard error (no JS glue emitted — genuinely incompatible). SPLIT_MODULE and MAIN_MODULE partial-coverage warnings are dropped; they are not true incompatibilities and add noise for an experimental feature. Per reviewer feedback (r3383144746).
- Drop -sENVIRONMENT=web (web is included by default) - Drop inline comment and docstrings - Drop section banner comment Per reviewer feedback.
- Use globalThis.navigator?.crossOriginStorage (matches codebase pattern) - Move var cosHash inside the if block - Hardcode 'SHA-256' in the template; drop <<< WASM_HASH_ALGORITHM >>> placeholder Per reviewer feedback.
Collapse the NotFoundError / NotAllowedError / else branches into a single bare catch that always falls back to a network fetch and attempts a fire-and-forget store into COS. This makes the code robust against any future renaming of the error (see WICG/cross-origin-storage#62).
- Remove duplicate test_deprecated_settings definition in test_other.py - Fix extra blank line before test_cross_origin_storage_fallback in test_browser.py - Fix extra blank line after WASM_BINDGEN block in tools/link.py
This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (24) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_codesize_cxx_ctors1.json: 151801 => 151847 [+46 bytes / +0.03%] codesize/test_codesize_cxx_ctors2.json: 151204 => 151250 [+46 bytes / +0.03%] codesize/test_codesize_cxx_except.json: 195695 => 195741 [+46 bytes / +0.02%] codesize/test_codesize_cxx_except_wasm.json: 166925 => 166971 [+46 bytes / +0.03%] codesize/test_codesize_cxx_except_wasm_legacy.json: 164809 => 164855 [+46 bytes / +0.03%] codesize/test_codesize_cxx_lto.json: 120628 => 120689 [+61 bytes / +0.05%] codesize/test_codesize_cxx_mangle.json: 262174 => 262220 [+46 bytes / +0.02%] codesize/test_codesize_cxx_noexcept.json: 153801 => 153847 [+46 bytes / +0.03%] codesize/test_codesize_cxx_wasmfs.json: 179676 => 179596 [-80 bytes / -0.04%] codesize/test_codesize_files_wasmfs.json: 63873 => 63677 [-196 bytes / -0.31%] codesize/test_codesize_hello_O0.json: 38629 => 38687 [+58 bytes / +0.15%] codesize/test_codesize_hello_dylink_all.json: 855713 => 855762 [+49 bytes / +0.01%] codesize/test_codesize_minimal_O0.json: 19682 => 19740 [+58 bytes / +0.29%] codesize/test_minimal_runtime_code_size_audio_worklet.json: 16401 => 16399 [-2 bytes / -0.01%] codesize/test_minimal_runtime_code_size_hello_embind.json: 14903 => 14904 [+1 bytes / +0.01%] codesize/test_minimal_runtime_code_size_hello_embind_val.json: 11642 => 11643 [+1 bytes / +0.01%] codesize/test_minimal_runtime_code_size_hello_wasm_worker.json: 4116 => 4128 [+12 bytes / +0.29%] codesize/test_minimal_runtime_code_size_hello_webgl2_wasm.json: 13161 => 13161 [+0 bytes / +0.00%] codesize/test_minimal_runtime_code_size_hello_webgl2_wasm2js.json: 18531 => 18534 [+3 bytes / +0.02%] codesize/test_minimal_runtime_code_size_hello_webgl_wasm.json: 12699 => 12699 [+0 bytes / +0.00%] codesize/test_minimal_runtime_code_size_hello_webgl_wasm2js.json: 18057 => 18060 [+3 bytes / +0.02%] codesize/test_minimal_runtime_code_size_random_printf_wasm.json: 11052 => 11057 [+5 bytes / +0.05%] codesize/test_minimal_runtime_code_size_random_printf_wasm2js.json: 17413 => 17417 [+4 bytes / +0.02%] codesize/test_unoptimized_code_size.json: 176861 => 177085 [+224 bytes / +0.13%] Average change: +0.04% (-0.31% - +0.29%) ```
…e.js - ChangeLog.md: remove duplicate RST-format entry and bad-merge artifact; keep single clean Markdown entry for -sCROSS_ORIGIN_STORAGE - src/settings.js: reflow CROSS_ORIGIN_STORAGE comment so first line is not shorter than the rest - src/preamble.js: drop redundant .length === 1 check in origins #if (Python link-time validation already prevents '*' from being mixed with explicit origins, so [0] === '*' implies length === 1)
If CROSS_ORIGIN_STORAGE is set and we reach this point in the link but wasm_target does not exist, crashing is the correct behavior rather than silently skipping the hash substitution.
Default ENVIRONMENT is empty which means ENVIRONMENT_MAY_BE_WEB=True, so the explicit flag adds no value in these error-path tests.
…cmds Default ENVIRONMENT already includes web, so the flag is redundant. Condense the two single-flag run_process calls into one line each.
This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (2) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_codesize_cxx_wasmfs.json: 179722 => 179596 [-126 bytes / -0.07%] codesize/test_codesize_hello_dylink_all.json: 855768 => 855762 [-6 bytes / -0.00%] Average change: -0.04% (-0.07% - -0.00%) ```
tomayac
force-pushed
the
cross-origin-storage
branch
from
0ee4946 to
57e1ec4
Compare
|
Amazing, it's merged 🥳! Thanks for making it happen, @sbc100, really excited to see what this will unlock in the future! Appreciate a lot all the help in getting this landed! |
3 participants
Integrates the WICG Cross-Origin Storage API into Emscripten's standard Wasm loading path as a progressive enhancement.
When
-sCROSS_ORIGIN_STORAGE=1is set at link time, Emscripten computes the SHA-256 hash of the final.wasmbinary and embeds it as a build-time constant. At runtime the generated JavaScript tries to retrieve the module from the cross-origin cache before falling back to the normalfetch()/WebAssembly.instantiateStreaming()path, so pages always load regardless of browser support.New settings
-sCROSS_ORIGIN_STORAGE=1— enables the feature (Web target only, disabled by default).-sCROSS_ORIGIN_STORAGE_ORIGINS— controls which origins can read the cached file:'*'(default, globally available), an explicit HTTPS origin list, or[](same-site only).Because no browser ships the API natively yet, testing requires the COS Chrome extension (source), which polyfills
navigator.crossOriginStorageon every page.Example
Without the COS API
Loading goes through the normal
fetch()/WebAssembly.instantiateStreaming()path:With the COS API
Now with the COS Chrome extension installed, see its popup window for details.
First time load
Loading goes through the normal
fetch()/WebAssembly.instantiateStreaming()path, but the resource is stored in COS for the next time:Repeated load
Loading now goes through the COS path:
Loading on a different origin
Loading now goes through the COS path and the resource is shared across origins: