Fixed-length lists, host side#12315
Conversation
Subscribe to Label Actioncc @fitzgen DetailsThis issue or pull request has been labeled: "fuzzing", "wasmtime:api", "wasmtime:c-api", "wasmtime:config"Thus the following users have been cc'd because of the following labels:
To subscribe or unsubscribe from this label, edit the |
Label Messager: wasmtime:configIt looks like you are changing Wasmtime's configuration options. Make sure to
DetailsTo modify this label's message, edit the To add new label messages or remove existing label messages, edit the |
|
TODO: Check whether the standard name fixed-length lists is used over fixed size lists. |
|
What's remaining for this? |
I didn't find the time to add a proper test case. And I (non-blockingly) wait for a new wasm-tools release to do the renaming in one row. |
8d18537 to
9157634
Compare
cpetig
left a comment
There was a problem hiding this comment.
I am not sure whether the O(n) constraint applies to the host side as the user always controls the data types (and dimensions) added on the host side, but if so I will gladly investigate into a different solution.
| /// Returns the abi for a fixed length list | ||
| pub const fn fixed_length_list_static( | ||
| element: &CanonicalAbiInfo, | ||
| count: u32, | ||
| ) -> CanonicalAbiInfo { | ||
| CanonicalAbiInfo { | ||
| size32: element.size32 * count, | ||
| align32: element.align32, | ||
| size64: element.size64 * count, | ||
| align64: element.align64, | ||
| flat_count: match element.flat_count { | ||
| None => None, | ||
| Some(c) => Some(c.saturating_mul(if count > 255 { 255u8 } else { count as u8 })), | ||
| }, | ||
| } | ||
| } |
There was a problem hiding this comment.
I've got some miscellaneous concerns about this function, so I'm going to list them here:
- This is more-or-less duplicated with
new_fixed_length_list_typeintypes_builder.rs. Could that call this function (or a similar signature?) - I'm worried about the unchecked multiplication here because the host could declare
[SomeBigStruct; HUGE_NUMBER]which exceeds 4GiB of data, and I'd want that to be a panic/error/something as opposed to the silent wrap here. - I think the
saturating_mulhere should bechecked_mul, and I also don't think the conversion here tou8is right. Ifcis 1 andcountis 256 thenflat_countshould beNone, notSome(255). - I'm a little worried that in Rust below we've got
N: usizewhere the parametercount: u32has a different type here. Could there be a separate entrypoint which takescount: usizeand performs a fallible/panicking conversion asserting that thecountis less thanu32::MAX?
There was a problem hiding this comment.
I thought about how to remove the duplication but this gets more difficult because one of the functions is const (this also makes the implementation less elegant as traits (and_then, try_from) don't work) and I didn't identify an obvious common dependency which is not core but component model specific.
I think I addressed all other concerns.
| // this is a reimplementation of array::try_from_fn, replace once that became stable | ||
| fn array_try_from_fn<E, F, T, const N: usize>(mut cb: F) -> Result<[T; N], E> | ||
| where | ||
| F: FnMut(usize) -> Result<T, E>, | ||
| { | ||
| let mut valid = 0; | ||
| let mut result: [MaybeUninit<T>; N] = [const { MaybeUninit::uninit() }; N]; | ||
| let mut error: Option<E> = None; | ||
| for n in 0..N { | ||
| match cb(n) { | ||
| Ok(v) => { | ||
| result[valid].write(v); | ||
| valid += 1; | ||
| } | ||
| Err(e) => { | ||
| error = Some(e); | ||
| // continue to consume all input | ||
| } | ||
| } | ||
| } | ||
| if let Some(e) = error { | ||
| // on error drop all valid elements | ||
| for n in 0..valid { | ||
| unsafe { | ||
| result[n].assume_init_drop(); | ||
| } | ||
| } | ||
| return Err(e); | ||
| } | ||
| assert!(valid == N); | ||
| // this is a copy of array_assume_init from stdlib to avoid requiring nightly | ||
| Ok(unsafe { (&result as *const _ as *const [T; N]).read() }) | ||
| } |
There was a problem hiding this comment.
Could this be moved to crates/core/src/array.rs so this could get miri-testing there, and additionally enable adding #[test] for this? Additionally, this doesn't currently handle panics of cb (leaks intermediate results) and a few more casts than I think are necessary.
I sketched out what I'm thinking here and feel free to copy that into this PR
There was a problem hiding this comment.
Sadly that implementation is only possible with 1.95 up (AsMut on Uninit array, see https://doc.rust-lang.org/beta/core/mem/union.MaybeUninit.html#impl-AsMut%3C%5BMaybeUninit%3CT%3E%5D%3E-for-MaybeUninit%3C%5BT;+N%5D%3E )
🤔
There was a problem hiding this comment.
I found a way which still feels sound to me, hopefully you agree ...
7e7faa7 to
be79c14
Compare
Replace the todo's with proper code and add a host side test case.
Implements #12279
Includes all the commits on #10619 , so that one should be merged first.