Force a std::Vec's capacity to shrink to equal length exactly

Question

According the documentation for std::Vec, calling shrink_to_fit() will cause the Vec's capacity to "drop down as close as possible to the length but the allocator may still inform the vector that there is space for a few more elements." Vec::with_capacity() and Vec::reserve_exact() each have a similar note saying that the reserved capacity may still be slightly greater than the length.

Meanwhile, the documentation for std::alloc::GlobalAlloc::dealloc() states that the layout used to deallocate a block of memory "must be the same layout that was used to allocate that block of memory," which means the layout that is passed to dealloc() needs to have the exact size of the block.

I am working on an FFI function that returns a list and a size. The C code that calls the function will have to call a free_X() function I provide to deallocate the list. To do that, it passes in a pointer to the list and the size of the list. In Rust, I am using a std::Vec for the list and want to shrink it so capacity == length and then std::mem::forget() it and return a pointer to it. The C code will pass in a pointer to a size_t that I will set to the size. Here are examples of what the function signatures will look like in C:

List *obtain_list(size_t *size);
void free_list(List *list, size_t size);

You can probably see the dilemma. I can shrink the std::Vec with shrink_to_fit(), but my_vec.len() might not equal my_vec.capacity(). Thus, if C passes the size it got from Rust to free_list(), free_list() will create a std::alloc::Layout that doesn't match the allocated block's size (because the block size was my_vec.capacity(), not my_vec.len()). This could result in undefined behavior, per std::alloc::dealloc()'s documentation.

I could return the capacity of the list by changing the function signatures to pass the capacity to C, like so:

List *obtain_list(size_t *size, size_t *capacity);
void free_list(List *list, size_t size, size_t capacity);

I don't like having multiple pointers that are supposed to be initialized by the called function, so I'd probably create a struct instead that holds the list pointer as well as the size and capacity.

That seems hairy to me. I would much rather just return a size. Is there a way to force std::Vec to reallocate its buffer to be exactly the same as the length?

Kinesthesia

This answer asserts that shrink_to_fit makes len and capacity equal. In any case, the problem you want to solve (how to give ownership of a vector to C code) seems to be the same as that question, so either the answers there are wrong or they should be helpful. – Leptophyllous

Perhaps use Vec::into_boxed_slice which "will drop any excess capacity", then obtain your raw pointer with Box::into_raw and later reconstruct with std::ptr::slice_from_raw_parts_mut followed by Box::from_raw? – Afraid

(You can always get back to a Vec again, if required, with <[T]>::into_vec). – Afraid

I think this is a hole in the documentation, because Vec itself acts is if the new capacity is the same as the old, and I don't even see how the allocator could inform it of excess allocated capacity. I encourage you to create a thread on IRLO to change that. Anyway, you can assert_eq!(vec.capacity(), vec.len()) but I don't see a way you can enforce it. – Compeer

@Leptophyllous I looked at that answer. It seems the assertion is there because the writer knew that they might not be equal (because the documentation says they might not be). My problem seems similar, but I think I am more focused on making sure I am following the "contracts" laid out in the documentation to the T. – Kinesthesia

Answer 1

Convert your Vec<T> into a Box<[T]> with Vec::into_boxed_slice(). A boxed slice has no capacity, only a length, which is exactly the information you want to pass to C and back, so you won't need anything extra to deallocate it.

Burkley

Thank you! That is exactly what I needed. From Vec::into_boxed_slice()'s documentation: "This will drop any excess capacity." – Kinesthesia

Answer 2

Kevin Reid's answer directly answers my question and is correct. Vec::into_boxed_slice() is the way to go.

However, before seeing Kevin's answer, I realized that I could just allocate my buffer manually with std::alloc::alloc() and then index into the buffer to create the list, just as one might do while writing C code. That is another alternative and is guaranteed to not have extra capacity.

However, the added convenience of using a Vec makes the Vec::into_boxed_slice() answer very good. I just wanted to add an alternative for anyone else looking at this post.

Kinesthesia