[API Proposal]: `RuntimeHelpers.AllocateTypeAssociatedMemory` overload with alignment

[MichalPetryka]

Background and motivation

RuntimeHelpers.AllocateTypeAssociatedMemory is an API that's very useful for allocating private, permanent buffers for lookup tables and such.

It doesn't however provide a way to specify the alignment which makes the users that require it for correctness or performance have to manually ensure that via for example overallocating today.
As such, I propose that an overload that allows it to be specified would be added.

API Proposal

namespace System.Runtime.CompilerServices;

public static class RuntimeHelpers
{
    public static IntPtr AllocateTypeAssociatedMemory(Type type, int size, int alignment);
}

API Usage

public static class SimdAlgorithm
{
    public static readonly byte* SimdLookupTable = (byte*)RuntimeHelpers.AllocateTypeAssociatedMemory(typeof(SimdAlgorithm), 4096, 64);
}

Alternative Designs

The API could return void* instead and take parameters as nuint, I decided to match the existing overload for now though.

Risks

Matching IntPtr return might be problematic with the new unsafe guidelines.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-runtime-compilerservices
See info in area-owners.md if you want to be subscribed.

[tannergooding]

This seems reasonable, but needs weigh-in from @AaronRobinsonMSFT before being marked ready-for-review.

I've needed this a few times and it typically just involves doing the extra step to overallocate and "align up". Anyone can do the same via a helper, especially using one of the new extension members features (such that you can define static extension APIs). However, the runtime could use a "proper" aligned memory allocator internally and be a bit more efficient

[AaronRobinsonMSFT]

I'm not sure how useful it really is for us to expose relative to people that need the associative/unloading functionality it provides, but it would align with the existing APIs on NativeMemory. Since this is allocating memory associated with an type/assembly it is going to use the LoaderHeap on LoaderAllocator. That does have an existing API for aligned memory, but in practice we are just going to do what @tannergooding stated above:

just involves doing the extra step to overallocate and "align up"

However, the runtime could use a "proper" aligned memory allocator internally and be a bit more efficient

True, we could do some additional optimizations in the future here. Admittedly, we do try to minimize the overallocation.

This was originally exposed for the newer COM interop scenario involving ComWrappers, so I get there are cases. The native AOT and mono implementations will simply defer to NativeMemory anyways since they don't support unloading. I would like to understand if there is really a need for the unloading angle? It was needed for COM because the new scenario explicitly supports unloading, eventhough in practice probably isn't all that interesting.

Is there a real interest in the unloadability angle here?

[jkotas]

I've needed this a few times and it typically just involves doing the extra step to overallocate

If possible, could you please share links to real-world examples where this would be useful?

I assume that lookup tables used by System.Runtime.Intririsics.* are the motivating use case for this API. I think it is reasonable that users want to write code compatible with unloading in these scenarios. I do not see a problem with adding the API for symmetry with NativeMemory given that it is trivial to implement.

[tannergooding]

I assume that lookup tables used by System.Runtime.Intririsics.* are the motivating use case for this API.

Right. This, as well as static buffers used in coordination with a graphics API, VTBLs or IIDs associated with a type, and some cases of helper structs that wrap a pointer where I want a reusable struct (UnmanagedArray<T>.Empty being one example)

Most of these weren't alignment related and were just to be doing the "right thing" with regards to an allocation that should live only until the assembly unloaded, so it doesn't leak. A handful of those scenarios are here: https://github.com/search?q=org%3Aterrafx%20AllocateTypeAssociatedMemory&type=code

I have some local, not yet checked in, code that's porting some C algorithms like https://github.com/amd/aocl-libm-ose/blob/aocl-5.1/src/optimized/log_data.h as well. Where I'm preserving the alignment C specifies (16) to avoid potential cache line splits for the log_table since it's sometimes used with non-temporal accesses for large data (normally I'd just use the ROSpan<T> approach for an RVA static instead).

---

Like I said, this is a relatively trivial thing to emulate. This is especially true moving forward with extension members since anyone can define an extension method exposing such a helper off RuntimeHelpers now.

So it's really just a question on if we want to provide that minor convenience in box and potentially allow it to micro-optimize and not overallocate (which realistically is going to be no more than 48 bytes on a typical machine).

[MichalPetryka]

Is there a real interest in the unloadability angle here?

Personally I never use unloadability but I always try to make my libraries compatible with it so kinda.

The native AOT and mono implementations will simply defer to NativeMemory anyways since they don't support unloading.

I've proposed to @MichalStrehovsky before to have the NAOT cctor interpreter handle RuntimeHelpers.AllocateTypeAssociatedMemory specially and emit it into the binary sections at build time, he wasn't interested in it but if things change the API could become special like that.

[AaronRobinsonMSFT]

A handful of those scenarios are here: https://github.com/search?q=org%3Aterrafx%20AllocateTypeAssociatedMemory&type=code

Thanks. I see the pattern here.

but I always try to make my libraries compatible with it so kinda.

+1

RuntimeHelpers.AllocateTypeAssociatedMemory specially and emit it into the binary sections at build time, he wasn't interested in it but if things change the API could become special like that.

This is an interesting idea and something that an official API would help enable.