Fix Mono AOT crash on struct fields with unsupported marshal conversion

[lewing]

Problem

The Mono AOT compiler crashes with SIGABRT when compiling assemblies that contain structs with reference-type fields (string, class, array) that resolve to MONO_MARSHAL_CONV_NONE — for example a string field with [MarshalAs(UnmanagedType.CustomMarshaler)].

This hits the linux-x64 Release AllSubsets_Mono_LLVMFULLAOT_RuntimeIntrinsicsTests leg on every build (#125168, 22 hits/month). The failing assembly is ICustomMarshaler_TargetUnix.dll which contains:

public struct StructWithCustomMarshalerField
{
    [MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof(StringForwardingCustomMarshaler))]
    public string Field;
}

The test that uses this struct (CustomMarshaler_StructWithCustomMarshalerFieldIsMarshaledCorrectly) expects a TypeLoadException at runtime — the struct is intentionally invalid. But the AOT compiler crashes before the test can run.

Root cause

In mono_marshal_shared_emit_struct_conv_full (marshal-shared.c):

1. mono_type_to_unmanaged sees MONO_TYPE_STRING + MONO_NATIVE_CUSTOM → returns MONO_NATIVE_CUSTOM with conv = MONO_MARSHAL_CONV_NONE
2. The MONO_MARSHAL_CONV_NONE switch enters the inner type switch
3. MONO_TYPE_STRING (0x0e) is not handled → falls to default → g_assert_not_reached() → SIGABRT

Fix

Handle MONO_TYPE_STRING, MONO_TYPE_CLASS, MONO_TYPE_OBJECT, MONO_TYPE_SZARRAY, and MONO_TYPE_ARRAY in the MONO_MARSHAL_CONV_NONE path by emitting a MarshalDirectiveException instead of asserting. This lets the AOT compiler complete successfully and the runtime throw the expected exception when the type is actually used.

The existing MONO_TYPE_OBJECT case already did this (with a less descriptive "COM support was disabled" message). This PR consolidates all reference types into the same error path with a more accurate message.

Fixes #125168

[Copilot]

Pull request overview

Fixes a Mono AOT compiler crash when encountering struct fields whose marshaling resolves to MONO_MARSHAL_CONV_NONE for reference types (e.g., string with UnmanagedType.CustomMarshaler), by emitting a MarshalDirectiveException path instead of asserting.

Changes:

• Extends the MONO_MARSHAL_CONV_NONE handling in mono_marshal_shared_emit_struct_conv_full to cover additional reference-type field kinds (STRING, CLASS, arrays) alongside OBJECT.
• Replaces the previous special-case MONO_TYPE_OBJECT message with a consolidated, more descriptive error message for these reference types.

---

You can also share your feedback on Copilot code review. Take the survey.

[lewing]

/ba-g failures are infra related

[lewing]

/backport to release/10.0

[github-actions]

Started backporting to release/10.0 (link to workflow run)