Marshal a va_list

Question

I have the following code:

[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
public delegate void PanicFuncDelegate(string str, IntPtr args);

private void PanicFunc(string str, IntPtr args)
{
    LogFunc("PANIC", str, args);
}

public void LogFunc(string severity, string str, IntPtr args)
{
    vprintf($"[{severity}] "+ str,args);
}

[DllImport("libc.so.6")]
private static extern int vprintf(string format, IntPtr args);

This prints to the console the messages correctly formatted. I want to retrieve the values from args to use them in my own logger.

If I try to get the value of each pointer from the array in args (as suggested here: Marshal va_list in C# delegate) I get segmentation fault.

Any suggestions?

Wraith

A string in a dll in a byte[] terminated with a '\0'. So normally a max size of byte[] is known so you can use Marshal.PtrToStructure(). The find where the '\0' is located to get end of data. If there is an array of strings then each string is terminated with a'\0' and the final string has two "\0\0". – Morality

You can't expect to get the marshaller to handle this. The only viable way to do this is to deal with the va_list in unmanaged code, and then pass the information on in a more tractable data structure. @Morality is talking nonsense, as usual on the topic of pinvoke. – Aciculate

David : It will if you know the max length like I said. Argument size is often a predetermined fixed length. – Morality

@Morality Do you know what a va_list is? Doesn't look like you do. – Aciculate

@DavidHeffernan thank you - unfortunately i can't do it in unmanaged code, i think i will drop the idea of a custom logger .. – Wraith

Check this out #10361869 – Isaiasisak

Answer 1

I have a function call with this working, here's what I do:

For the DLLImport I use an __arglist to marshall to the va_list,

[DllImport("libc.so.6")]
private static extern int vprintf(string format, __arglist);

Then when calling the function I create the __arglist,

vprintf(string format, __arglist(arg1, arg2, arg3...))

Ofcourse you would need to either call the function with all the arguments statically or build that __arglist dynamically, I don't have the code here but it's possible.

I wonder if you get a segmentation fault because the elements in the object[] are not pinned? Maybe if you pin the object[] and all elements within that would help? Just a guess though.

Chest

Answer 2

Just think on how C program gets variables from va_list, and there is the solution:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;

namespace VaTest
{
    class Program
    {
        static void Main(string[] args)
        {
            MarshalVaArgs(vaList => vprintf("%c%d%s", vaList), false, 'a', 123, "bc");
        }

        [DllImport("msvcrt")]  //windows
      //[DllImport("c")]       //linux
        private static extern int vprintf(string format, IntPtr vaList);

        private static int IntSizeOf(Type t)
        {
            return (Marshal.SizeOf(t) + IntPtr.Size - 1) & ~(IntPtr.Size - 1);
        }

        public static void MarshalVaArgs(Action<IntPtr> action, bool? isUnicode, params object[] args)
        {
            var sizes = new int[args.Length];
            for (var i = 0; i < args.Length; i++)
            {
                sizes[i] = args[i] is string ? IntPtr.Size : IntSizeOf(args[i].GetType());
            }

            var allocs = new List<IntPtr>();
            var offset = 0;

            var result = Marshal.AllocHGlobal(sizes.Sum());
            allocs.Add(result);

            for (var i = 0; i < args.Length; i++)
            {
                if (args[i] is string)
                {
                    var s = (string)args[i];
                    var data = default(IntPtr);
                    if (isUnicode.HasValue)
                    {
                        if (isUnicode.Value)
                        {
                            data = Marshal.StringToHGlobalUni(s);
                        }
                        else
                        {
                            data = Marshal.StringToHGlobalAnsi(s);
                        }
                    }
                    else
                    {
                        data = Marshal.StringToHGlobalAuto(s);
                    }
                    allocs.Add(data);
                    Marshal.WriteIntPtr(result, offset, data);
                    offset += sizes[i];
                }
                else
                {
                    Marshal.StructureToPtr(args[i], result + offset, false);
                    offset += sizes[i];
                }
            }

            action(result);

            foreach (var ptr in allocs)
            {
                Marshal.FreeHGlobal(ptr);
            }
        }
    }
}

The code is written and tested with .NET Core 3.0 preview 5, compatible with .NET Framework 4.0 and C# 3.0.

Outputs:

a123bc

Melodimelodia

Answer 3

As this isn't solved yet i post a long solution that worked for me.

I found the solution in an abandoned project
https://github.com/GoaLitiuM/libobs-sharp

Use like this (tested with FFmpeg):

var objects = va_list_Helper.VaListToArray(format, va_List_Ptr);

// format: frame=%4d QP=%.2f NAL=%d Slice:%c Poc:%-3d I:%-4d P:%-4d SKIP:%-4d size=%d bytes%s

// format (filled): frame=   3 QP=13.00 NAL=0 Slice:B Poc:4   I:0    P:8    SKIP:912  size=32 bytes
// va_List objects: 3, 13, 0, 'B', 4, 0, 8, 912, 32

The classes needed:

public class va_list_Helper
{
    public static unsafe object[] VaListToArray(string format, byte* va_list)
    {
        var vaList = new va_list((IntPtr)va_list);
        return vaList.GetObjectsByFormat(format);
    }
}
    
public static class Printf
{
    // used
    public static string[] GetFormatSpecifiers(string format)
    {
        if (format.IndexOf('%') == -1)
            return null;

        // find specifiers from format string 
        List<int> indices = new List<int>();
        for (int j = 0; j < format.Length; j++)
        {
            j = format.IndexOf('%', j);

            if (j == -1)
                break;

            indices.Add(j);

            if (format[j + 1] == '%') // ignore "%%"
                j++;
        }

        if (indices.Count == 0)
            return null;

        List<string> formats = new List<string>(indices.Count);
        for (int mi = 0; mi < indices.Count; mi++)
        {
            string formatSpecifier = format.Substring(indices[mi], (mi + 1 < indices.Count ? indices[mi + 1] : format.Length) - indices[mi]);
            if (!string.IsNullOrWhiteSpace(formatSpecifier))
                formats.Add(formatSpecifier);
        }

        return formats.ToArray();
    }

    public class FormatSpecificationInfo
    {
        public string specification;
        //public int parameter;
        public char type;
        public int width;
        public int precision;
        public FormatFlags flags;
    };

    [Flags]
    public enum FormatFlags
    {
        // Type length 
        IsLong = 0x0001,        // l
        IsLongLong = 0x0002,    // ll
        IsShort = 0x0004,       // h
        IsChar = 0x0008,        // hh
        IsLongDouble = 0x0016,  // L

        // Flags 
        LeftAlign = 0x0100,     // '-' left align within the width
        Sign = 0x0200,          // '+' use - or + signs for signed types
        Alternate = 0x0400,     // '#' prefix non-zero values with hex types
        ZeroPad = 0x0800,       // '0' pad with zeros
        Blank = 0x1000,         // ' ' pad sign with blank
        Grouping = 0x2000,      // '\' group by thousands
        ArchSize = 0x4000,      // '?' use arch precision

        // Dynamic parameters 
        DynamicWidth = 0x10000,
        DynamicPrecision = 0x20000,
    };

    // used
    public static FormatSpecificationInfo GetFormatSpecifierInfo(string specification)
    {
        if (string.IsNullOrWhiteSpace(specification))
            return null;

        FormatSpecificationInfo info = new FormatSpecificationInfo()
        {
            type = '\0',
            width = int.MinValue,
            precision = 6,
        };

        string width = "";
        string precision = "";
        int start = -1;
        int fsLength = 1;

        // TODO: parse parameter index 

        for (int i = 0; i < specification.Length && info.type == '\0'; i++)
        {
            char c = specification[i];

            switch (c)
            {
                case '%':
                    if (start == -1)
                        start = i;
                    else
                        info.type = c;
                    info.specification = specification.Substring(start, i + 1 - start);
                    fsLength = i + 1;
                    break;

                // flags
                case '-':
                    info.flags |= FormatFlags.LeftAlign;
                    break;
                case '+':
                    info.flags |= FormatFlags.Sign;
                    break;
                case ' ':
                    info.flags |= FormatFlags.Blank;
                    break;
                case '#':
                    info.flags |= FormatFlags.Alternate;
                    break;
                case '\'':
                    info.flags |= FormatFlags.Grouping;
                    break;
                case '?':
                    info.flags |= FormatFlags.ArchSize;
                    break;

                // precision
                case '.':
                    {
                        for (int j = i + 1; j < specification.Length; j++)
                        {
                            if (specification[j] == '*')
                                info.flags |= FormatFlags.DynamicPrecision;
                            else if (char.IsNumber(specification[j]))
                                precision += specification[j];
                            else
                                break;

                            i++;
                        }
                    }
                    break;

                // length flags
                case 'h':
                    info.flags += (int)FormatFlags.IsShort;
                    break;
                case 'l':
                    info.flags += (int)FormatFlags.IsLong;
                    break;
                case 'L':
                    info.flags |= FormatFlags.IsLongDouble;
                    break;
                case 'z':
                case 'j':
                case 't':
                    // not supported
                    break;

                // dynamic width
                case '*':
                    info.flags |= FormatFlags.DynamicWidth;
                    break;

                default:
                    {
                        if (char.IsNumber(c))
                        {
                            if (width == "" && c == '0')
                                info.flags |= FormatFlags.ZeroPad;
                            else
                                width += c;
                        }
                        else if (char.IsLetter(c) && info.type == '\0')
                        {
                            info.type = c;
                            info.specification = specification.Substring(start, i + 1 - start);
                            fsLength = i + 1;
                        }
                    }
                    break;
            }
        }

        // sign overrides space
        if (info.flags.HasFlag(FormatFlags.Sign) && info.flags.HasFlag(FormatFlags.Blank))
            info.flags &= ~FormatFlags.Blank;

        if (info.flags.HasFlag(FormatFlags.LeftAlign) && info.flags.HasFlag(FormatFlags.ZeroPad))
            info.flags &= ~FormatFlags.ZeroPad;

        // unsupported precision for these types
        if (info.type == 's' ||
            info.type == 'c' ||
            Char.ToUpper(info.type) == 'X' ||
            info.type == 'o')
        {
            info.precision = int.MinValue;
        }

        if (!string.IsNullOrWhiteSpace(precision))
            info.precision = Convert.ToInt32(precision);
        if (!string.IsNullOrWhiteSpace(width))
            info.width = Convert.ToInt32(width);

        return info;
    }

    
}

public class va_list 
{
    internal IntPtr instance;    //unmanaged pointer to va_list

    public va_list(IntPtr ptr)
    {
        instance = ptr;
    }

    /// <summary> Returns unmanaged pointer to argument list. </summary>
    public IntPtr GetPointer()
    {
        return instance;
    }

    /// <summary> Returns array of objects with help of printf format string. </summary>
    /// <param name="format"> printf format string. </param>
    public object[] GetObjectsByFormat(string format)
    {
        return GetObjectsByFormat(format, this);
    }

    public static unsafe object[] GetObjectsByFormat(string format, va_list va_list)
    {
        string[] formatSpecifiers = Printf.GetFormatSpecifiers(format);
        if (formatSpecifiers == null || va_list == null || va_list.GetPointer() == IntPtr.Zero)
            return null;

        IntPtr args = va_list.GetPointer();
        List<object> objects = new List<object>(formatSpecifiers.Length);

        //var bytesDebug = new byte[format.Length];
        //Marshal.Copy(va_list.GetPointer(), bytesDebug, 0, bytesDebug.Length);

        int offset = 0;
        foreach (string spec in formatSpecifiers)
        {
            var info = Printf.GetFormatSpecifierInfo(spec);
            if (info.type == '\0')
                continue;

            // dynamic width and precision arguments
            // these are stored in stack before the actual value
            if (info.flags.HasFlag(Printf.FormatFlags.DynamicWidth))
            {
                int widthArg = Marshal.ReadInt32(args, offset);
                objects.Add(widthArg);
                offset += Marshal.SizeOf(typeof(IntPtr));
            }
            if (info.flags.HasFlag(Printf.FormatFlags.DynamicPrecision))
            {
                int precArg = Marshal.ReadInt32(args, offset);
                objects.Add(precArg);
                offset += Marshal.SizeOf(typeof(IntPtr));
            }

            int iSize = info.flags.HasFlag(Printf.FormatFlags.IsLongLong)
                ? Marshal.SizeOf(typeof(Int64)) : Marshal.SizeOf(typeof(IntPtr));

            // marshal objects from pointer
            switch (info.type)
            {
                // 8/16-bit integers
                // char / wchar_t (promoted to int)
                case 'c':
                    char c = (char)Marshal.ReadByte(args, offset);
                    objects.Add(c);
                    //offset += Marshal.SizeOf(typeof(Int32));

                    offset += Marshal.SizeOf(typeof(IntPtr));
                    break;

                // signed integers
                case 'd':
                case 'i':
                    {
                        if (info.flags.HasFlag(Printf.FormatFlags.IsShort)) // h
                        {
                            short sh = (short)Marshal.ReadInt32(args, offset);
                            objects.Add(sh);
                            offset += Marshal.SizeOf(typeof(Int32));
                        }
                        else if (info.flags.HasFlag(Printf.FormatFlags.IsLongLong)) // ll
                        {
                            long l = Marshal.ReadInt64(args, offset);
                            objects.Add(l);
                            offset += iSize;
                        }
                        else // int and long types
                        {
                            var i = Marshal.ReadInt32(args, offset);
                            objects.Add(i);
                            offset += iSize;
                        }
                    }
                    break;

                // unsigned integers
                case 'u':
                case 'o':
                case 'x':
                case 'X':
                    {
                        if (info.flags.HasFlag(Printf.FormatFlags.IsShort)) // h
                        {
                            ushort su = (ushort)Marshal.ReadInt32(args, offset);
                            objects.Add(su);
                            offset += Marshal.SizeOf(typeof(Int32));
                        }
                        else if (info.flags.HasFlag(Printf.FormatFlags.IsLongLong)) // ll
                        {
                            ulong lu = (ulong)(long)Marshal.ReadInt64(args, offset);
                            objects.Add(lu);
                            offset += iSize;
                        }
                        else // uint and ulong types
                        {
                            uint u = (uint)Marshal.ReadInt32(args, offset);
                            objects.Add(u);
                            offset += iSize;
                        }
                    }
                    break;

                // floating-point types
                case 'f':
                case 'F':
                case 'e':
                case 'E':
                case 'g':
                case 'G':
                    {
                        if (info.flags.HasFlag(Printf.FormatFlags.IsLongDouble))  // L
                        {
                            // not really supported but read it as long
                            long lfi = Marshal.ReadInt64(args, offset);
                            double d = *(double*)(void*)&lfi;
                            objects.Add(d);
                            offset += Marshal.SizeOf(typeof(double));
                        }
                        else // double
                        {
                            long lfi = Marshal.ReadInt64(args, offset);
                            double d = *(double*)(void*)&lfi;
                            objects.Add(d);
                            offset += Marshal.SizeOf(typeof(double));
                        }
                    }
                    break;

                // string
                case 's':
                    {
                        string s = null;

                        // same:
                        //var addr1 = new IntPtr(args.ToInt64() + offset);
                        //var intPtr4 = Marshal.ReadIntPtr(addr1);

                        var intPtr3 = Marshal.ReadIntPtr(args, offset);

                        if (info.flags.HasFlag(Printf.FormatFlags.IsLong))
                        {
                            s = Marshal.PtrToStringUni(intPtr3);
                        }
                        else
                        {
                            s = Marshal.PtrToStringAnsi(intPtr3);
                        }

                        objects.Add(s);
                        offset += Marshal.SizeOf(typeof(IntPtr));
                    }
                    break;

                // pointer
                case 'p':
                    IntPtr ptr = Marshal.ReadIntPtr(args, offset);
                    objects.Add(ptr);
                    offset += Marshal.SizeOf(typeof(IntPtr));
                    break;

                // non-marshallable types, ignored
                case ' ':
                case '%':
                case 'n':
                    break;

                default:
                    throw new ApplicationException("printf specifier '%" + info.type + "' not supported");
            }
        }

        return objects.ToArray();
    }
}

Niki