What is marshalling? What is happening when something is "marshalled?"

Question

I know this question has been asked, at least here.

But there wasn't a satisfactory answer, at least not to me. There is a lot of talk about marshalling as regards interoperating with unmanaged code, but what about marshalling from one thread to another, as we have to do in .NET sometimes.

This makes me ask, what is marshalling, really? When you give a definition of marshalling, how would you define it so that it is explaining the case of interoperability, as well as the cases where you are "marshalling" between threads?

Glossotomy

You are just getting more of what was already in that thread. Which is inevitable, it is a very generic term. Doesn't mean anything more than 'change the runtime environment'. – Sciurine

Answer 1

Computations often need to move data from one site to another, and don't have any shared memory. So one computation sends a message containing the data to the other.

How should that data, if it is arbitrarily complicated, be sent in a message?

Marshalling is the process of converting a data field, or an entire set of related structures, into a serialized string that can be sent in a message. To marshall a binary number, one might convert it to hexadecimal digit string, if the message format must be text. If the message will carry binary data, the binary number might be converted into 4 little-endian normalized binary bytes and sent that way. Pointers are harder; one often has to convert them into an abstract reference (e.g., a "node number") that is independent of the actual memory locations.

Of course, if you "marshall" data, you must eventually "unmarshall", which is the process of reading the serial stream and reconstructing the transmitted data (structure).

Often there are (un)marshalling routines in a library that are used to accomplish this purpose, and sometimes there are even tools that will manufacture all the calls needed on the (un)marshalling routines to send/recieve the data.

Benildas

Copsey is right: in general, marshalling is used to describe translating usually from the data structure representation in one computation in one language, into a generic representation, to be handed to another computation, possibly in a different representation. Cross language calls even within the same process may require marshalling to move the data between them. I donated a point to Reed's answer for his clarification. – Benildas

So, marshal means the same thing as serialize? – Recap

Answer 2

Marshalling is taking data, of some form, and translating it into a separate form. It's a very generic term, and used in many places with subtle differences in meaning.

For example, in .NET, the interop layer when you're working with native types "marshals" your data from the .NET type into the appropriate form to call the native method, then "marshals" the results back.

As for "marshalling" between threads - Often, you'll need to have code to run on a different thread than the current one. For example, if you're using Windows Forms, you can't change a UI element on a threadpool thread, so you'll need to "marshal" the call back to the UI thread. This is done by creating a delegate, and passing the delegate back to the user interface thread via Control.Invoke (which uses a rather complex system to post this back to the proper synchronization context), which in turn runs the delegate on the user interface thread for you.

Scissure

Answer 3

Wikipedia's definition is actually pretty good.

The overall concept of marshalling is the same as "serialization:" moving from an in-memory representation (which, in a way, is like no representation at all - when something is in memory it simply "exists") to a "hard copy" representation, whether that's XML or maybe a binary stream or something. However, depending on what you're doing, it can also imply some kind of transformation or translation to a target format.

For process marshalling: one thread doesn't simply "call" another - data has to be packaged up and "sent" from one thread to another. Marshalling is the process of packaging that data (for example, data about the method you want to call, and its parameters).

If you're marshalling in terms of interop, you are packaging up a method call and its parameters into a data structure that can be sent to a process/thread running the COM component. That package needs to be in a format that the COM component can understand.

Matsuyama

Answer 4

From Wikipedia - Marshalling (computer science):

Marshalling (similar to serialization) is the process of transforming the memory representation of an object to a data format suitable for storage or transmission. It is typically used when data must be moved between different parts of a computer program or from one program to another.

In the case of calling an unmanaged function from .NET, marshalling is used to convert .NET's data into data that the unmanaged function can consume. For instance, System.String is Unicode based, but that string might need to be converted to an ANSI string to be passed into a unmanaged C function.

For threading, marshalling typically refers to transfer of ownership of some data from one thread to another thread. For example, a program has two threads. The first thread reads data from the network, and the second thread computes that data. After the network thread reads some data it transfers (i.e., "marshals") the data over to computation thread to process. It might do this by writing the data to a queue shared between the two threads.

Marshalling in threading almost always involves synchronization of the data being marshalled.

Phrenic

like the concrete example of System.String of C# converted to ANSI string in C. This helps much better than abstract explanation. However, what about custom type in C# ,say a class, how do you marshal that to C ? – Jo

Specifics are out of scope for a comment, but essentially you mirror your C struct definition in C#. You use some attributes (primarly StructLayout and MarshalAs) on the C# struct to tell the .NET runtime how to marshal the struct's data members. Have a look at Marshaling Classes, Structures, and Unions for some examples and more information. – Phrenic

Answer 5

The way I understand marshaling is that it provides a way for you to transfer data in a consistent manner across various operating environments.

In the context of marshaling data from managed to unmanaged code, it's more or less the same.

I have some data, say an array of integers or any data type of my choosing, and I want to make it available for use within my C# code after my C++ code does some operations on it.

I can't just say "Hey, this is where the array is, do what you want" to the C# code. An array of ints in C++ may not be stored the same way as in C#. Marshaling let's us transmit this data in an environment independent manner so that either side sees the data the same exact way.

Another example would be in networking. You usually don't call this marshaling, but if you want to transmit it over the network, you have to typically transmit it in such a way that whoever receives it interprets the data the same way you do. Your computer could represent data in little endian order, and the other could represent it in big endian order.

tl;dr: Marshaling provides you a way to consistently represent data across various operating environments

Actinomycin

Answer 6

It's usually used in the context of "written in an XML format" but it could be marshalled to any format.

2.  To arrange, place, or set in methodical order.
   (from American Heritage® Dictionary of the English Language)

So it means you're arranging the data in the methodical order/format you want. Often this is in XML format.

Hallow