How interoperability works

Question

I know that many large-scale applications such as video games are created using multiple langages. For example, it's likely the game/physics engines are written in C++ while gameplay tasks, GUI are written in something like Python or Lua.

I understand why this division of roles is done; use lower-level languages for tasks that require extreme optimization, tweaking, efficiency and speed, while using higher-level languages to speed up production time, reduce nasty bugs ect.

Recently, I've decided to undertake a larger personal project and would like to divy-up parts of the project similar to above. At this point in time, I'm really confused about how this interoperability between languages (especially compiled vs interpreted) works.

I'm quite familiar with the details of going from ANSCII code test to loading an executable, when written in something like C/C++. I'm very curious at how something like a video game, built from many different languages, works. This is a large/broad question, but specifically I'm interested in:

• How does the code-level logic work? I.e. how can I call Python code from a C++ program? Especially since they don't support the same built-in types?
• What does the program image look like? From what I can tell, a video game is running in a single process, so what does the runtime image look like when running a C/C++ program that calls a Python function?
• If calling code from an interpreted language from a compiled program, what are the sequence of events that occur? I.e If I'm inside my compiled executable, and for some reason have a call to an interpreted language inside a loop, do I have to wait for the interpreter every iteration?

I'm actually finding a hard time finding information on what happening at the machine-level, so any help would be appreciated. Although I'm curious in general about interoperation of software, I'm specifically interested in C++ and Python interaction.

Thank you very much for any insight, even if it's just pointing me to where I can find more information.

Burdette

en.wikipedia.org/wiki/Foreign_function_interface you probably didn't know right term to search for – Fournier

@aryjczyk Thank you for the link. Yes, I did not know about that term. This is great in helping understanding how this is done. I'm still confused on how something like this physically works, especially in GC vs non-GC environments. I have no idea what the process image would look like. Anyways, thanks, and I'll keep looking! – Burdette

Answer 1

In the specific case of python, you have basically three options (and this generally applies across the board):

1. Host python in C++: From the perspective of the C++ programme, the python interpreter is a C library. On the python side, you may or may not need to use something like ctypes to expose the C(++) api.

2. Python uses C++ code as DLLs/SOs - C++ code likely knows nothing of python, python definitely has to use a foreign function interface.

3. Interprocess communication - basically, two separate processes run, and they talk over a socket. These days you'd likely use some kind of web services architecture to accomplish this.

Electromagnetism

Ah yes I knew about (3). Concerning (1), I kind of understand how that would be a natural idea; though I don't quite understand how it would be implemented as the Py interpreter is clearly not a C shared/dynamic library. Additionally, how does the GC-environment function without the non-GC environment? Furthermore, for example, if I made some application in C/C++ that used embedded Python, would all users necessarily need the Py interpretor installed on their disks? – Burdette

@Zak I think you just need to study the basics of how processes work, and how binary programmes are built under the hood. Python would just manage the objects it allocated itself. Python can be built for embedding. Consider what embedding means to answer your final question. – Electromagnetism

Ok thank you Marcin. I appreciate the response - I will look into it! – Burdette

Answer 2

Depending on what you want to do:

• Have a look at SWIG: http://www.swig.org/ It's a tool that aims to connect C/C++ code with Python, Tcl, Perl, Ruby, etc. The common use case is a Python interface (graphical or not) that will call the C/C++ code. SWIG will parse the C/C++ code in order to generate the interfaces.

• Libpython: it's a lib that allows you to embed Python code. You have some examples here: http://docs.python.org/3.0/extending/embedding.html

Jerol

My question is more about how something like SWIG works. But thank you for your repsonse – Burdette

@Zak Indeed, it's just that you can find on their website more information about how SWIG works. Anyway, I've edited my answer to add more information. – Spectrochemistry