How dynamic linking works, its usage and how and why you would make a dylib
Asked Answered
D

2

11

I have read several posts on stack overflow and read about dynamic linking online. And this is what I have taken away from all those readings -

Dynamic linking is an optimization technique that was employed to take full advantage of the virtual memory of the system. One process can share its pages with other processes. For example the libc++ needs to be linked with all C++ programs but instead of copying over the executable to every process, it can be linked dynamically with many processes via shared virtual pages.

However this leads me to the following questions

  1. When a C++ program is compiled. It needs to have references to the C++ library functions and code (say for example the code of the thread library). How does the compiler make the executable have these references? Does this not result in a circular dependency between the compiler and the operating system? Since the compiler has to make a reference to the dynamic library in the executable.
  2. How and when would you use a dynamic library? How do you make one? What is the specific compiling command that is used to produce such a file from a standard *.cpp file?
  3. Usually when I install a library, there is a lib/ directory with *.a files and *.dylib (on mac-OSX) files. How do I know which ones to link to statically as I would with a regular *.o file and which ones are supposed to be dynamically linked with? I am assuming the *.dylib files are dynamic libraries. Which compiler flag would one use to link to these?
  4. What are the -L and -l flags for? What does it mean to specify for example a -lusb flag on the command line?

If you feel like this question is asking too many things at once, please let me know. I would be completely ok with splitting this question up into multiple ones. I just ask them together because I feel like the answer to one question leads to another.

Disembody answered 11/4, 2016 at 22:36 Comment(1)
You can find a good potion of what you need here... #497164Kiva
K
8

When a C++ program is compiled. It needs to have references to the C++ library functions and code (say for example the code for the library).

Assume we have a hypothetical shared library called libdyno.so. You'll eventually be able to peek inside it using using objdump or nm.

objdump --syms libdyno.so

You can do this today on your system with any shared library. objdump on a MAC is called gobjdump and comes with brew in the binutils package. Try this on a mac...

gobjdump --syms /usr/lib/libz.dylib

You can now see that the symbols are contained in the shared object. When you link with the shared object you typically use something like

g++ -Wall -g -pedantic -ldyno DynoLib_main.cpp -o dyno_main

Note the -ldyno in that command. This is telling the compiler (really the linker ld) to look for a shared object file called libdyno.so wherever it normally looks for them. Once it finds that object it can then find the symbols it needs. There's no circular dependency because you the developer asked for the dynamic library to be loaded by specifying the -l flag.

How and when would you use a dynamic library? How do you make one? As in what is the specific compiling command that is used to produce such a file from a standard .cpp file

Create a file called DynoLib.cpp

#include "DynoLib.h"
DynamicLib::DynamicLib() {}
int DynamicLib::square(int a) {
  return a * a;
}

Create a file called DynoLib.h

#ifndef DYNOLIB_H
#define DYNOLIB_H
class DynamicLib {
  public:
  DynamicLib();
  int square(int a); 
};
#endif

Compile them to be a shared library as follows. This is linux specific...

g++ -Wall -g -pedantic -shared -std=c++11 DynoLib.cpp -o libdyno.so

You can now inspect this object using the command I gave earlier ie

objdump --syms libdyno.so

Now create a file called DynoLib_main.cpp that will be linked with libdyno.so and use the function we just defined in it.

#include "DynoLib.h"    
#include <iostream>     
using namespace std;
int main(void) {
  DynamicLib *lib = new DynamicLib();
  std::cout << "Square " << lib->square(1729) << std::endl;
  return 1;
}

Compile it as follows

g++ -Wall -g -pedantic -L. -ldyno DynoLib_main.cpp -o dyno_main
./dyno_main
Square 2989441

You can also have a look at the main binary using nm. In the following I'm seeing if there is anything with the string square in it ie is the symbol I need from libdyno.so in any way referenced in my binary.

nm dyno_runner |grep square
U _ZN10DynamicLib6squareEi

The answer is yes. The uppercase U means undefined but this is the symbol name for our square method in the DynamicLib Class that we created earlier. The odd looking name is due to name mangling which is it's own topic.

How do I know which ones to link to statically as I would with a regular .o file and which ones are supposed to be dynamically linked with?

You don't need to know. You specify what you want to link with and let the compiler (and linker etc) do the work. Note the -l flag names the library and the -L tells it where to look. There's a decent write up on how the compiler finds thing here

gcc Linkage option -L: Alternative ways how to specify the path to the dynamic library

Or have a look at man ld.

What are the -L and -l flags for? What does it mean to specify for example a -lusb flag on the command line?

See the above link. This is from man ld..

-L searchdir

Add path searchdir to the list of paths that ld will search for archive libraries and ld control scripts. You may use this option any number of times. The directories are searched in the order in which they are specified on the command line. Directories specified on the command line are searched before the default directories. All -L options apply to all -l options, regardless of the order in which the options appear. -L options do not affect how ld searches for a linker script unless -T option is specified.`

If you managed to get here it pays dividends to learn about the linker ie ld. It plays an important job and is the source of a ton of confusion because most people start out dealing with a compiler and think that compiler == linker and this is not true.

Kiva answered 12/4, 2016 at 23:2 Comment(0)
P
3

The main difference is that you include static linked libraries with your app. They are linked when you build your app. Dynamic libraries are linked at run time, so you do not need to include them with your app. These days dynamic libraries are used to reduce the size of apps by having many dynamic libraries on everyone's computer.

Dynamic libraries also allow users to update libraries without re-building the client apps. If a bug is found in a library that you use in your app and it is statically linked, you will have to rebuild your app and re-issue it to all your users. If a bug is found in a dynamically linked library, all your users just need to update their libraries and your app does not need an update.

Paintbrush answered 11/4, 2016 at 22:44 Comment(8)
Thank you! That is the conceptual explanation I read online! I was looking into more details about how these shared libraries work, how the compiler generates code that makes calls to the OS on start and how to go about achieving this. Thanks anyway though!Disembody
The compiler generates object files which are used by the loader on the system and edited as required (setting up the addresses). How you build them depends on your environment. In XCode, for example, when you create a project you can specify that you are building a library when you create the project.Paintbrush
But then that would make the compiler dependent on the Operating System leading to a circular dependance?Disembody
Yes, every library is dependent on the OS it's built for. A library built for a Mac won't run in Windows. A version is built separately for each host OS.Paintbrush
But the compiler is supposed to be independent of the system right? Then how would you build the system?Disembody
Languages are system independent, but compilers are not. A C compiler, for example, must generate an object file in the format specified by the host system. The host system will have a loader which can only load programs which are written in a format it understand.Paintbrush
Then how would you make the system itself?Disembody
One way is to write a cross compiler. This will run on System A but output the required formats for System B. If you look around you can see examples of these.Paintbrush

© 2022 - 2024 — McMap. All rights reserved.