With VS2005, I want to create a DLL and automatically export all symbols without adding __declspec(dllexport) everywhere, and without hand-creating .def files. Is there a way to do this?
Export all symbols when creating a DLL
Asked Answered
Short answer
You can do it with help of the new version of the CMake (any version cmake-3.3.20150721-g9cd2f-win32-x86.exe or higher).
Currently it's in the dev branch. Later, the feature will be added in the release version of the cmake-3.4.
Link to the cmake dev:
Link to an article which describe the technic:
Create dlls on Windows without declspec() using new CMake export all feature
Link to an example project:
cmake_windows_export_all_symbols
Long answer
Caution: All information below is related to the MSVC compiler or Visual Studio.
If you use other compilers like gcc on Linux or MinGW gcc compiler on Windows you don't have linking errors due to not exported symbols, because gcc compiler export all symbols in a dynamic library (dll) by default instead of MSVC or Intel windows compilers.
In windows you have to explicitly export symbol from a dll.
More info about this is provided by links:
HowTo: Export C++ classes from a DLL
So if you want to export all symbols from dll with MSVC (Visual Studio compiler) you have two options:
- Use the keyword __declspec(dllexport) in the class/function's definition.
- Create a module definition (.def) file and use the .def file when building the DLL.
1. Use the keyword __declspec(dllexport) in the class/function's definition
1.1. Add "__declspec(dllexport) / __declspec(dllimport)" macros to a class or method you want to use. So if you want to export all classes you should add this macros to all of them
More info about this is provided by link:
Exporting from a DLL Using __declspec(dllexport)
Example of usage (replace "Project" by real project name):
// ProjectExport.h
#ifndef __PROJECT_EXPORT_H
#define __PROJECT_EXPORT_H
#ifdef USEPROJECTLIBRARY
#ifdef PROJECTLIBRARY_EXPORTS
#define PROJECTAPI __declspec(dllexport)
#else
#define PROJECTAPI __declspec(dllimport)
#endif
#else
#define PROJECTAPI
#endif
#endif
Then add "PROJECTAPI" to all classes. Define "USEPROJECTLIBRARY" only if you want export/import symbols from dll. Define "PROJECTLIBRARY_EXPORTS" for the dll.
Example of class export:
#include "ProjectExport.h"
namespace hello {
class PROJECTAPI Hello {}
}
Example of function export:
#include "ProjectExport.h"
PROJECTAPI void HelloWorld();
Caution: don't forget to include "ProjectExport.h" file.
1.2. Export as C functions. If you use C++ compiler for compilation code is written on C, you could add extern "C" in front of a function to eliminate name mangling
More info about C++ name mangling is provided by link:
Example of usage:
extern "C" __declspec(dllexport) void HelloWorld();
More info about this is provided by link:
Exporting C++ Functions for Use in C-Language Executables
2. Create a module definition (.def) file and use the .def file when building the DLL
More info about this is provided by link:
Exporting from a DLL Using DEF Files
Further I describe three approach about how to create .def file.
2.1. Export C functions
In this case you could simple add function declarations in the .def file by hand.
Example of usage:
extern "C" void HelloWorld();
Example of .def file (__cdecl naming convention):
EXPORTS
_HelloWorld
2.2. Export symbols from static library
I tried approach suggested by "user72260".
He said:
- Firstly, you could create static library.
- Then use "dumpbin /LINKERMEMBER" to export all symbols from static library.
- Parse the output.
- Put all results in a .def file.
- Create dll with the .def file.
I used this approach, but it's not very convinient to always create two builds (one as a static and the other as a dynamic library). However, I have to admit, this approach really works.
2.3. Export symbols from .obj files or with help of the CMake
2.3.1. With CMake usage
Important notice: You don't need any export macros to a classes or functions!
Important notice: You can't use /GL (Whole Program Optimization) when use this approach!
- Create CMake project based on the "CMakeLists.txt" file.
- Add the following line to the "CMakeLists.txt" file: set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
- Then create Visual Studio project with help of "CMake (cmake-gui)".
- Compile the project.
Example of usage:
Root folder
CMakeLists.txt (Root folder)
cmake_minimum_required(VERSION 2.6)
project(cmake_export_all)
set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
set(dir ${CMAKE_CURRENT_SOURCE_DIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "${dir}/bin")
set(SOURCE_EXE main.cpp)
include_directories(foo)
add_executable(main ${SOURCE_EXE})
add_subdirectory(foo)
target_link_libraries(main foo)
main.cpp (Root folder)
#include "foo.h"
int main() {
HelloWorld();
return 0;
}
Foo folder (Root folder / Foo folder)
CMakeLists.txt (Foo folder)
project(foo)
set(SOURCE_LIB foo.cpp)
add_library(foo SHARED ${SOURCE_LIB})
foo.h (Foo folder)
void HelloWorld();
foo.cpp (Foo folder)
#include <iostream>
void HelloWorld() {
std::cout << "Hello World!" << std::endl;
}
Link to the example project again:
cmake_windows_export_all_symbols
CMake uses the different from "2.2. Export symbols from static library" approach.
It does the following:
1) Create "objects.txt" file in the build directory with information of .obj files are used in a dll.
2) Compile the dll, that is create .obj files.
3) Based on "objects.txt" file information extract all symbols from .obj file.
Example of usage:
DUMPBIN /SYMBOLS example.obj > log.txt
More info about this is provided by link:
4) Parse extracted from .obj file information.
In my opinion I would use calling convection, for example "__cdecl/__fastcall", "SECTx/UNDEF" symbol field (the third column), "External/Static" symbol field (the fifth column), "??", "?" information for parsing an .obj files.
I don't know how exactly CMake parse an .obj file. However, CMake is open source, so you could find out if it's interested for you.
Link to the CMake project:
5) Put all exported symbols in a .def file.
6) Link a dll with usage of a .def created file.
Steps 4)-5), that is parse .obj files and create a .def file before linking and using the .def file CMake does with help of "Pre-Link event". While "Pre-Link event" fires you could call any program you want. So in case of "CMake usage" "Pre-Link event" call the CMake with the following information about where to put the .def file and where the "objects.txt" file and with argument "-E __create_def". You could check this information by creating CMake Visusal Studio project with "set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)" and then check the ".vcxproj" project file for dll.
If you try to compile a project without "set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)" or with "set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS OFF)" you will get linking errors, due to the fact that symbols are not exported from a dll.
More info about this is provided by link:
Understanding Custom Build Steps and Build Events
2.3.2. Without CMake usage
You simple could create a small program for parsing .obj file by youself without CMake usege. Hovewer, I have to admit that CMake is very usefull program especially for cross-platform development.
This is great info. I just want to add that Option 1 is specifically what the OP did not want to do. Option 2 answers his question. Specifically 2.3 is the new information over the accepted anwser and that of @user72260. –
Nola
On Windows, as of Visual Studio 2015 Update 2, the linker has a /WHOLEARCHIVE option. See learn.microsoft.com/en-us/cpp/build/reference/… –
Huesman
Just tried to play with Pre-Link event. Seems to work good. Thank you! Below is a result of my experiements with liblmdb. `dumpbin /SYMBOLS $(Platform)\$(Configuration)\mdb.obj | findstr /R "().*External.*mdb_.*" > $(Platform)\$(Configuration)\mdb_symbols & (echo EXPORTS & for /F "usebackq tokens=2 delims==|" %%E in (
type $(Platform)\$(Configuration)\mdb_symbols) do @echo %%E) > $(Platform)\$(Configuration)\lmdb.def –
Robinette For mingw users, the variable CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS in ambiguously named, it only applies to msvc. –
Furlough
The mentioned above MSC compiler option /WHOLEARCHIVE:<static_lib> unfortunately is not a replacement for the DEF file. Despite the misleading message on docs.microsoft.com/en-us/cpp/build/reference/… that it "can be used to re-export all the symbols from a static library" it does not force the linker to behave as if these symbols had been specified in the DEF file. "...all of your library code, resources, and metadata are included..."... Sure, "included", but just to the linking command as a part of *.obj and not to the DLL. –
Output
It can be done...
The way we do it here is to use the /DEF option of the linker to pass a "module definition file" containing a list of our exports. I see from your question that you know about these files. However, we do not do it by hand. The list of exports itself is created by the dumpbin /LINKERMEMBER command, and manipulating the output via a simple script to the format of a module definition file.
It is a lot of work to setup, but it allows us to compile code created without dllexport declarations for Unix on Windows.
It's usually better to add your export macros, which expand to
__declspec(dllexport) on Windows, __attribute__ ((dllexport)) on gcc, and empty on other compilers. Then pass -fvisibility=hidden on gcc. You'll get a smaller, cleaner symbol table and you'll catch mistakes that'd break the Windows build when testing on Linux. –
Aweless The OP did not want to write
__declspec(dllexport) everywhere. Adding some other export macro everywhere is just as difficult. –
Nola I want to create a DLL and automatically export all symbols without adding __declspec(dllexport) everywhere and without hand-creating .def files. Is threre a way to do this?
This is a late answer, but it provides the details for Maks's answer in Section (2). It also avoids scripts and uses a C++ program called dump2def. The source code for dump2def is below.
Finally, the steps below assume you are working from a Visual Studio Developer Prompt, which is a Windows Terminal where vcvarsall.bat has been run. You need to ensure the build tools like cl.exe, lib.exe, link.exe and nmake.exe are on-path.
More info about this is provided by link:
The instruction below use:
static.lib- static library archive (*.a file on Linux)dynamic.dll- dynamic library (*.so file on Linux)import.lib- dynamic library (import library on Windows)
Also note that though you are exporting everything from the DLL, clients still must use declspec(dllimport) on all symbols (classes, functions and data) that they use. Also see on MSDN.
First, take your objects and create a static archive:
AR = lib.exe
ARFLAGS = /nologo
CXX_SRCS = a.cpp b.cpp c.cpp ...
LIB_OBJS = a.obj b.obj c.obj ...
static.lib: $(LIB_OBJS)
$(AR) $(ARFLAGS) $(LIB_OBJS) /out:$@
Second, run dumpbin.exe /LINKERMEMEBER on the archive to create a *.dump file:
dynamic.dump:
dumpbin /LINKERMEMBER static.lib > dynamic.dump
Third, run dump2def.exe on the *.dump file to produce the *.def file. The source code for dump2def.exe is below.
dynamic.def: static.lib dynamic.dump
dump2def.exe dynamic.dump dynamic.def
Fourth, build the DLL:
LD = link.exe
LDFLAGS = /OPT:REF /MACHINE:X64
LDLIBS = kernel32.lib
dynamic.dll: $(LIB_OBJS) dynamic.def
$(LD) $(LDFLAGS) /DLL /DEF:dynamic.def /IGNORE:4102 $(LIB_OBJS) $(LDLIBS) /out:$@
/IGNORE:4102 is used to avoid this warning. It is expected in this case:
dynamic.def : warning LNK4102: export of deleting destructor 'public: virtual v
oid * __ptr64 __cdecl std::exception::`scalar deleting destructor'(unsigned int)
__ptr64'; image may not run correctly
When the dynamic.dll recipe is invoked, it creates a dynamic.lib import file and dynamic.exp file, too:
> cls && nmake /f test.nmake dynamic.dll
...
Creating library dynamic.lib and object dynamic.exp
And:
C:\Users\Test\testdll>dir *.lib *.dll *.def *.exp
Volume in drive C is Windows
Volume Serial Number is CC36-23BE
Directory of C:\Users\Test\testdll
01/06/2019 08:33 PM 71,501,578 static.lib
01/06/2019 08:33 PM 11,532,052 dynamic.lib
Directory of C:\Users\Test\testdll
01/06/2019 08:35 PM 5,143,552 dynamic.dll
Directory of C:\Users\Test\testdll
01/06/2019 08:33 PM 1,923,070 dynamic.def
Directory of C:\Users\Test\testdll
01/06/2019 08:35 PM 6,937,789 dynamic.exp
5 File(s) 97,038,041 bytes
0 Dir(s) 139,871,186,944 bytes free
Gluing it together here is what the Nmake makefile looks like. It is part of a real Nmake file:
all: test.exe
test.exe: pch.pch static.lib $(TEST_OBJS)
$(LD) $(LDFLAGS) $(TEST_OBJS) static.lib $(LDLIBS) /out:$@
static.lib: $(LIB_OBJS)
$(AR) $(ARFLAGS) $(LIB_OBJS) /out:$@
dynamic.map:
$(LD) $(LDFLAGS) /DLL /MAP /MAPINFO:EXPORTS $(LIB_OBJS) $(LDLIBS) /out:dynamic.dll
dynamic.dump:
dumpbin.exe /LINKERMEMBER static.lib /OUT:dynamic.dump
dynamic.def: static.lib dynamic.dump
dump2def.exe dynamic.dump
dynamic.dll: $(LIB_OBJS) dynamic.def
$(LD) $(LDFLAGS) /DLL /DEF:dynamic.def /IGNORE:4102 $(LIB_OBJS) $(LDLIBS) /out:$@
clean:
$(RM) /F /Q pch.pch $(LIB_OBJS) pch.obj static.lib $(TEST_OBJS) test.exe *.pdb
And here is the source code for dump2def.exe:
#include <iostream>
#include <fstream>
#include <sstream>
#include <string>
#include <vector>
#include <set>
typedef std::set<std::string> SymbolMap;
void PrintHelpAndExit(int code)
{
std::cout << "dump2def - create a module definitions file from a dumpbin file" << std::endl;
std::cout << " Written and placed in public domain by Jeffrey Walton" << std::endl;
std::cout << std::endl;
std::cout << "Usage: " << std::endl;
std::cout << " dump2def <infile>" << std::endl;
std::cout << " - Create a def file from <infile> and write it to a file with" << std::endl;
std::cout << " the same name as <infile> but using the .def extension" << std::endl;
std::cout << " dump2def <infile> <outfile>" << std::endl;
std::cout << " - Create a def file from <infile> and write it to <outfile>" << std::endl;
std::exit(code);
}
int main(int argc, char* argv[])
{
// ******************** Handle Options ******************** //
// Convenience item
std::vector<std::string> opts;
for (size_t i=0; i<argc; ++i)
opts.push_back(argv[i]);
// Look for help
std::string opt = opts.size() < 3 ? "" : opts[1].substr(0,2);
if (opt == "/h" || opt == "-h" || opt == "/?" || opt == "-?")
PrintHelpAndExit(0);
// Add <outfile> as needed
if (opts.size() == 2)
{
std::string outfile = opts[1];
std::string::size_type pos = outfile.length() < 5 ? std::string::npos : outfile.length() - 5;
if (pos == std::string::npos || outfile.substr(pos) != ".dump")
PrintHelpAndExit(1);
outfile.replace(pos, 5, ".def");
opts.push_back(outfile);
}
// Check or exit
if (opts.size() != 3)
PrintHelpAndExit(1);
// ******************** Read MAP file ******************** //
SymbolMap symbols;
try
{
std::ifstream infile(opts[1].c_str());
std::string::size_type pos;
std::string line;
// Find start of the symbol table
while (std::getline(infile, line))
{
pos = line.find("public symbols");
if (pos == std::string::npos) { continue; }
// Eat the whitespace after the table heading
infile >> std::ws;
break;
}
while (std::getline(infile, line))
{
// End of table
if (line.empty()) { break; }
std::istringstream iss(line);
std::string address, symbol;
iss >> address >> symbol;
symbols.insert(symbol);
}
}
catch (const std::exception& ex)
{
std::cerr << "Unexpected exception:" << std::endl;
std::cerr << ex.what() << std::endl;
std::cerr << std::endl;
PrintHelpAndExit(1);
}
// ******************** Write DEF file ******************** //
try
{
std::ofstream outfile(opts[2].c_str());
// Library name, cryptopp.dll
std::string name = opts[2];
std::string::size_type pos = name.find_last_of(".");
if (pos != std::string::npos)
name.erase(pos);
outfile << "LIBRARY " << name << std::endl;
outfile << "DESCRIPTION \"Crypto++ Library\"" << std::endl;
outfile << "EXPORTS" << std::endl;
outfile << std::endl;
outfile << "\t;; " << symbols.size() << " symbols" << std::endl;
// Symbols from our object files
SymbolMap::const_iterator it = symbols.begin();
for ( ; it != symbols.end(); ++it)
outfile << "\t" << *it << std::endl;
}
catch (const std::exception& ex)
{
std::cerr << "Unexpected exception:" << std::endl;
std::cerr << ex.what() << std::endl;
std::cerr << std::endl;
PrintHelpAndExit(1);
}
return 0;
}
I've written a small program to parse the output of "dumpbin /linkermember" on the .lib file. I have upwards of 8,000 function references to export from one DLL.
The problem with doing it on a DLL is that you have to link the DLL without the exported definitions once to create the .lib file, then generate the .def which means you now have to relink the DLL again with the .def file to actually have the references exported.
Working with static libraries is easier. Compile all your sources into static libs, run dumbin, generate a .def with your little program, then link the libs together into a DLL now that the export names are available.
Unfortunately my company won't allow me to show you the source. The work involved is recognizing which "public symbols" in the dump output are not needed in your def file. You have to throw away a lot of those references, NULL_IMPORT_DESCRIPTOR, NULL_THUNK_DATA, __imp*, etc.
How do you deal with templates that have members on cpp files? –
Desalinate
@Desalinate - Use Explicit Instantiations to force the instantiation before hand. Or, ensure it is a header-only implementation (which it sounds like you don't have). –
Gambell
@Paternal - Build a static lib instead of DLL using the same objects. Run
dumpbin.exe on the static lib. You won't have NULL_IMPORT_DESCRIPTOR, NULL_THUNK_DATA, __imp*, etc. Then, create the DLL with the same objects and the new DEF file. –
Gambell Thanks @Maks for the detailed answer.
Below is an example of what I used in Pre-Link event to generate def file from obj. I hope it will be helpful for someone.
dumpbin /SYMBOLS $(Platform)\$(Configuration)\mdb.obj | findstr /R "().*External.*mdb_.*" > $(Platform)\$(Configuration)\mdb_symbols
(echo EXPORTS & for /F "usebackq tokens=2 delims==|" %%E in (`type $(Platform)\$(Configuration)\mdb_symbols`) do @echo %%E) > $(Platform)\$(Configuration)\lmdb.def
Basically I just took one of objects (mdb.obj) and grepped mdb_* functions. Then parsed output to keep just names taking into account amount of spaces for indentation (one after splitting into tokens and another in echo. I don't know if it's matter though).
Real world script probably will kind of more complex though.
Perhaps somebody finds useful my Python script for converting .dump to .def.
import sys, os
functions = []
startPoint = False
# Exclude standard API like sprintf to avoid multiple definition link error
excluded_functions = [ 'sprintf', 'snprintf', 'sscanf', 'fprintf' ]
if len(sys.argv) < 2:
print('Usage: %s <Input .dump file> <Output .def file>.' % sys.argv[0])
print('Example: %s myStaticLib.dump exports.def' % sys.argv[0])
sys.exit(1)
print('%s: Processing %s to %s' % (sys.argv[0], sys.argv[1], sys.argv[2]))
fin = open(sys.argv[1], 'r')
lines = fin.readlines()
fin.close()
# Reading
for l in lines:
l_str = l.strip()
if (startPoint == True) and (l_str == 'Summary'): # end point
break
if (startPoint == False) and ("public symbols" in l_str):
startPoint = True
continue
if (startPoint == True) and l_str is not '':
funcName = l_str.split(' ')[-1]
if funcName not in excluded_functions:
functions.append(" " + funcName)
# Writing
fout = open(sys.argv[2], 'w')
fout.write('EXPORTS\n')
for f in functions:
fout.write('%s\n' % f)
fout.close()
With this script you can get the .def file for your .lib in two steps:
dumpbin /LINKERMEMBER:1 myStaticLib.lib > myExports.dump
python dump2def.py myExports.dump myExports.def
No, you will need a macro that resolves to __declspec(dllexport) when it's included by the .cpp file that implements the exported functions, and resolves to __declspec(dllimport) otherwise.
© 2022 - 2024 — McMap. All rights reserved.