Mocking with Boost::Test

Question

I'm using the Boost::Test library for unit testing, and I've in general been hacking up my own mocking solutions that look something like this:

//In header for clients
struct RealFindFirstFile
{
    static HANDLE FindFirst(LPCWSTR lpFileName, LPWIN32_FIND_DATAW lpFindFileData) {
        return FindFirstFile(lpFileName, lpFindFileData);
    };
};

template <typename FirstFile_T = RealFindFirstFile>
class DirectoryIterator {
//.. Implementation
}

//In unit tests (cpp)
#define THE_ANSWER_TO_LIFE_THE_UNIVERSE_AND_EVERYTHING 42
struct FakeFindFirstFile
{
    static HANDLE FindFirst(LPCWSTR lpFileName, LPWIN32_FIND_DATAW lpFindFileData) {
        return THE_ANSWER_TO_LIFE_THE_UNIVERSE_AND_EVERYTHING;
    };
};
BOOST_AUTO_TEST_CASE( MyTest )
{
    DirectoryIterator<FakeFindFirstFile> LookMaImMocked;
    //Test
}

I've grown frustrated with this because it requires that I implement almost everything as a template, and it is a lot of boilerplate code to achieve what I'm looking for.

Is there a good method of mocking up code using Boost::Test over my Ad-hoc method?

I've seen several people recommend Google Mock, but it requires a lot of ugly hacks if your functions are not virtual, which I would like to avoid.

Oh: One last thing. I don't need assertions that a particular piece of code was called. I simply need to be able to inject data that would normally be returned by Windows API functions.

EDIT: Here is an example class set and the tests I have for it:

The classes under test:

#include <list>
#include <string>
#include <boost/noncopyable.hpp>
#include <boost/make_shared.hpp>
#include <boost/iterator/iterator_facade.hpp>
#include <Windows.h>
#include <Shlwapi.h>
#pragma comment(lib, "shlwapi.lib")
#include "../Exception.hpp"

namespace WindowsAPI { namespace FileSystem {

//For unit testing
struct RealFindXFileFunctions
{
    HANDLE FindFirst(LPCWSTR lpFileName, LPWIN32_FIND_DATAW lpFindFileData) {
        return FindFirstFile(lpFileName, lpFindFileData);
    };
    BOOL FindNext(HANDLE hFindFile, LPWIN32_FIND_DATAW lpFindFileData) {
        return FindNextFile(hFindFile, lpFindFileData);
    };
    BOOL Close(HANDLE hFindFile) {
        return FindClose(hFindFile);
    };
};

class Win32FindData {
    WIN32_FIND_DATA internalData;
    std::wstring rootPath;
public:
    Win32FindData(const std::wstring& root, const WIN32_FIND_DATA& data) :
        rootPath(root), internalData(data) {};
    DWORD GetAttributes() const {
        return internalData.dwFileAttributes;
    };
    bool IsDirectory() const {
        return (internalData.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) != 0;
    };
    bool IsFile() const {
        return !IsDirectory();
    };
    unsigned __int64 GetSize() const {
        ULARGE_INTEGER intValue;
        intValue.LowPart = internalData.nFileSizeLow;
        intValue.HighPart = internalData.nFileSizeHigh;
        return intValue.QuadPart;
    };
    std::wstring GetFolderPath() const {
        return rootPath;
    };
    std::wstring GetFileName() const {
        return internalData.cFileName;
    };
    std::wstring GetFullFileName() const {
        return rootPath + L"\\" + internalData.cFileName;
    };
    std::wstring GetShortFileName() const {
        return internalData.cAlternateFileName;
    };
    FILETIME GetCreationTime() const {
        return internalData.ftCreationTime;
    };
    FILETIME GetLastAccessTime() const {
        return internalData.ftLastAccessTime;
    };
    FILETIME GetLastWriteTime() const {
        return internalData.ftLastWriteTime;
    };
};

template <typename FindXFileFunctions_T>
class BasicEnumerationMethod : public boost::noncopyable {
protected:
    WIN32_FIND_DATAW currentData;
    HANDLE hFind;
    std::wstring currentDirectory;
    FindXFileFunctions_T FindFileFunctions;
    BasicEnumerationMethod(FindXFileFunctions_T functor) :
        hFind(INVALID_HANDLE_VALUE),
        FindFileFunctions(functor) {};
    void IncrementCurrentDirectory() {
        if (hFind == INVALID_HANDLE_VALUE) return;
        BOOL success =
            FindFileFunctions.FindNext(hFind, &currentData);
        if (success)
            return;
        DWORD error = GetLastError();
        if (error == ERROR_NO_MORE_FILES) {
            FindFileFunctions.Close(hFind);
            hFind = INVALID_HANDLE_VALUE;
        } else {
            WindowsApiException::Throw(error);
        }
    };
    virtual ~BasicEnumerationMethod() {
        if (hFind != INVALID_HANDLE_VALUE)
            FindFileFunctions.Close(hFind);
    };
public:
    bool equal(const BasicEnumerationMethod<FindXFileFunctions_T>& other) const {
        if (this == &other)
            return true;
        return hFind == other.hFind;
    };
    Win32FindData dereference() {
        return Win32FindData(currentDirectory, currentData);
    };
};

template <typename FindXFileFunctions_T>
class BasicNonRecursiveEnumeration : public BasicEnumerationMethod<FindXFileFunctions_T>
{
public:
    BasicNonRecursiveEnumeration(FindXFileFunctions_T functor = FindXFileFunctions_T())
        : BasicEnumerationMethod<FindXFileFunctions_T>(functor) {};
    BasicNonRecursiveEnumeration(const std::wstring& pathSpec,
            FindXFileFunctions_T functor = FindXFileFunctions_T())
            : BasicEnumerationMethod<FindXFileFunctions_T>(functor) {
        std::wstring::const_iterator lastSlash =
            std::find(pathSpec.rbegin(), pathSpec.rend(), L'\\').base();
        if (lastSlash != pathSpec.begin())
            currentDirectory.assign(pathSpec.begin(), lastSlash-1);
        hFind = FindFileFunctions.FindFirst(pathSpec.c_str(), &currentData);
        if (hFind == INVALID_HANDLE_VALUE
            && GetLastError() != ERROR_PATH_NOT_FOUND
            && GetLastError() != ERROR_FILE_NOT_FOUND)
            WindowsApiException::ThrowFromLastError();
        while (hFind != INVALID_HANDLE_VALUE && (!wcscmp(currentData.cFileName, L".") ||
                !wcscmp(currentData.cFileName, L".."))) {
            IncrementCurrentDirectory();
        }
    };
    void increment() {
        IncrementCurrentDirectory();
    };
};

typedef BasicNonRecursiveEnumeration<RealFindXFileFunctions> NonRecursiveEnumeration;

template <typename FindXFileFunctions_T>
class BasicRecursiveEnumeration : public BasicEnumerationMethod<FindXFileFunctions_T>
{
    //Implementation ommitted
};

typedef BasicRecursiveEnumeration<RealFindXFileFunctions> RecursiveEnumeration;

struct AllResults
{
    bool operator()(const Win32FindData&) {
        return true;
    };
}; 

struct FilesOnly
{
    bool operator()(const Win32FindData& arg) {
        return arg.IsFile();
    };
};

template <typename Filter_T = AllResults, typename Recurse_T = NonRecursiveEnumeration>
class DirectoryIterator : 
    public boost::iterator_facade<
        DirectoryIterator<Filter_T, Recurse_T>,
        Win32FindData,
        std::input_iterator_tag,
        Win32FindData
    >
{
    friend class boost::iterator_core_access;
    boost::shared_ptr<Recurse_T> impl;
    Filter_T filter;
    void increment() {
        do {
            impl->increment();
        } while (! filter(impl->dereference()));
    };
    bool equal(const DirectoryIterator& other) const {
        return impl->equal(*other.impl);
    };
    Win32FindData dereference() const {
        return impl->dereference();
    };
public:
    DirectoryIterator(Filter_T functor = Filter_T()) :
        impl(boost::make_shared<Recurse_T>()),
        filter(functor) {
    };
    explicit DirectoryIterator(const std::wstring& pathSpec, Filter_T functor = Filter_T()) :
        impl(boost::make_shared<Recurse_T>(pathSpec)),
        filter(functor) {
    };
};

}}

Tests for this class:

#include <queue>
#include "../WideCharacterOutput.hpp"
#include <boost/test/unit_test.hpp>
#include "../../WindowsAPI++/FileSystem/Enumerator.hpp"
using namespace WindowsAPI::FileSystem;

struct SimpleFakeFindXFileFunctions
{
    static std::deque<WIN32_FIND_DATAW> fakeData;
    static std::wstring insertedFileName;

    HANDLE FindFirst(LPCWSTR lpFileName, LPWIN32_FIND_DATAW lpFindFileData) {
        insertedFileName.assign(lpFileName);
        if (fakeData.empty()) {
            SetLastError(ERROR_PATH_NOT_FOUND);
            return INVALID_HANDLE_VALUE;
        }
        *lpFindFileData = fakeData.front();
        fakeData.pop_front();
        return reinterpret_cast<HANDLE>(42);
    };
    BOOL FindNext(HANDLE hFindFile, LPWIN32_FIND_DATAW lpFindFileData) {
        BOOST_CHECK_EQUAL(reinterpret_cast<HANDLE>(42), hFindFile);
        if (fakeData.empty()) {
            SetLastError(ERROR_NO_MORE_FILES);
            return 0;
        }
        *lpFindFileData = fakeData.front();
        fakeData.pop_front();
        return 1;
    };
    BOOL Close(HANDLE hFindFile) {
        BOOST_CHECK_EQUAL(reinterpret_cast<HANDLE>(42), hFindFile);
        return 1;
    };
};

std::deque<WIN32_FIND_DATAW> SimpleFakeFindXFileFunctions::fakeData;
std::wstring SimpleFakeFindXFileFunctions::insertedFileName;

struct ErroneousFindXFileFunctions
{
    virtual HANDLE FindFirst(LPCWSTR, LPWIN32_FIND_DATAW) {
        return reinterpret_cast<HANDLE>(42);
    };
    virtual BOOL FindNext(HANDLE hFindFile, LPWIN32_FIND_DATAW) {
        BOOST_CHECK_EQUAL(reinterpret_cast<HANDLE>(42), hFindFile);
        return 1;
    };
    virtual BOOL Close(HANDLE hFindFile) {
        BOOST_CHECK_EQUAL(reinterpret_cast<HANDLE>(42), hFindFile);
        return 1;
    };
};

struct ErroneousFindXFileFunctionFirst : public ErroneousFindXFileFunctions
{
    HANDLE FindFirst(LPCWSTR, LPWIN32_FIND_DATAW) {
        SetLastError(ERROR_ACCESS_DENIED);
        return INVALID_HANDLE_VALUE;
    };
};

struct ErroneousFindXFileFunctionNext : public ErroneousFindXFileFunctions
{
    BOOL FindNext(HANDLE hFindFile, LPWIN32_FIND_DATAW) {
        BOOST_CHECK_EQUAL(reinterpret_cast<HANDLE>(42), hFindFile);
        SetLastError(ERROR_INVALID_PARAMETER);
        return 0;
    };
};

struct DirectoryIteratorTestsFixture
{
    typedef SimpleFakeFindXFileFunctions fakeFunctor;
    DirectoryIteratorTestsFixture() {
        WIN32_FIND_DATAW test;
        wcscpy_s(test.cFileName, L".");
        wcscpy_s(test.cAlternateFileName, L".");
        test.dwFileAttributes = FILE_ATTRIBUTE_DIRECTORY;
        GetSystemTimeAsFileTime(&test.ftCreationTime);
        test.ftLastWriteTime = test.ftCreationTime;
        test.ftLastAccessTime = test.ftCreationTime;
        test.nFileSizeHigh = 0;
        test.nFileSizeLow = 0;
        fakeFunctor::fakeData.push_back(test);

        wcscpy_s(test.cFileName, L"..");
        wcscpy_s(test.cAlternateFileName, L"..");
        fakeFunctor::fakeData.push_back(test);

        wcscpy_s(test.cFileName, L"File.txt");
        wcscpy_s(test.cAlternateFileName, L"FILE.TXT");
        test.nFileSizeLow = 1024;
        test.dwFileAttributes = FILE_ATTRIBUTE_NORMAL;
        fakeFunctor::fakeData.push_back(test);

        wcscpy_s(test.cFileName, L"System32");
        wcscpy_s(test.cAlternateFileName, L"SYSTEM32");
        test.nFileSizeLow = 0;
        test.dwFileAttributes = FILE_ATTRIBUTE_DIRECTORY;
        fakeFunctor::fakeData.push_back(test);
    };
    ~DirectoryIteratorTestsFixture() {
        fakeFunctor::fakeData.clear();
    };
};

BOOST_FIXTURE_TEST_SUITE( DirectoryIteratorTests, DirectoryIteratorTestsFixture )

BOOST_AUTO_TEST_CASE( BasicEnumeration )
{
    typedef DirectoryIterator<AllResults
        ,BasicNonRecursiveEnumeration<SimpleFakeFindXFileFunctions> > testType;
    testType begin(L"C:\\Windows\\*");
    testType end;
    BOOST_CHECK_EQUAL(fakeFunctor::insertedFileName, L"C:\\Windows\\*");
    BOOST_CHECK(begin->GetFolderPath() == L"C:\\Windows");
    BOOST_CHECK(begin->GetFileName() == L"File.txt");
    BOOST_CHECK(begin->GetFullFileName() == L"C:\\Windows\\File.txt");
    BOOST_CHECK(begin->GetShortFileName() == L"FILE.TXT");
    BOOST_CHECK_EQUAL(begin->GetSize(), 1024);
    BOOST_CHECK(begin->IsFile());
    BOOST_CHECK(begin != end);
    begin++;
    BOOST_CHECK(begin->GetFileName() == L"System32");
    BOOST_CHECK(begin->GetFullFileName() == L"C:\\Windows\\System32");
    BOOST_CHECK(begin->GetShortFileName() == L"SYSTEM32");
    BOOST_CHECK_EQUAL(begin->GetSize(), 0);
    BOOST_CHECK(begin->IsDirectory());
    begin++;
    BOOST_CHECK(begin == end);
}

BOOST_AUTO_TEST_CASE( NoRootDirectories )
{
    typedef DirectoryIterator<AllResults
        ,BasicNonRecursiveEnumeration<SimpleFakeFindXFileFunctions> > testType;
    fakeFunctor::fakeData.pop_front();
    fakeFunctor::fakeData.pop_front();
    testType begin(L"C:\\Windows\\*");
    testType end;
    BOOST_CHECK_EQUAL(fakeFunctor::insertedFileName, L"C:\\Windows\\*");
    BOOST_CHECK(begin->GetFolderPath() == L"C:\\Windows");
    BOOST_CHECK(begin->GetFileName() == L"File.txt");
    BOOST_CHECK(begin->GetFullFileName() == L"C:\\Windows\\File.txt");
    BOOST_CHECK(begin->GetShortFileName() == L"FILE.TXT");
    BOOST_CHECK_EQUAL(begin->GetSize(), 1024);
    BOOST_CHECK(begin->IsFile());
    BOOST_CHECK(begin != end);
    begin++;
    BOOST_CHECK(begin->GetFileName() == L"System32");
    BOOST_CHECK(begin->GetFullFileName() == L"C:\\Windows\\System32");
    BOOST_CHECK(begin->GetShortFileName() == L"SYSTEM32");
    BOOST_CHECK_EQUAL(begin->GetSize(), 0);
    BOOST_CHECK(begin->IsDirectory());
    begin++;
    BOOST_CHECK(begin == end);
}

BOOST_AUTO_TEST_CASE( Empty1 )
{
    fakeFunctor::fakeData.clear();
    typedef DirectoryIterator<AllResults
        ,BasicNonRecursiveEnumeration<SimpleFakeFindXFileFunctions> > testType;
    testType begin(L"C:\\Windows\\*");
    testType end;
    BOOST_CHECK(begin == end);
}

BOOST_AUTO_TEST_CASE( Empty2 )
{
    fakeFunctor::fakeData.erase(fakeFunctor::fakeData.begin() + 2, fakeFunctor::fakeData.end());
    typedef DirectoryIterator<AllResults
        ,BasicNonRecursiveEnumeration<SimpleFakeFindXFileFunctions> > testType;
    testType begin(L"C:\\Windows\\*");
    testType end;
    BOOST_CHECK(begin == end);
}

BOOST_AUTO_TEST_CASE( Exceptions )
{
    typedef DirectoryIterator<AllResults,BasicNonRecursiveEnumeration<ErroneousFindXFileFunctionFirst> >
        firstFailType;
    BOOST_CHECK_THROW(firstFailType(L"C:\\Windows\\*"), WindowsAPI::ErrorAccessDeniedException);
    typedef DirectoryIterator<AllResults,BasicNonRecursiveEnumeration<ErroneousFindXFileFunctionNext> >
        nextFailType;
    nextFailType constructedOkay(L"C:\\Windows\\*");
    BOOST_CHECK_THROW(constructedOkay++, WindowsAPI::ErrorInvalidParameterException);
}

BOOST_AUTO_TEST_CASE( CorrectDestruction )
{
    typedef DirectoryIterator<AllResults
        ,BasicNonRecursiveEnumeration<SimpleFakeFindXFileFunctions> > testType;
    testType begin(L"C:\\Windows\\*");
    testType end;
    BOOST_CHECK(begin != end);
}

BOOST_AUTO_TEST_SUITE_END()

Tonguetied

If you want to mock out Windows API calls I wouldn't worry about making functions virtual. Why do you avoid this? – Mullah

@epronk: Simply because there is no reason to do so -- it's worse off than my current method. At least with the templates there is no runtime penalty. – Tonguetied

So, your tests run really fast. How long does compiling and linking the test take? – Mullah

@epronk: I don't care about the test code run time. I care about production code run time. Compiling and linking is still plenty fast. If compile time becomes too long I'll look into other solutions but at present time I like the template method. Particularly given that a lot of my code is templates to begin with. – Tonguetied

The runtime penalty of using virtual relative to the time it takes to do the Windows API call is not even measurable. You complicate your production code to make it testable. This is sub-optimization. – Mullah

Why do you think a Windows API call takes a long time? – Tonguetied

@epronk: Granted, the FindXFile functions take a long time but there are several API functions that are as fast as the functions in your own code. – Tonguetied

@Billy, have you measured the overhead imposed by virtual functions? – Amazed

@Marcelo Cantos: No, but runtime cost is not why I did this with templates in the first place. Geez -- I mention one thing about runtime and everyone jumps down my throat. I'm just not going to replace perfectly working template code with code relying on virtual functions without good reason. Personally, I think solving this issue with either templates or virtuals work like garbage which is why I was wondering if there was a better way. – Tonguetied

@Billy, everyone "jumps down [your] throat" because you weren't very clear on your motivations. You simply stated that templates were frustrating you (though you say in a comment that you like the template method) but that you wanted to avoid virtual functions. The logical conclusion was that you have some beef with virtual functions, and the usual beef that people have is performance. Sorry for misunderstanding you. – Amazed

@Marcelo Cantos: No problem :) I'm sorry for misstating my goals. Three cheers for feeling sorry for ourselves! :) – Tonguetied

This is a pretty cool solution and before I go around posting a possibly duplicate, is there some simple alternative to mocking own code that is including unavailable source code (reason: proprietary code that is only available in certain machines)? – Brandonbrandt

Answer 1

Avoid using this template construction for this purpose unless you have some really core piece of code which has to run as fast as possible. If you want to avoid virtual for performance reasons, measure the difference.

Use the template construction only in places where it really makes a difference.

Try Google Mock. EXPECT_CALL is really powerful and saves a lot of code time compared to writing a custom mock.

Avoid mixing the terms Fake and Mock as they have different meaning.

DirectoryIterator<FakeFindFirstFile> LookMaImMocked; // is it a fake or a mock?

Mullah

#1 Why should I avoid templates? Half the code I'm testing is already a template anyway, so I don't see why I'd bother creating additional inheritance headaches by defining runtime polymorphism as well. #2: How do I provide test data for Google Mock's EXPECT_CALL? Honestly, I do not care of the function is called. I just want to be able to inject test data. #3 Ok, I will attempt to avoid mixing the meanings. Would you care to explain the difference? – Tonguetied

@epronk: As for the templates, I think you should know I got the idea from Google Mock: code.google.com/p/googlemock/wiki/… – Tonguetied

You're asking help with mocks and are not interested in setting up expectations? There is nothing wrong with this technique of mocking non-virtual methods, but you should just not apply it in the wrong context. – Mullah

@epronk: Yes, that is correct. Perhaps your understanding of Mock and my understanding are different (Mine is probably wrong). The only thing I need this for is to test wrappers around legacy C APIs. I don't care if things are called or not, I just need to be able to fake the C API upon which a wrapper class depends. – Tonguetied

@epronk: I have edited an example class and example tests into my question if you want to see what I mean. – Tonguetied

Did some more research and it appears that what I want to do is not "Mocking" but "Faking". Accepting this answer because if I really wanted mocking this would probably be what I'd need to do. – Tonguetied

Answer 2

Fwiw, I just ran across a new (c. 2011) mock framework called "turtle" that's designed to complement boost::test. It's on sourceforge. I'm just starting down the mocking path on a project and turtle will be my first choice unless it just doesn't work right.

Judon

I like the looks of Turtle (sourceforge.net/apps/mediawiki/turtle/index.php) too. – Hydrograph

Answer 3

Disclaimer: I'm the author of HippoMocks

I suggest using a mocking framework that can directly mock those API functions. HippoMocks can mock plain C functions and should have no trouble mocking Windows API functions directly. I'll give it a test run tonight and see if it runs.

Hope you're still reading replies :-)

Mayce