In fall of 2010 we decided to go cross platform with our quantitative finance tool UnRisk-Q. The library was initially developed for Windows only, but the ongoing shift in platform popularity made us consider also offering it for Linux and Mac OS X. Mathematica, which forms the basis of UnRisk Financial Language, is also available for these three platforms.
When we started, the whole build process of UnRisk-Q was based on manually maintained Visual Studio C++ projects. We looked at different cross platform build tools and finally settled on using CMake as our build tool for the following reasons:
It generates native build solutions (Visual Studio projects on Windows, Xcode projects on Mac OS X and Makefiles under Linux).
Unlike other build tools, it does not have a platform bias, but works equally well on the three target platforms Windows, Linux and Mac OS X.
CMake covers the whole software build process consisting of building, automated testing, code coverage, continuous integration and packaging.
A CMake installation is fully self contained and does not depend on a third-party scripting language.
Once the build system was chosen, the existing C++ code needed to be made cross platform. This is a straight-forward process, which requires replacing platform specific code with platform agnostic one where possible and insulating the platform specific code that remains. In doing that, we often had to make changes to widely used project header files, which triggered a rebuild of the whole project. Since UnRisk-Q’s code base consists of about a half a million lines of C++ code, this meant that we had to wait almost half an hour for a build to finish.
The Preprocessor Takes the Blame
A short C++ program, which consists of about 100 lines of source code, is turned into a 40000 line compilation unit by the preprocessor which handles the inclusion of standard headers. So all a C++ compiler does these days is to continually parse massive compilation units. Since any complex C++ project consists of dozens of C++ source code files and many of the source files use the same standard headers, the C++ compiler has to do a lot of redundant work.
The downsides of the preprocessor have been known for a long time. In his book The Design and Evolution of C++ Bjarne Stroustrup made the following statement about the preprocessor (Cpp): “Furthermore I am of the opinion that Cpp must be destroyed.” The book was released in 1994. 20 years later the preprocessor is still alive and kicking in the world of C++ programming.
The preprocessor is here to stay, so two different techniques have been developed to speed up preprocessing. The first one is precompiled header (PCH), the other one is single computation unit which is more commonly known as “unity builds”. Both techniques are good ideas in principle, they however failed to gain wide use in many C++ projects for the following reasons:
- Precompiled headers require the creation and continuous manual maintenance of a prefix header.
- C++ compiler vendors have implemented PCH support in different, incompatible ways.
- Unity builds break the use of many C++ language features. They may cause unintended collisions of global variable and macro definitions. Thus unity builds rarely work without source code modifications.
- Most C++ projects start out small and grow over time. When the need for adding PCH or unity build support is felt, it is too much work to incorporate it into the existing build system.
Given the modern build infrastructure that CMake provides, I thought that adding support for precompiled headers and unity builds should be as easy as stealing candy from a baby. I couldn’t be more wrong. The existing solutions at that time only were hacks divorced from software engineering reality. So this was clearly a case, where Jean-Baptiste Emanuel Zorg’s rule applies. On top of that it was an interesting weekend project to take on.
Designing the Interface
Interface wise I wanted to be able to speed up the compilation of a CMake project by using one of the simplest technical interfaces known to man:
In programming terms, this means that if you have a CMake project which creates an executable:
add_executable(example main.cpp example.cpp log.cpp)
you just call a function with the corresponding CMake target:
cotire(example)
cotire is an acronym for compile time reducer. The function then should do its magic of speeding up the build process. It should hide all the nasty details of setting up the necessary build processes and should work seamlessly for the big four C++ compiler vendors, Clang, GCC, Intel and MSVC.
Once you have designed an interface that you think succinctly solves your problem, it is extremely important to fight the urge to make the interface more complicated than it needs to be just to make it cope with some edge cases. Giving in to that urge too early is the reason why software developers have to deal with subpar tools and libraries on a daily basis.
Implementation
A well designed interface should give you a crystal-clear view of the technical problems that need to be solved in order to make the interface work in reality. For cotire, the following problems needed to be solved:
- Generate a unity build source file.
- Add a new CMake target that lets you build the original target as a unity build.
- Generate a prefix header.
- Precompile the prefix header.
- Apply the precompiled prefix header to the CMake target to make it compile faster.
Using CMake custom build rules, cotire sets up rules to have the build system generate the following files at build time:
The unity build source file is generated from the information in the CMake list file by querying the target’s SOURCES property. It consists of preprocessor include directives for each of the target source files. The files
are included in the same order that is used in the CMake add_executable or add_library call.
This is the unity source generated for the example project under Linux:
#ifdef __cplusplus
#include "/home/kratky/example/src/main.cpp"
#include "/home/kratky/example/src/example.cpp"
#include "/home/kratky/example/src/log.cpp"
#endif
The prefix header is then produced from the unity source file by running the unity source file through the preprocessor and keeping track of each header file used. Cotire will automatically choose headers that are outside of the project root directory and thus are likely to change only infrequently.
For a complex CMake target, the prefix header may contain system headers from many different software packages, as can be seen in the example prefix header below generated for one of UnRisk-Q’s core libraries under Linux:
#pragma GCC system_header
#ifdef __cplusplus
#include "/usr/local/include/boost/tokenizer.hpp"
#include "/usr/local/include/boost/algorithm/string.hpp"
#include "/usr/include/c++/4.6/iostream"
#include "/usr/local/include/boost/lexical_cast.hpp"
#include "/usr/local/include/boost/date_time/gregorian/gregorian.hpp"
#include "/usr/local/include/boost/numeric/ublas/matrix.hpp"
#include "/usr/local/include/boost/numeric/ublas/matrix_proxy.hpp"
#include "/usr/local/include/boost/numeric/ublas/matrix_sparse.hpp"
#include "/usr/local/include/boost/numeric/ublas/banded.hpp"
#include "/usr/local/include/boost/numeric/ublas/triangular.hpp"
#include "/usr/local/include/boost/numeric/ublas/lu.hpp"
#include "/usr/local/include/boost/numeric/ublas/io.hpp"
#include "/usr/include/c++/4.6/set"
#include "/usr/include/c++/4.6/bitset"
#include "/usr/include/c++/4.6/cmath"
#include "/usr/local/include/boost/foreach.hpp"
#include "/usr/local/include/boost/regex.hpp"
#include "/usr/local/include/boost/function.hpp"
#include "/usr/local/include/cminpack-1/cminpack.h"
#include "/usr/local/include/cminpack-1/minpack.h"
#include "/usr/include/c++/4.6/fstream"
#include "/usr/include/c++/4.6/ctime"
#include "/usr/include/c++/4.6/numeric"
#include "/usr/include/c++/4.6/cfloat"
#include "/usr/include/c++/4.6/cstdlib"
#include "/usr/include/c++/4.6/cstring"
#endif
The precompiled header, which is a binary file, is then produced from the generated prefix header by using a proprietary precompiling mechanism depending on the compiler used. For the precompiled header compilation, the compile options (flags, include directories and preprocessor defines) must match the target’s compile options exactly. Cotire extract the necessary information automatically from the target’s build properties that CMake provides.
As a final step cotire then modifies the COMPILE_FLAGS property of the CMake target to force the inclusion of the precompiled header.
Speedup
With cotire we were able to cut the build time of the Windows version of UnRisk-Q by 40 percent:
Users who have adopted cotire for adding precompiled headers have reported similar speedup numbers.
With tools that are developed with a special in-house purpose in mind, it’s all too easy to fall into it works on my machine trap. Therefore we also applied cotire to some popular open source projects in order to test its general-purpose applicability. One project we tested it on is LLVM. LLVM is a huge C++ project with close to a million lines of code, yet the change set that is needed to apply cotire to it is just 100 lines of code. A cotire PCH build reduces the build time for LLVM 3.4 by about 20 percent:
One project where unity builds work out of the box without having to make changes to the source code is an example text editor application for Qt5. Applying a cotire generated precompiled headers to this project reduces compile time by the usual 20 percent, but doing a cotire unity build results in a reduction of 70 percent:
Other users of cotire have reported even larger speedups with cotire unity builds.
Conclusion
As described in the book The Cathedral and the Bazaar, one of the lessons for creating good open source software is that every good work of software starts by scratching a developer’s personal itch. Cotire has been released as an open source project in March 2012. Since then it has beed adopted by hundreds of open and closed source projects that use CMake as a build system. Among those are projects from Facebook and Netflix.
