Last time, we briefly explained how to setup a C++ project for cross-compilation using CMake, and applied that to a tiny toy project. Today, we are going to dive a bit deeper into our project’s setup, and see how we can import dependencies in it.
We want to add a unit-testing library to the project we started on Part 1, in order to write tests validating the correctness of our code. How can we achieve that easily and painlessly ?
The problem
Dependency management is a hot topic in C++, because unlike some other languages, we actually lack a standardized dependency management tool. As a result, the competing standards effect applied to this field, and we can now find quite a bunch of competing solutions to allow dependency management in C++, each of which has its specific pros and cons.
Existing solutions
Without taking part in the (sometimes holy) wars between the existing tools, let’s make a quick recap of the existing solutions for managing your project’s dependencies.
Manual import
This is the most primitive form of dependency management: when something is required in the project, you just pull it in manually. This can be achieved for example through Git submodules, or even by simply copying the dependency’s source code into your project’s directory.
- It’s simple to setup
- It doesn’t require any new tool
- You have to figure by yourself whether your dependencies have their own dependencies, add those, and repeat this process
- You have to update everything by hand
The system’s package manager
Another solution is to use the local package manager, that is, the one already shipped with your operating system. For example, if you are a fellow Debian user, you can find many famous libraries (eg. Boost) available through APT.
- It doesn’t require any new tool either
- Installation and updates are both easy, and could be scripted
- A package manager is tied to a specific platform, so the installation procedures will differ for each platform supported by the project
- The packages are installed in a system-wide manner, which can cause version clashes, or simply break things: more generally these package managers are not meant to be used for development
- You cannot choose the version of the dependencies you install, you have to pick the one offered by the package manager
- Worse, even slightly different platforms might not ship the same versions of the dependencies
Per-project package managers
In reaction to the former category, many “per-project” or “project-centric” package managers were designed. Unlike system-wide package managers, these ones only manage packages at the project level, thus avoiding many of the issues listed above.
At the time there are many of them, and we cannot describe them fully in a single article, yet here is a short list of the most famous ones:
- Hunter, a CMake-based tool, thus supposed to be entirely compatible with any platform/libraries for which CMake can be used
- VCPKG, an ecosystem managed by Microsoft and fueled by its community, primarily designed for Windows but also compatible with Linux and OSX through CMake
- Conan, a decentralized package manager supporting multiple build systems, with many community-contributed packages
Each of them of course has its advantages and its issues (not only in the technical field, but also regarding its ecosystem), however discussing them is way beyond the scope of this article, and would be (almost) as much a matter of personal taste as it is a matter of features and trade-offs.
For this article, we will choose Conan as a package manager, because it has a good CMake integration (but not only), and because its “public federation” approach makes it easy for library developers to package and upload their own projects. Additionally, it is quite well-documented.
We don’t claim that Conan is perfect or that it is the absolute best tool around, however it is a pretty good solution to our package management problem.
Getting to know Conan
First, we obviously have to install Conan on our local machine. This can be achieved using Python’s PIP tool:
1 |
pip3 install conan |
Note that if you don’t want to use PIP, there are also other ways of installing Conan, however this is the preferred way.
Remotes
In order to fetch your dependencies and download them to your machine when needed, Conan defines the concept of remotes. On the user side, a remote is merely a name and a URL pointing to a server. That server acts like an index referencing packages and ways to download them.
By default, Conan comes with a single pre-configured remote: the Conan Center, the official remote indexing the packages approved by the Conan team. However, a bunch of community-driven remotes can be added to access more packages.
1 |
conan remote add <name> <url> |
For example, if we want to add the Bincrafters remote, which is one of the most important remotes in the community:
We can then check the available remotes, and notice that the newly-added Bincrafters remote is now part of the list:
Adding a bunch of remotes manually can be quite cumbersome, so here is a tip to export your remotes:
And then re-install them all at once, for example on another machine:
If your project requires additional remotes, this could be a way of sharing them in your repository, so that contributors can add them easily.
Finding packages
Now that we have some remotes configured, we can use them to browse the available packages. For that, we can use the conan search
subcommand, like so:
1 |
conan search "search_pattern" --remote=<remote_name> |
Let’s use that command to demonstrate how one can find the packages associated with the Boost libraries:
As we notice above, the command found a few packages, and displayed some information about them. Let’s take a quick pause to make sure we understand what the results mean.
For the result displayed as boost/1.70.0@conan/stable
, we have:
boost
, which is the name of the package1.70.0
, which is the version of the packaged library@conan
, which designates that the package is owned by the Conan Center, the official remote for Conanstable
, which is called the “channel”, and introduces a tag for the variants of the library (stable
,testing
, etc…)
All these fields collected together are called a package reference. Such a reference can be used to uniquely identify a package.
Installing packages
Since we now know how to look for the packages we want, the only thing left to learn is how to install them. Conan provides two different ways of achieving just that, both through the conan install
subcommand.
Using the command line
If you want to install a package using the command line, all you have to do is find its package reference, and pass it as an argument to conan install
, like shown below:
The install
subcommand has many options, for example --build
, which allows building dependencies from sources if no binary version could be found for your platform. Explore them !
Using a conanfile
While installing from the command line is fine for a one-time installation, it is not simple enough for, say, distributing a project. In that case, you would want a way of installing of the project’s dependencies at once, with a single command. Luckily, Conan has just what you need !
All you have to do is state the packages you need in a file named conanfile.txt
, and ship it with your project. However, a conanfile is not only a way to list dependencies, but also acts as a manifest for the project, and allows various options to be specified.
Integrating Conan within a project
Now that we know our way around Conan a little bit, we can start using it within our toy project. Let’s take the problem stated at the beginning of the article and solve it using Conan.
Ideally, we would like our targets to be built using just a few commands, like so:
For the rest of this post, we chose doctest as the unit-testing library, because it is both lightweight and feature-rich. However, the procedure to install other libraries should not differ too much from what we’ll show here.
Writing the conanfile
We will start by writing a conanfile.txt
, and fill it. Just as we did before with boost
, we first search the available packages for doctest
.
Perfect ! The Conan Center seems to be referencing some packages hosted by Bincrafters that match our needs. Let’s add it to our conanfile.txt
:
1 2 |
[requires] doctest/2.3.1@bincrafters/stable |
Before we are done with configuring Conan, there is an extra step we have to take care about. Since we use CMake as a build system, we have to make the required dependencies available for use within CMake. To achieve that, Conan accepts a generators
section in the conanfile.txt
, in which we can specify that we want to use cmake
.
1 2 |
[generators] cmake |
Integrating into CMake
At the previous step, we asked Conan to use CMake as a generator. As a result of that, Conan will generate a file named conanbuildinfo.cmake
when we run conan install
. That file must be included in our CMakeLists.txt
to access the required dependencies.
We could just include
that file, however in order to keep reasonable error messages, we can ensure that the file exists, and otherwise ask the user to execute Conan.
1 2 3 4 5 |
if (NOT EXISTS conanbuildinfo.cmake) message(FATAL_ERROR "Conan needs to be executed: conan install <path_to_your_conanfile.txt>") endif () include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake) |
Right after that, we also need to call a Conan-provided macro, conan_basic_setup
. This macro can be used in two different ways:
- Without arguments, in which case it will make all the dependencies’ flags available globally in CMake.
- With
TARGETS
as argument, which will cause CMake to define custom library targets for each of the dependencies. We can then select the dependencies to link against individually
In order to avoid bloating CMake’s global configuration, it is usually advised to go with the latter, so we will stick to it here:
1 |
conan_basic_setup(TARGETS) |
Using the imported library
At this point, the only thing left to do is to actually use our newly-imported dependency ! We will append the following few lines to the client’s sub-CMakeLists.txt file:
1 2 3 |
add_executable(client-test test/client.test.cpp) target_compile_features(client-test PRIVATE cxx_std_17) target_link_libraries(client-test PRIVATE CONAN_PKG::doctest) |
The first two lines are pretty straightforward: we create a new executable target, specify its sources and configure it. The third line is used to link against the target created by Conan for our dependency, which is exactly how we would proceed for a regular library.
As you noticed in the code above, the target we created for our unit tests requires the file test/client.test.cpp
. Let’s create that file, and fill it with the default doctest configuration:
1 2 |
#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN #include <doctest.h> |
We can now compile it and run it:
The steps for this section should be repeated for each of the sub-CMakeLists.txt in the project.
Conclusion
We now have a way to import dependencies into our project, and to use them in our code.
It’s time for a commit ! Like last time, everything written during this article can be found on the corresponding GitHub repository.
That’s it for today ! Next time, we will talk about writing unit tests using doctest. Stay tuned !
I cloned this repo and I was having issues building this part of your project.
After checking-out
article_part_two
I ran the commands to build everything, but I received compiler warnings when I ranmake
:In file included from /Users/andfranklin/Projects/zero-cost-cpp-project/client/test/client.test.cpp:6:
/Users/andfranklin/.conan/data/doctest/2.3.1/bincrafters/stable/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/include/doctest/doctest.h:395:18: error: use of
undeclared identifier 'nullptr'
typedef decltype(nullptr) nullptr_t;
[...]
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make[2]: *** [client/CMakeFiles/client-test.dir/test/client.test.cpp.o] Error 1
make[1]: *** [client/CMakeFiles/client-test.dir/all] Error 2
make: *** [all] Error 2
It seems that the compiler is not directed to use a c++ standard compatible with doctest. I don’t know much about cmake, but after some digging I added
set(CMAKE_CXX_STANDARD 17)
to the rootCMakeLists.txt
. This appears to fix all of the errors.Is this the correct approach? Moreover, do you know of any good references where one could learn more about cmake?
Hello Andrew,
It seems that the default compiler / settings used by your machine were configured to use a pre-C++11 environment, which is not compatible with doctest.
For CMake, I would recommend this book: https://crascit.com/professional-cmake.
Also, if you are running CMake >= 3.8, you can use
target_compile_features(your_target PRIVATE cxx_std_17)
to configure a single target. Usually, target-based configuration is preferred as it avoids affecting CMake’s global configuration.Best regards,
Roman
Hi Roman,
Thank you for the response. I am using cmake version 3.14.3 — I just updated it today. Maybe there was some change between 3.8 and 3.14 that is causing the issues?
Regards,
Andrew
Hello,
Probably a problem with your
conan profile
you can tryconan profile show default
and manually set the cpp standard to c++ 17:conan install ../ -s cppstd=17
Best Regards,
Roman
Hi,
I tried running with
conan install ../ -s cppstd=17
. I also even addedcppstd=17
under settings in~/.conan/profiles/default
. Neither fixed the problem.The only thing that seems to fix the problem is adding
set(CMAKE_CXX_STANDARD 17)
toCMakeLists.txt
. =/Thank you for your diligence in working through this with me.
Regards,
Andrew
Hello,
I have no more idea, maybe a bug from CMake
Glad to help you,
Best Regards,
Roman