Professional, zero-cost setup for C++ projects (Part 2 of N)

Last time, we briefly explained how to setup a C++ project for cross-compilation using CMake, and applied that to a tiny toy project. Today, we are going to dive a bit deeper into our project’s setup, and see how we can import dependencies in it.

We want to add a unit-testing library to the project we started on Part 1, in order to write tests validating the correctness of our code. How can we achieve that easily and painlessly ?

The problem

Dependency management is a hot topic in C++, because unlike some other languages, we actually lack a standardized dependency management tool. As a result, the competing standards effect applied to this field, and we can now find quite a bunch of competing solutions to allow dependency management in C++, each of which has its specific pros and cons.

Existing solutions

Without taking part in the (sometimes holy) wars between the existing tools, let’s make a quick recap of the existing solutions for managing your project’s dependencies.

Manual import

This is the most primitive form of dependency management: when something is required in the project, you just pull it in manually. This can be achieved for example through Git submodules, or even by simply copying the dependency’s source code into your project’s directory.

  • It’s simple to setup
  • It doesn’t require any new tool
  • You have to figure by yourself whether your dependencies have their own dependencies, add those, and repeat this process
  • You have to update everything by hand

The system’s package manager

Another solution is to use the local package manager, that is, the one already shipped with your operating system. For example, if you are a fellow Debian user, you can find many famous libraries (eg. Boost) available through APT.

  • It doesn’t require any new tool either
  • Installation and updates are both easy, and could be scripted
  • A package manager is tied to a specific platform, so the installation procedures will differ for each platform supported by the project
  • The packages are installed in a system-wide manner, which can cause version clashes, or simply break things: more generally these package managers are not meant to be used for development
  • You cannot choose the version of the dependencies you install, you have to pick the one offered by the package manager
  • Worse, even slightly different platforms might not ship the same versions of the dependencies

Per-project package managers

In reaction to the former category, many “per-project” or “project-centric” package managers were designed. Unlike system-wide package managers, these ones only manage packages at the project level, thus avoiding many of the issues listed above.

At the time there are many of them, and we cannot describe them fully in a single article, yet here is a short list of the most famous ones:

  • Hunter, a CMake-based tool, thus supposed to be entirely compatible with any platform/libraries for which CMake can be used
  • VCPKG, an ecosystem managed by Microsoft and fueled by its community, primarily designed for Windows but also compatible with Linux and OSX through CMake
  • Conan, a decentralized package manager supporting multiple build systems, with many community-contributed packages

Each of them of course has its advantages and its issues (not only in the technical field, but also regarding its ecosystem), however discussing them is way beyond the scope of this article, and would be (almost) as much a matter of personal taste as it is a matter of features and trade-offs.

For this article, we will choose Conan as a package manager, because it has a good CMake integration (but not only), and because its “public federation” approach makes it easy for library developers to package and upload their own projects. Additionally, it is quite well-documented.

We don’t claim that Conan is perfect or that it is the absolute best tool around, however it is a pretty good solution to our package management problem.

Getting to know Conan

First, we obviously have to install Conan on our local machine. This can be achieved using Python’s PIP tool:

Note that if you don’t want to use PIP, there are also other ways of installing Conan, however this is the preferred way.

Remotes

In order to fetch your dependencies and download them to your machine when needed, Conan defines the concept of remotes. On the user side, a remote is merely a name and a URL pointing to a server. That server acts like an index referencing packages and ways to download them.

By default, Conan comes with a single pre-configured remote: the Conan Center, the official remote indexing the packages approved by the Conan team. However, a bunch of community-driven remotes can be added to access more packages.

For example, if we want to add the Bincrafters remote, which is one of the most important remotes in the community:

conan remote add bincrafters https://api.bintray.com/conan/bincrafters/public-conan

We can then check the available remotes, and notice that the newly-added Bincrafters remote is now part of the list:

conan remote list
conan-center: https://conan.bintray.com [Verify SSL: True]
bincrafters: https://api.bintray.com/conan/bincrafters/public-conan [Verify SSL: True]

Adding a bunch of remotes manually can be quite cumbersome, so here is a tip to export your remotes:

mkdir my_conan_setup
conan remote list --raw > my_conan_setup/remotes.txt

And then re-install them all at once, for example on another machine:

conan config install my_conan_setup/
Defining remotes from remotes.txt

If your project requires additional remotes, this could be a way of sharing them in your repository, so that contributors can add them easily.

Finding packages

Now that we have some remotes configured, we can use them to browse the available packages. For that, we can use the conan search subcommand, like so:

Let’s use that command to demonstrate how one can find the packages associated with the Boost libraries:

conan search “boost” --remote conan-center
Existing package recipes:
boost/1.64.0@conan/stable
boost/1.65.1@conan/stable
boost/1.66.0@conan/stable
boost/1.67.0@conan/stable
boost/1.68.0@conan/stable
boost/1.69.0@conan/stable
boost/1.70.0@conan/stable

As we notice above, the command found a few packages, and displayed some information about them. Let’s take a quick pause to make sure we understand what the results mean.

For the result displayed as boost/1.70.0@conan/stable, we have:

  • boost, which is the name of the package
  • 1.70.0, which is the version of the packaged library
  • @conan, which designates that the package is owned by the Conan Center, the official remote for Conan
  • stable, which is called the “channel”, and introduces a tag for the variants of the library (stable, testing, etc…)

All these fields collected together are called a package reference. Such a reference can be used to uniquely identify a package.

Installing packages

Since we now know how to look for the packages we want, the only thing left to learn is how to install them. Conan provides two different ways of achieving just that, both through the conan install subcommand.

Using the command line

If you want to install a package using the command line, all you have to do is find its package reference, and pass it as an argument to conan install, like shown below:

conan install boost/1.70.0@conan/stable
[…]

The install subcommand has many options, for example --build, which allows building dependencies from sources if no binary version could be found for your platform. Explore them !

Using a conanfile

While installing from the command line is fine for a one-time installation, it is not simple enough for, say, distributing a project. In that case, you would want a way of installing of the project’s dependencies at once, with a single command. Luckily, Conan has just what you need !

All you have to do is state the packages you need in a file named conanfile.txt, and ship it with your project. However, a conanfile is not only a way to list dependencies, but also acts as a manifest for the project, and allows various options to be specified.

cat conanfile.txt
[requires]
boost/1.70.0@conan/stable
conan install .
[…]

Integrating Conan within a project

Now that we know our way around Conan a little bit, we can start using it within our toy project. Let’s take the problem stated at the beginning of the article and solve it using Conan.

Ideally, we would like our targets to be built using just a few commands, like so:

mkdir build
cd build
conan install ..
cmake ..
make

For the rest of this post, we chose doctest as the unit-testing library, because it is both lightweight and feature-rich. However, the procedure to install other libraries should not differ too much from what we’ll show here.

Writing the conanfile

We will start by writing a conanfile.txt, and fill it. Just as we did before with boost, we first search the available packages for doctest.

conan search “doctest” --remote conan-center
Existing package recipes:
doctest/1.2.6@bincrafters/stable
doctest/2.0.0@bincrafters/stable
doctest/2.0.1@bincrafters/stable
doctest/2.2.0@bincrafters/stable
doctest/2.3.1@bincrafters/stable

Perfect ! The Conan Center seems to be referencing some packages hosted by Bincrafters that match our needs. Let’s add it to our conanfile.txt:

Before we are done with configuring Conan, there is an extra step we have to take care about. Since we use CMake as a build system, we have to make the required dependencies available for use within CMake. To achieve that, Conan accepts a generators section in the conanfile.txt, in which we can specify that we want to use cmake.

Integrating into CMake

At the previous step, we asked Conan to use CMake as a generator. As a result of that, Conan will generate a file named conanbuildinfo.cmake when we run conan install. That file must be included in our CMakeLists.txt to access the required dependencies.

We could just include that file, however in order to keep reasonable error messages, we can ensure that the file exists, and otherwise ask the user to execute Conan.

Right after that, we also need to call a Conan-provided macro, conan_basic_setup. This macro can be used in two different ways:

  • Without arguments, in which case it will make all the dependencies’ flags available globally in CMake.
  • With TARGETS as argument, which will cause CMake to define custom library targets for each of the dependencies. We can then select the dependencies to link against individually

In order to avoid bloating CMake’s global configuration, it is usually advised to go with the latter, so we will stick to it here:

Using the imported library

At this point, the only thing left to do is to actually use our newly-imported dependency ! We will append the following few lines to the client’s sub-CMakeLists.txt file:

The first two lines are pretty straightforward: we create a new executable target, specify its sources and configure it. The third line is used to link against the target created by Conan for our dependency, which is exactly how we would proceed for a regular library.

As you noticed in the code above, the target we created for our unit tests requires the file test/client.test.cpp. Let’s create that file, and fill it with the default doctest configuration:

We can now compile it and run it:

mkdir build
cd build
conan install ..
cmake ..
make client-test
./bin/client-test
[doctest] doctest version is “2.3.1”
[doctest] run with “--help” for options
===============================================================================
[doctest] test cases:      0 |      0 passed |      0 failed |      0 skipped
[doctest] assertions:      0 |      0 passed |      0 failed |
[doctest] Status: SUCCESS!

The steps for this section should be repeated for each of the sub-CMakeLists.txt in the project.

Conclusion

We now have a way to import dependencies into our project, and to use them in our code.
It’s time for a commit ! Like last time, everything written during this article can be found on the corresponding GitHub repository.

That’s it for today ! Next time, we will talk about writing unit tests using doctest. Stay tuned !


Comments (6)

  1. I cloned this repo and I was having issues building this part of your project.

    After checking-out article_part_two I ran the commands to build everything, but I received compiler warnings when I ran make:


    In file included from /Users/andfranklin/Projects/zero-cost-cpp-project/client/test/client.test.cpp:6:
    /Users/andfranklin/.conan/data/doctest/2.3.1/bincrafters/stable/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/include/doctest/doctest.h:395:18: error: use of
    undeclared identifier 'nullptr'
    typedef decltype(nullptr) nullptr_t;

    [...]

    fatal error: too many errors emitted, stopping now [-ferror-limit=]
    20 errors generated.
    make[2]: *** [client/CMakeFiles/client-test.dir/test/client.test.cpp.o] Error 1
    make[1]: *** [client/CMakeFiles/client-test.dir/all] Error 2
    make: *** [all] Error 2

    It seems that the compiler is not directed to use a c++ standard compatible with doctest. I don’t know much about cmake, but after some digging I added set(CMAKE_CXX_STANDARD 17) to the root CMakeLists.txt. This appears to fix all of the errors.

    Is this the correct approach? Moreover, do you know of any good references where one could learn more about cmake?

    1. Hello Andrew,

      It seems that the default compiler / settings used by your machine were configured to use a pre-C++11 environment, which is not compatible with doctest.

      For CMake, I would recommend this book: https://crascit.com/professional-cmake.

      Also, if you are running CMake >= 3.8, you can use target_compile_features(your_target PRIVATE cxx_std_17) to configure a single target. Usually, target-based configuration is preferred as it avoids affecting CMake’s global configuration.

      Best regards,

      Roman

      1. Hi Roman,

        Thank you for the response. I am using cmake version 3.14.3 — I just updated it today. Maybe there was some change between 3.8 and 3.14 that is causing the issues?

        Regards,
        Andrew

        1. Hello,

          Probably a problem with your conan profile you can try conan profile show default and manually set the cpp standard to c++ 17: conan install ../ -s cppstd=17

          Best Regards,
          Roman

          1. Hi,

            I tried running with conan install ../ -s cppstd=17. I also even added cppstd=17 under settings in ~/.conan/profiles/default. Neither fixed the problem.

            The only thing that seems to fix the problem is adding set(CMAKE_CXX_STANDARD 17) to CMakeLists.txt. =/

            Thank you for your diligence in working through this with me.

            Regards,
            Andrew

Leave a Reply

Your email address will not be published. Required fields are marked *