Javier G. Sogo

Javier G. Sogo

Software developer

May the forc++ be with you

Magnum example using Emscripten

November 10, 2021
c++
wasm
emsdk
magnum
conan

Emscripten

If you are only interested in reproducing the example, go to the last section of this blogpost.

Some months ago I joined a 3D course using Blender. The main purpose was to recover this hobby I enjoyed a lot in the good old days, but soon my mind started to think about showing those 3D models in the browser. Using C++, of course. Spoiler: I didn't finish the course.

Anyway, before moving forwards, I want to show you the viking shield I managed to do in the first lessons. I really enjoyed it and I can only recommend all the courses that Pablo and Koré team will organize in the future.

Viking shield
Viking shield recreation

Looking for alternatives

When it comes to web and C++ there are not too many options out there. So far I can only think about Qt and Wt, any other alternative would involve dealing with HTML and CSS. In this case I didn't want a fully featured application, I just wanted to render some 3D.

For this example I was going to need some graphics library able to import the 3D data and the corresponding graphic engine to render it. There are many libraries that satisfies these two requirements (it remembers me two years ago when for some reason lot of people from the C++ community started to build their own raytracing implementation), but when you start to search for 3D and web, and you add Emscripten to the equation the alternatives are narrowed. In fact, I only found Magnum satisfying all these requirements.

Magnum libraries

Magnum looked like a good approach: modern library, active development, modular architecture, plugins,... with a lot of examples showcasing different examples built using Emscripten.

Magnum features chart
Magnum architecture is organize into layers that add functionality on top of each other. The foundation are the core libraries that provides some platform abstraction and very basic features. On top of these layers, some extra modules contribute with actual functionaliy for different use-cases. This architecture makes it easy to contribute new modules and extensions.

For this example I just needed to build the core libraries with some OpenGL (WebGL) support, and the required components to import my 3D models. Anyway, those were a bunch of libraries using different build systems.

Building the libraries

Magnum project is quite good at documentation, and a lot of attention has been paid to the build system. Magnum provides good and modern CMakeLists.txt files and every single option is exhaustively documented.

The core library Magnum depends only on a bunch of third party libraries, and some of them are optional depending on the active features. Many other libraries are required if we start to add plugins related to other features: mesh importers, fonts, images, sound,...

Magnum direct dependencies
Magnum direct dependencies.

There are plenty of plugins in the magnum-plugins

Created by @mosra
Plugins for the Magnum C++11/C++14 graphics engine

47 forks 78 stars
repository. I won't need all of them, but the following graph shows the third party libraries that are required if we want to activate them. Of course, for this little example we don't need all of them, they are just listed here for completeness.

Magnum plugins direct dependencies
Magnum plugins dependencies.

If you have some experience with C++ ecosystem, soon you will think that these are too many libraries and it will be very challenging to build all of them from sources. Thankfully, some of them can be installed at system level using the system package manager or similar tools. Nevertheless this would introduce some limitations and compatibility issues, but this is not the place to talk about it.

Given all these libraries I decided to approach the problem using a package manager. Of course, the choosen one was Conan (I work in the project). Thanks to the community effort most of the libraries were already available in ConanCenter, but still there were some missing libraries.

Conanize all the things

It took some time to create all the missing recipes and improve existing ones to be able to conanize all the libraries in the Magnum graph, but hopefully this effort is worth it and other people from the community will benefit from it.

In this issue you can find most of the changes that were required to add magnum/2020.06 to ConanCenter. As you can see it took me around three months to contribute all the changes and there are still some missing improvements, mostly related to Emscripten, but it's not clear to me how to address them in the context of conan-center-index

Created by @conan-io
Recipes for the ConanCenter repository

700 forks 479 stars
repository.

Of course, it hasn't been only my work, but the effort of several contributors from the community that helped me creating other recipes. Thanks and kudos to all of them. Now you can just install all required libraries with Conan and use it in your project:

conan install magnum/2020.06@

Emscripten

Emscripten is a toolchain based on LLVM to generate WebAssembly binaries. These binaries can run in the browser. In my honest opinion is one of the most interesting technologies today: compact output, near-native speed and same source code as desktop.

From a C++ developer point of view, it's just a cross-building toolchain that generates some .wasm and .js files that can be loaded and executed by the browser. Due to security reason, access to system resources is limited and that's the reason by some libraries don't build out of the box and need some modifications (see emscripten-ports). There are two main limitation we will face in this example: OpenGL is substituted by WebGL (it supports only a subset of OpenGL ES 2.0/3.0), and access to filesystem is forbidden (although it can be virtualized).

Thanks to werto87 there is a recipe for emsdk available in ConanCenter, so crossbuilding our libraries to WebAssembly will be straightforward using this package as a Conan's build-requirement. Below I will list the commands you need to execute.

Magnum and SDL

However, when it comes to SDL requirement in Magnum there is a problem with several root causes. Magnum, when building for desktop, depends on SDL v2, but when building for web it uses SDL v1. Why? SDL v1 is basically deprecated, and that's probably the reason why Magnum started to use the new version, but Emscripten still provides vendorized SDL v1 so it is probably easier to keep using old functions.

From Conan perspective this is quite unfortunate. We only have SDL v2 available in ConanCenter so we cannot switch the requirement when building for the web and we need to use the vendorized SDL version in emsdk. It works so far, but we loose track of this dependencie and possible transitive requirements.

I don't have enough SDL expertise to contribute a PR to Magnum and move the requirement to v2 and, on the other hand, I didn't manage to conanize SDL v1 (and it's probably not worth it since it is mainly deprecated).

Here we are likely in a dead-end if we want to do the things right, I would love to update this blogpost in the near future saying that everything has been fixed and ready. Meanwhile we need to accept the workarounds mentioned above and keep going.

Show me the code! (reproducible steps)

Time for fun! In this first attemp I'm just copying the viewer example from the magnum-examples repository

Created by @mosra
Examples for the Magnum C++11/C++14 graphics engine

79 forks 205 stars
and modify it a little bit to load the STL model. In this blogpost I'm not explaining the logic or entering the source code, the intention here is to show that it works and provide a starting point for some actual development.

All required sources can be found in my GitHub account

Created by @jgsogo
Example using `magnum` to render 3D in the browser

0 forks 1 stars
, so the first step is to clone the repository to your local computer:

git clone https://github.com/jgsogo/blog-emsdk-magnum-viewer
cd blog-emsdk-magnum-viewer

In this project I will use Conan revisions and lockfiles, the purpose is to achieve reproducibility and guarantee that it will work in the future. Using these two features together we can force the recipe revisions that we know that work.

Let's configure Conan in a local folder, so we don't disturb global installation:

export CONAN_USER_HOME=$(pwd)
conan config set general.revisions_enabled=1

Now we are going to export two recipes with some modifications that are needed to work with Emscripten. These changes haven't been contributed to the conan-center-index

Created by @conan-io
Recipes for the ConanCenter repository

700 forks 479 stars
repository yet, we probably need to think and decide first about this issue.

conan export .conan/recipes/magnum/all/conanfile.py magnum/2020.06@
conan export .conan/recipes/libjpeg/all/conanfile.py libjpeg/9d@

Now create a move to the build directory:

mkdir cmake-build-emsdk && cd cmake-build-emsdk

And it's time to use the lockfiles. A Conan lockfile is a file that states the exact revision of the packages involved in a dependency graph. Conan can use one of these files to download the packages from the remote and reproduce exactly the same scenario in another computer. However, as we said before, the dependencies are not exactly the same when building for desktop or emscripten (sometimes they are different for different configurations), so a true lockfile won't fit our needs. Fortunately, Conan provides base lockfiles that capture only the recipe revision and this is what I'm using here and it is stored in the lockfile.json file.

Using this base lockfile, we can create the full lockfile that matches our configuration. Different configs will use different packages (different binaries), but still they will use the same recipe revision.

conan lock create --profile:host=../.conan/profiles/emsdk --profile:build=default --lockfile=../lockfile.json --lockfile-out=lockfile.json --name=viewer --version=0.1 ../conanfile.txt --build --update

Now, we can use the generated file to install all the requirements in our dependency graph:

conan install --lockfile=lockfile.json ../conanfile.txt --build=missing --generator=virtualenv

Conan has generated everything we need to build the project: the module files that will be consumed by CMake (the well-known FindXXX.cmake files for the libraries) and some bash scripts to populate the environment with the information that the build-system needs to use the Emscripten toolchain.

Next step is to activate the environment and build the project as usual:

source activate.sh
cmake .. -DCMAKE_MODULE_PATH=$(pwd) -DCMAKE_TOOLCHAIN_FILE=$CONAN_CMAKE_TOOLCHAIN_FILE
make

The build should succeed, at least it does in the CI (see badge at the top). In order to see it working in your computer, you just need to start a web server and point your browser to the HTML file:

python -m http.server --directory bin
http://localhost:8000/viewer.html

Enjoy!

Here you have it: the Conan cube rendered using WebAssembly, use the moouse to navigate and move the object:

Initialization...