How to Compile Open Source Code

This book serves as a gentle introduction to anyone interested in how software is developed and how to participate in open source projects. It provides readers with the basic information they need to understand how the process works, interact with others and start a project of their own.

Open source projects make great enhancements to software, but compiling them can be confusing. Here are step-by-step instructions for compiling an open source package on Ubuntu Linux.

Download source code

Obtaining source code for an application is much like getting any downloadable software. You go to a website or a code management site like GitLab, SourceForge, or GitHub. Typically, open source software is available in both a work-in-progress (“current” or “nightly”) form as well as a packaged “stable” release version. Use the stable version when possible unless you have reason to believe otherwise or are good enough with code to fix things when they break. The term stable suggests the code got tested and that the programmers of the application feel confident enough in the code to package it into a .zip or .tar archive, give it an official number and sometimes a release name, and offer it for download to the general non-programmer public.

For this exercise, I’m using Angband, an open source (GPLv2) ASCII dungeon crawler. It’s a simple application with just enough complications to demonstrate what you need to consider when compiling software for yourself.

Download the source code from the website.

Understanding the Build System of the program

We usually talk about “compiling the sources”, but the compilation is only one of the phases required to produce a working software from its source. A build system is a set of tool and practices used to automate and articulate those different tasks in order to build entirely the software just by issuing few commands.

If the concept is simple, the reality is somewhat more complicated. Because different projects or programming language may have different requirements. Or because of the programmer’s tastes. Or the supported platforms. Or for historical reason. Or… or.. there is an almost endless list of reasons to choose or create another build system. All that to say there are many different solutions used out there.

NodeJS uses a GNU-style build system, it is a popular choice in the open source community and once again, a good way to start your journey.

Writing and tuning a build system is a pretty complex task, but for the “end user”, GNU-style build systems ease the task by using two tools: configure and make.

The configure file is a project-specific script that will check the destination system configuration and available feature in order to ensure the project can be built, eventually dealing with the specificities of the current platform.

An important part of a typical configure job is to build the Makefile. That is the file containing the instructions required to effectively build the project.

The make tool, on the other hand, is a POSIX tool available on any Unix-like system. It will read the project-specific Makefile and perform the required operations to build and install your program.

But, as always in the Linux world, you still have some leniency in customising the build to your specific needs.

./configure --help

The configure -help command will show you all the available configuration options. Once again, this is very project-specific. And to be honest, it is sometimes necessary to dig into the project before fully understanding the meaning of each and every configure option.

But there is at least one standard GNU Autotools option that you must know: the --prefix option. This has to do with the file system hierarchy and the place your software will be installed.

Unarchive the source code

Source code is often delivered as an archive because source code usually consists of multiple files. You have to extract it before interacting with it, whether it’s a tarball, a zip file, a 7z file, or something else entirely. 

tar --extract --file Angband-x.y.z.tar.gz

Once you’ve unarchived it, change the directory into the extracted directory and have a look around. There’s usually a README file at the top level of the directory. This file, ideally, contains guidance on what you need to do to compile the code. The README often contains information on these important aspects of the code:

  • Language: What language the code is in (for instance, C, C++, Rust, Python).
  • Dependencies: What other software you need to have installed on your system for this application to build and run.
  • Instructions: The literal steps you need to take to build the software. Occasionally, they include this information within a dedicated file intuitively entitled INSTALL.

If the README file doesn’t contain that information, consider filing a bug report with the developer. You’re not the only one who needs an introduction to source code. Regardless of how experienced they are, everyone is new to source code they’ve never seen before, and documentation is important!

Angband’s maintainers link to online instructions to describe how to compile the code. This document also describes what other software you need to have installed, although it doesn’t exactly spell it out. The site says, “There are several different front ends that you can optionally build (GCU, SDL, SDL2, and X11) using arguments to configure such as --enable-sdl--disable-x11, etc.” This may mean something to you or look like a foreign language, but this is the kind of stuff you get used to after compiling code frequently. Whether or not you understand what X11 or SDL2 is, they’re both requirements that you see pretty often after regularly compiling code over a few months. You get comfortable with the idea that most software needs other software libraries because they build upon other technologies. In this case, though, Angband is very flexible and compiles with or without these optional dependencies, so for now, you can pretend that there are no additional dependencies.

Getting the source code from GitHub

Like many open-source projects, the sources of NodeJS can be found on GitHub: https://github.com/nodejs/node

So, let’s go directly there.

The NodeJS official GitHub repository

If you’re not familiar with GitHubgit or any other version control system worth mentioning the repository contains the current source for the software, as well as a history of all the modifications made through the years to that software. Eventually up to the very first line written for that project. For the developers, keeping that history has many advantages. For us today, the main one is we will be able to get the sources from for the project as they were at any given point in time. More precisely, I will be able to get the sources as they were when the 8.1.1 version I want was released. Even if there were many modifications since then.

Choose the v8.1.1 tag in the NodeJS GitHub repository

On GitHub, you can use the “branch” button to navigate between different versions of the software. “Branch” and “tags” are somewhat related concepts in Git. Basically, the developers create “branch” and “tags” to keep track of important events in the project history, like when they start working on a new feature or when they publish a release. I will not go into the details here, all you need to know is I’m looking for the version tagged “v8.1.1”

The NodeJS GitHub repository as it was at the time the v8.1.1 tag was created

After having chosen on the “v8.1.1” tag, the page is refreshed, the most obvious change being the tag now appears as part of the URL. In addition, you will notice the file change date are different too. The source tree you are now seeing is the one that existed at the time the v8.1.1 tag was created. In some sense, you can think of a version control tool like git as a time travel machine, allowing you to go back and forth into a project history.

NodeJS GitHub repository download as a ZIP button

At this point, we can download the sources of NodeJS 8.1.1. You can’t miss the big blue button suggesting to download the ZIP archive of the project. As for me, I will download and extract the ZIP from the command line for the sake of the explanation. But if you prefer using a GUI tool, don’t hesitate to do that instead:

wget https://github.com/nodejs/node/archive/v8.1.1.zip
unzip v8.1.1.zip
cd node-8.1.1/

Downloading the ZIP archive works great. But if you want to do it “like a pro”, I would suggest using directly the git tool to download the sources. It is not complicated at all— and it will be a nice first contact with a tool you will often encounter:

# first ensure git is installed on your system
sh$ sudo apt-get install git
# Make a shallow clone the NodeJS repository at v8.1.1
sh$ git clone --depth 1 \
              --branch v8.1.1 \
              https://github.com/nodejs/node
sh$ cd node/

By the way, if you have an issue, just consider the first part of this article as a general introduction. Later I have more detailed explanations for Debian- and RedHat-based distributions in order to help you troubleshoot common issues.

Anyway, whenever you downloaded the source using git or as a ZIP archive, you should now have exactly the same source files in the current directory:

sh$ ls
android-configure  BUILDING.md            common.gypi      doc            Makefile   src
AUTHORS            CHANGELOG.md           configure        GOVERNANCE.md  node.gyp   test
benchmark          CODE_OF_CONDUCT.md     CONTRIBUTING.md  lib            node.gypi  tools
BSDmakefile        COLLABORATOR_GUIDE.md  deps             LICENSE        README.md  vcbuild.bat

Conclusion

Everything you need to know about compiling code using the GCC compiler on different operating systems.

0 Comments

No Comment.