This book serves as a gentle introduction to anyone interested in how software is developed and how to participate in open source projects. It provides readers with the basic information they need to understand how the process works, interact with others and start a project of their own.
Open source projects make great enhancements to software, but compiling them can be confusing. Here are step-by-step instructions for compiling an open source package on Ubuntu Linux.
Download source code
Obtaining source code for an application is much like getting any downloadable software. You go to a website or a code management site like GitLab, SourceForge, or GitHub. Typically, open source software is available in both a work-in-progress (“current” or “nightly”) form as well as a packaged “stable” release version. Use the stable version when possible unless you have reason to believe otherwise or are good enough with code to fix things when they break. The term stable suggests the code got tested and that the programmers of the application feel confident enough in the code to package it into a
.tar archive, give it an official number and sometimes a release name, and offer it for download to the general non-programmer public.
For this exercise, I’m using Angband, an open source (GPLv2) ASCII dungeon crawler. It’s a simple application with just enough complications to demonstrate what you need to consider when compiling software for yourself.
Download the source code from the website.
Understanding the Build System of the program
We usually talk about “compiling the sources”, but the compilation is only one of the phases required to produce a working software from its source. A build system is a set of tool and practices used to automate and articulate those different tasks in order to build entirely the software just by issuing few commands.
If the concept is simple, the reality is somewhat more complicated. Because different projects or programming language may have different requirements. Or because of the programmer’s tastes. Or the supported platforms. Or for historical reason. Or… or.. there is an almost endless list of reasons to choose or create another build system. All that to say there are many different solutions used out there.
NodeJS uses a GNU-style build system, it is a popular choice in the open source community and once again, a good way to start your journey.
Writing and tuning a build system is a pretty complex task, but for the “end user”, GNU-style build systems ease the task by using two tools:
configure file is a project-specific script that will check the destination system configuration and available feature in order to ensure the project can be built, eventually dealing with the specificities of the current platform.
An important part of a typical
configure job is to build the
Makefile. That is the file containing the instructions required to effectively build the project.
make tool, on the other hand, is a POSIX tool available on any Unix-like system. It will read the project-specific
Makefile and perform the required operations to build and install your program.
But, as always in the Linux world, you still have some leniency in customising the build to your specific needs.
configure -help command will show you all the available configuration options. Once again, this is very project-specific. And to be honest, it is sometimes necessary to dig into the project before fully understanding the meaning of each and every configure option.
But there is at least one standard GNU Autotools option that you must know: the
--prefix option. This has to do with the file system hierarchy and the place your software will be installed.
Unarchive the source code
Source code is often delivered as an archive because source code usually consists of multiple files. You have to extract it before interacting with it, whether it’s a tarball, a zip file, a 7z file, or something else entirely.
$ tar --extract --file Angband-x.y.z.tar.gz
Once you’ve unarchived it, change the directory into the extracted directory and have a look around. There’s usually a
README file at the top level of the directory. This file, ideally, contains guidance on what you need to do to compile the code. The
README often contains information on these important aspects of the code:
- Language: What language the code is in (for instance, C, C++, Rust, Python).
- Dependencies: What other software you need to have installed on your system for this application to build and run.
- Instructions: The literal steps you need to take to build the software. Occasionally, they include this information within a dedicated file intuitively entitled
README file doesn’t contain that information, consider filing a bug report with the developer. You’re not the only one who needs an introduction to source code. Regardless of how experienced they are, everyone is new to source code they’ve never seen before, and documentation is important!
Angband’s maintainers link to online instructions to describe how to compile the code. This document also describes what other software you need to have installed, although it doesn’t exactly spell it out. The site says, “There are several different front ends that you can optionally build (GCU, SDL, SDL2, and X11) using arguments to configure such as
--disable-x11, etc.” This may mean something to you or look like a foreign language, but this is the kind of stuff you get used to after compiling code frequently. Whether or not you understand what X11 or SDL2 is, they’re both requirements that you see pretty often after regularly compiling code over a few months. You get comfortable with the idea that most software needs other software libraries because they build upon other technologies. In this case, though, Angband is very flexible and compiles with or without these optional dependencies, so for now, you can pretend that there are no additional dependencies.
Getting the source code from GitHub
Like many open-source projects, the sources of NodeJS can be found on GitHub: https://github.com/nodejs/node
So, let’s go directly there.
If you’re not familiar with GitHub, git or any other version control system worth mentioning the repository contains the current source for the software, as well as a history of all the modifications made through the years to that software. Eventually up to the very first line written for that project. For the developers, keeping that history has many advantages. For us today, the main one is we will be able to get the sources from for the project as they were at any given point in time. More precisely, I will be able to get the sources as they were when the 8.1.1 version I want was released. Even if there were many modifications since then.
On GitHub, you can use the “branch” button to navigate between different versions of the software. “Branch” and “tags” are somewhat related concepts in Git. Basically, the developers create “branch” and “tags” to keep track of important events in the project history, like when they start working on a new feature or when they publish a release. I will not go into the details here, all you need to know is I’m looking for the version tagged “v8.1.1”
After having chosen on the “v8.1.1” tag, the page is refreshed, the most obvious change being the tag now appears as part of the URL. In addition, you will notice the file change date are different too. The source tree you are now seeing is the one that existed at the time the v8.1.1 tag was created. In some sense, you can think of a version control tool like git as a time travel machine, allowing you to go back and forth into a project history.
At this point, we can download the sources of NodeJS 8.1.1. You can’t miss the big blue button suggesting to download the ZIP archive of the project. As for me, I will download and extract the ZIP from the command line for the sake of the explanation. But if you prefer using a GUI tool, don’t hesitate to do that instead:
wget https://github.com/nodejs/node/archive/v8.1.1.zip unzip v8.1.1.zip cd node-8.1.1/
Downloading the ZIP archive works great. But if you want to do it “like a pro”, I would suggest using directly the
git tool to download the sources. It is not complicated at all— and it will be a nice first contact with a tool you will often encounter:
# first ensure git is installed on your system sh$ sudo apt-get install git # Make a shallow clone the NodeJS repository at v8.1.1 sh$ git clone --depth 1 \ --branch v8.1.1 \ https://github.com/nodejs/node sh$ cd node/
By the way, if you have an issue, just consider the first part of this article as a general introduction. Later I have more detailed explanations for Debian- and RedHat-based distributions in order to help you troubleshoot common issues.
Anyway, whenever you downloaded the source using
git or as a ZIP archive, you should now have exactly the same source files in the current directory:
sh$ ls android-configure BUILDING.md common.gypi doc Makefile src AUTHORS CHANGELOG.md configure GOVERNANCE.md node.gyp test benchmark CODE_OF_CONDUCT.md CONTRIBUTING.md lib node.gypi tools BSDmakefile COLLABORATOR_GUIDE.md deps LICENSE README.md vcbuild.bat
Everything you need to know about compiling code using the GCC compiler on different operating systems.