Native Client

From Unvanquished
(Redirected from NaCl)
Jump to: navigation, search

Native Client or NaCl is the current technology used for securely virtualizing the game code in the Dæmon engine.

It replaced the Q3VM (Quake3 Virtual Machine) in the engine.

Unlike Q3VM for which the .qvm binaries were limited to be produced with the old C supported by the proprietary LCC compiler, the .nexe NaCl binaries can be compiled with Clang-based open source compilers supporting modern-enough C++ versions and allowing to reuse off-the-shelf C++ libraries that may even be shared between the engine and the game source code.

Unlike QVMs, NaCl gamelogic runs in a secure sandbox. Native Client lets you run mods downloaded from the internet without giving them free reign on your computer. QVM was equivalent to running a DLL with no protection.

The Native Client documentation can be found on chrome.jscn.org/docs/native-client/welcome-to-native-client/.

Platforms

Platform NaCl compatibility
Linux Windows macOS FreeBSD
amd64 i686 arm64 armhf amd64 i686 amd64 arm64 amd64
✅️ native ✅️ native ☑️ through armhf
compatibility
✅️ native ✅️ native ✅️ native ✅️ native ☑️ through amd64
compatibility
☑️ through Linux
compatibility

The NaCl loader (the virtual machine running the virtualised code) is available for three systems: Linux, Windows and macOS. The supported architectures differs per system:

  • Linux: amd64, i686, armhf;
  • Windows: amd64, i686;
  • macOS: amd64.

It is possible to run native engines with other platforms when there is a compatibility layer to run the NaCl loader:

  • Linux, arm64: running the armhf NaCl loader through built-in Linux 32-bit compatibility;
  • macOS, arm64: running the amd64 NaCl loader through Rosetta2 compatibility;
  • FreeBSD, amd64: running the Linux NaCl loader through built-in Linuxulator compatibility.

The Linux armhf NaCl loader requires a 4k PageSize kernel:

The Linux i686 and armhf NaCl loaders may run on FreeBSD using Linuxulator but it is not tested.

NaCl also supports 32-bit MIPS on Linux but this architecture is not supported by the Dæmon engine and the Unvanquished game project.

NaCl upstream development has slowed down since the competing solution WebAssembly became the industry standard, so it is very unlikely support for new systems and architecture would be added. To overcome this problem, we would have to switch to WebAssembly (that unfortunately also lacks support for some platforms NaCl supports, like i686 or armhf, but more importantly lacks support for exceptions or setjmp/longjmp and then isn't complete enough for our needs yet.

Compilers

PNaCl Clang

PNaCl Clang is the current NaCl compiler we use to build the nexe game binaries. It is based on an old Clang 3.6 and does not support C++ above C++14. To overcome some of the related limitations we may switch to Saigo.

The PNaCl compilation workflow is to build a single .pexe then to translate them to per-architecture .nexe (amd64, i686, armhf) using a specific translator tool.

The PNaCl SDK consists on Python-based wrappers around Clang internals. It was initially using now-obsolete and hard-to-find Python2 but we ported it to Python3 to extend its lifespan.

The provided PNaCl SDK runs on Linux, Windows and macOS systems, on amd64 architectures.

While the .pexe compiler itself runs on FreeBSD with the Linuxulator, the translator does not, meaning it's not possible to build .nexe binaries on FreeBSD.

PNaCl supports C++ exception. There exist newer PNaCl SDKs (from chromium canary releases) that do not support exceptions and may not provide more what we already have with latest stable PNaCl.

Saigo

Saigo is a new NaCl compiler based on latest Clang and supporting latest C++ standards. It compiles directly to .nexe.

The Unvanquished game code is now buildable with Saigo.

Saigo may not support C++ exceptions as far as we know but supports setjmp/longjmp.

Unlike PNaCl, Saigo do not provide released binaries, meaning a switch to Saigo requires a convenient way to provide it to contributors. Google provides some nightly Linux amd64 Saigo toolchain snapshots that aren't easy to get without some Google scripts. The Saigo compiler itself isn't hard to build (same as Clang itself), the binutils are a bit more ticky but not that hard (10 years-old configure build). What is tricky to rebuild is the libc that requires some very old NaCl GCC. Fortunately there exists libc and libc++ snapshots provided by Google that can be reused with a self-built toolchain.

The Saigo compiler and NaCl binutils should build everywhere their upstream build, but they will always target the usual platforms as the NaCl loader is still the same: a more recent NaCl loader is also buildable but doesn't bring any new platform support.

Moving to Saigo is considered as a migration step that can be achievable before migrating to WebAssembly, as it allows us to migrate to a new C++ standard and update libraries we use without waiting for the migration to WebAssembly.

Some redesign of our CMake configuration for Saigo may help later when migrating to WebAssembly as it makes it more generic for different VM toolchains.

Here is a GitHub issue about the ongoing efforts for migrating to Saigo:

The workspace for our attempt to rebuild and repackage Saigo is there:

Interactive Debugging

Most of the time it is easiest to debug with a native DLL. But if you have a problem that cannot be reproduced outside of NaCl, you may be forced to resort to the following tutorial.

These steps for attaching the NaCl debugger[1] have been tested on Windows with a PNaCl toolchain. They should also work on binaries built with Saigo, using the old PNaCl GDB. On Linux or Mac you may experience more difficulties due to the lack of support for old binaries (meaning nacl-gdb). illwieckz reports that on amd64 Linux in 2025, using the old GDB which is a 32-bit binary, debugging an amd64 nexe does not work (despite the documentation's claims of debugger platform independence), but debugging an i686 nexe does.

  1. Build the NaCl gamelogic with debugging symbols (build type Debug or RelWithDebInfo).
  2. Launch Daemon configured for NaCl debugging. For example, ./daemonded -set vm.sgame.type 1 -set vm.sgame.debug 1 +map chasm When the relevant gamelogic starts, the engine should pause and emit the message Waiting for GDB connection on localhost:4014.
  3. Start NaCl GDB, e.g. daemon\external_deps\windows-amd64-mingw_10\nacl-gdb.exe This works better out of a command prompt rather than an MSYS window.
  4. Use the "file" command to tell the debugger where the binary with symbols is. For example, file sgame-amd64.nexe
  5. OPTIONAL, probably a waste of time: load the IRT (Integrated Runtime) symbols: nacl-irt irt_core-amd64.nexe
  6. Connect to the NaCl program with the command target remote :4014
  7. You should see a message like 0x000000000ffc00a0 in ?? () (or 0x000000000ffc00a0 in __pnacl_start () if you did step 5). This means the program is paused at the beginning.
  8. Set some breakpoints, e.g. b G_RunFrame. Alternatively, just wait for the debugger to break when the game crashes.
  9. Start the program by continuing (c).

Postmortem Debugging

See Breakpad.
  1. https://www.chromium.org/nativeclient/how-tos/debugging-documentation/debugging-with-debug-stub-recommended/getting-started-with-debug-stub/