I built a multiplayer voxel browser game engine

TLDR;

This post is a retrospective on a multiplayer voxel browser game engine I built. This post will cover the squishy storytelling side of the project, and the accompanying code repository has all of the intricate technical details. This post is essentially a 2 part post. Read this and read the repo README.

Context

This project is a bit old and stale at the time of this post. Some of the tech decisions made sense during development but are questionable today. I will try to address that in this post. I started this project with very minimal game development experience. I had no idea how many things worked, shaders, 3d asset files, raw binary buffer manipulation, scene graphs, and physics engines. I learned all these skills and more during this project. This experience also helped me later land some major corporate projects. This project was primarily active from 2017-2019; however, exploration for the project goes back to late 2016.

Inspiration & motivation

I wanted to build a multiplayer browser game. I was very inspired by .io games and sites like iogames.space. I love the .io gaming experience of copy-pasting a URL around and getting in a game with your friends. I wanted to recreate a canceled MMO called Exteel in an .io style. My background was in web development and design. I stuck with the tools and environment I was most familiar with.

Target Features

• network multiplayer
• playable cross-platform in the browser
• Player avatar customization
• lazy loading assets
• shared client & server game code
• third person camera
• platforming physics
• destructible environments

I completed most of these features except for destructible environments. Destructible environments were a big driver for the voxel asset pipeline, assuming voxels would enable the destructibility. Another browser game VoxLords inspired the goal of destructible environments. VoxLords had good environmental destruction and captured my imagination. I started this out by forking and modifying VoxLords to be multiplayer.

https://twitter.com/lallassu/status/725401257377169409?s=20

After hacking on VoxLords, it was clear that making a captivating 3D multiplayer browser experience was possible. Unfortunately, Lallassu was correct, and there were some major performance issues in the VoxLords engine that would require extensive refactoring. I decided to start a new project.

Why I didn't use one of the megacorp engines

I was not happy with Unity and Unreal's HTML5 browser experience. Unity's HTML5 export creates a giant asset bundle where users must sit and watch a loading bar before playing. This loading bar experience was against my goals of lazy loading assets on demand. Additionally, Unity's multiplayer server-side support was sketchy and still is. Unreal had much stronger multiplayer support but similar issues with browser builds, and in addition, Unreal itself was too large to install on my laptop.

The work

After hacking the VoxLords multiplayer fork, I started by exploring multiple tech stacks concurrently and building a prototype in each. The stacks I looked at were Babylon.js, Three.js, & Stackgl.

Tech Stack Requirements

The first requirement was support for the 3d asset pipeline. I had been working with voxel models in MagicaVoxel .vox format. The .vox format was great for static entities. There are primarily two ways to animate a voxel model. You can create frame-based animation, in which each frame is a separate voxel animation, and then you morph or swap them out at runtime. Or you can apply skeletal transformation data to voxel meshes. If you decide on the skeletal route, you must break the voxel mesh into smaller meshes corresponding to the skeleton bones.

I had a character mesh that I wanted to use in .vox format, but at the time MagicaVoxel did not support multiple meshes. I converted the .vox file to quiblce .qb file and broke the character up into separate meshes.

I then imported the segmented Qubicle mesh into Blender and built the skeletal animations in Blender.

The engine then combines the animation data and voxel mesh data at runtime.

Tech Stack Evaluation

• three.js

  I looked at Three.js first because VoxLords was built with it. The Three.js - Blender pipeline had issues, and we could not utilize skeletal animation imports. A colleague of mine who was working on a Three.js project captured some of the three.js blender exporter issues at the time. The Three.js - Blender compatibility issues were a deal breaker for me.

  Furthermore, Three.js did not have a viable headless server-side offering. Three.js uses custom math libraries that were not easily extractable from the rendering code. This tight coupling of math primitives and client render code meant multiplayer support would take a lot of extra duplicate work, re-implementing the game logic on the server. Three.js has advanced significantly since then, and I don't think these are issues anymore.

• babylon.js

  I made some immediate promising progress in Babylon.js. The asset pipeline was robust and supported skeletal animation data. Babylon has a camera and physics out of the box. The Babylon team and community were super responsive and supportive. I hit a wall with Babylon.js when building the multiplayer server. Like Three.js, Babylon.js had no headless mode. Again, building multiplayer would require duplicate work reproducing all the simulation logic and physics in the server-side codebase. Recent versions of Babylon have a robust headless mode and have decoupled the framework into smaller libraries. These days Three.js is the king of the normie mindshare lexicon, but I still find Babylon.js a much better offering.

• stackgl

  Stackgl is the dark horse of this list. Stackgl is not trying to be a complete 3D engine like Three.js and Babylon.js. Stackgl is an ecosystem of small functional tools that can be composed to create 3d browser experiences. Stackgl follows the Unix philosophy of "small sharp tools." Stackgl has some main pillars: Rendering, Math, and Data. Stackgl handles rendering by focusing on direct shader code. Babylon.js and Three.js try to abstract shaders heavily away. Stackgl's primary rendering module is Regl, which provides an entirely novel stateless rendering pattern. Stackgl then provides many small functional math modules for vertex and geometry data. I was quickly more productive in Stackgl than in the previous two stacks.

  The modular nature made it very easy to share code between the client and server, easy to optimize performance and rendering, and easy to add custom functionality. For example, Babylon.js and Three.js have their own vector math implementation libraries. Then if you use a physics engine, the physics engines have their internal vector math libraries. Furthermore, if you use a multiplayer server, you will also need a vector math library there.

In Babylon and Three, it becomes a lot of fake work converting and serializing between different vector implementations. In Stackgl, you can easily use the same gl-matrix vector math library across a project which is a huge productivity gain.

When I started progressing with Stackgl, I took a year off and worked on it full-time throughout 2017.

What went wrong

Unfortunately, I consider this project unfinished. I did not ship a complete game experience, and there are some issues with the engine that would be risky for production deployment. I'm writing this post to put closure on the project before moving on to new work and in the hopes that this experience may be helpful to any others interested in this space.

React vs. Vanilla Js

Most of my web development experience has been with the React framework, and I thought it would complement Regl and the Stackgl ecosystem's functional paradigm. I created a react reconciler to bridge React and Regl. I started using a React stack with Redux to manage all the game state and state transitions. This React+Redux combo turned out to be a huge boon when writing UI. Game developers in traditional engines constantly complain about UI being their least favorite task. The game dev UI complaint is because game engines use geriatric 2-way mutable state management UI code that is error-prone and brittle. Web developers have solved these UI complaints and built tools like React that make creating UIs easy.

React+Redux setup was too much performance overhead for the real-time gameplay section. The state updates through the Redux action reducer pipeline, and the minimal React render updates were enough to cause noticeable hiccups in the gameplay frame rate. Performance in the browser environment is susceptible to garbage collector management. To minimize garbage collector hits, you need to use object pooling. Object pooling is a mutable state management pattern in which you pre-allocate a pool of objects. The collection of allocated objects gets mutated and reused during the program's life to minimize runtime memory allocations. This object pooling pattern conflicts with the immutable update patterns of React and Redux. Hitting these performance issues was a significant roadblock and essentially became a 'rewrite' in which I had to rewrite the game state management to be performance optimized. This rewrite was costly and took a lot of time.

Hobby time

When I started progressing with Stackgl, I took a year off and worked on it full-time throughout 2017. I lived off some savings and unemployment checks until late 2018. I got back into corporate roles from 2018-2020. I was invited to Animecon 2019 indie developer section to shill my game in the Indie game section.

During this time, I switched to hobby mode and started putting less and less time into development. I was maybe averaging 4 hours a week. I was distracted by day job work. This time shortage slowed development progress. Development picked up during the 2020 Covid lockdowns. Unfortunately, returning to development after a hiatus presented some new challenges.

Software entropy

The JavaScript ecosystem is known for moving incredibly fast. I spent much time-fighting severe software entropy when I returned to development. Getting the project to build with the latest dependency chains of Node+NPM+webpack etc., took a lot of work. The animation pipeline using Landon had severe entropy. Landon is a mix of Rust, JavaScript, and Blender versions that are not strictly pinned. The Landon developer was changing the data output format and abandoned the skeletal animation system I used. I had to do work to upgrade to the latest Blender export versions but then write backward compatibility translation layers for the animation data. I upgraded my laptop to an M1 Mac. The M1 transition triggered a roadblock on the project because the WebRTC module I used would not compile on the new M1 Apple silicon chip. The WebRTC module now supports M1 architectures but was pretty much the show-stopper.

Byproducts

The work put into this project spun off some valuable libraries I have open-sourced and put on GitHub.

• react-regl

  A react reconciler to bridge React and Regl

• parse-magica-voxel

  A JavaScript parser for the Magicavoxel .vox binary file format

• parse-quibicle

  A JavaScript parser for the .qb binary file format.

• gl-swept-sphere-triangle

  A swept sphere to polygon collision detection algorithm

Shout outs

I built this project with the help and research of other highly skilled software engineers. Their contributions were critical to the development of this project.

• Mikola lysenko

  Mikola is a primary driver of the Stackgl ecosystem and a certified 10x genius developer. His blog 0fps.net has some mind-melting content on high-performance computer science and geometry. Mikola assembled a team and built a Roblox clone for China in the browser. The team got together for a podcast to discuss their Reflections on building web-based voxel MMO in China (Mikola Lysenko, et al) - YouTube It is a highly insightful talk, highly recommend if you found this post interesting.

• Chinedu Francis Nwafili

  Chinedu did terrific work by building the stackgl skeletal animation system, which is a vital piece of the Stackgl ecosystem, and this project was highly dependent on and probably couldn't have come together without. While I was building this project, Chinedu also created his own game and engine. We kept in correspondence while making and sharing ideas. Chinedu also built out a very cool suite of tools for Blender called landon. Landon makes Blender a source of truth for game engine data. Chinedu's blog at https://chinedufn.com/ has in-depth development journal posts on his game engine development.

• James Warlloyd

  James created the original mech model that I used as the bases for the character art in this engine.

The state of the art

Overall this project was a significant educational return on investment for me. I learned about hardcore browser optimization and many other data-processing techniques. I have continued to do game ( and engine ) development as a hobby and have already looked into some other emerging technologies. My focus is still on browser multiplayer experiences.

Despite their popularity, I am dissatisfied with the multiplayer offerings provided by the major engines, Unreal, Unity, and Godot, as they fail to deliver user-friendly solutions catered to the needs of indie developers. I have previously written about multiplayer game engine design for indies. The big engines use coding patterns that couple game simulation code to rendering code. This coupling makes it challenging to build a server as you have to decouple the simulation and run it headless.

• DotBigBang

  DotBigBang is everything this project attempted to be and more. It is a multiplayer voxel 3d engine, all fully in the browser. DotBigBang is focused on user-generated content and is essentially a multiplayer game engine. You can get a group of people together, bring your avatars into a 3D scene, and then collaboratively edit and program that scene like a multiplayer Unity editor experience. I had the fortune to hang with the founder of DotBigBang at GDC 2023 and hear about some impressive browser performance optimization they were doing. He told me about forking Chromium to have better performance debugging visibility into how the JIT compilation and memory usage of the browser was working.

• Manifold engine

  The Manifold engine is a stealthy engine currently in development by the matrix.org team. Manifold is embedded in a repository for another project with a disclaimer they might release in the future. The host project is 'Thirdroom,' a multiplayer VR metaverse platform.

  When I learned about Manifold I was very excited because it has the same design philosophy and high-performance computation architecture that I had been building towards. It uses Web Workers to create multiple threads to process rendering, physics, and networking separately. All the threads share memory over a SharedArrayBuffer, and they use object pooling to reduce garbage collection. Manifold also has WebRTC Data channels for networking. I built a promising prototype in Manifold; unfortunately, Manifold is tightly coupled to a matrix.org backend. The matrix backend has a clunky API to use, which would make custom server-side optimization challenging. If the matrix.org team ever decouples and delivers Manifold, it would be a very compelling option.

• Ambient engine

  Ambient is a promising new Rust lang engine. It is a default multiplayer engine. It shares code between the client and server. It compiles game code to WASM and then acts as a cross-platform binary runtime to execute that WASM. I have built a promising prototype in Ambient but unfortunately Ambient does not currently have a browser build. The Ambient team is actively working to target browsers and is aiming for an upcoming release.

Thanks for reading. checkout the multiplayer voxel browser game engine repository for more details.