Aug 25, 2020 Tutorials

Servers Deep Dive Part 3: Using WebSockets

Master advanced tools for creating rich multiplayer experiences in Decentraland.

In the previous post of this series we covered how to launch your own server to handle HTTP requests to sync players and store a persistent state for the scene. This works well enough for scenes where the changes that you keep track of don’t happen continuously and where updates don’t need to happen immediately for everyone. But what if you want to keep track of something that is constantly moving and make sure that all players in the scene are seeing it in the same position?

Using WebSockets is the most robust solution if you want players to be in perfect sync with each other and it’s what most AAA multiplayer games use. In a WebSockets connection, data ‘frames’ can be sent at any time by the player or the server whenever there’s something to update.

A WebSockets connection is first established with a handshake between the player and the server which opens up a two-way stream of data to send and receive updates at any time, resulting in lower latency and better use of resources.

The Moonshot football stadium, where the Decentraland World Cup was held, is a perfect example of what’s possible thanks to WebSockets. Using WebSockets was essential to keep the ball’s position updated continuously and consistently for all players and spectators. Player positions were also synced via WebSockets to have less latency, their WebSocket-synced position was represented by an astronaut avatar to make them visible on all realms.

Create a server

Firebase, which we used in the previous blogpost of this series, is unfortunately not a viable option for what we’re trying to build in this case. The service Firebase provides is ‘serverless’, meaning that the server we deploy only really gets instanced for a brief instant each time a request arrives and is then dissolved when complete. In this case we need to maintain a persistent connection open for each player that’s around our scene.

This means we’re going to have to roll up our sleeves and establish a more traditional server from scratch. There are many services out there that rent out virtual machines and for this tutorial we’ll use DigitalOcean, but any other similar service would work just as well. If you feel comfortable enough with this part of the process, feel free to skip over this section.

To create your server in DigitalOcean, follow these steps:

Visit DigitalOcean and create an account.

TIP: If you don’t have an account already, we recommend that you register via this referral link so that you start out with $ 100 USD of credit in your favor over 60 days.
Create a new project on the left-hand side menu. Give your project a name and description. When prompted to Move Resources, select Skip for now.
Click Get Started with a Droplet to create a server.

Configure the server:

i) Select **Debian v10** as the OS version. At the time of writing this, the exact version I’m selecting is **10.3 x 64**
ii) Pick a pricing option. For this example the cheapest option should be fine, which costs 5 USD a month

> TIP: You may need to scroll the menu left to find the cheaper options.

iii) Choose a region for your hosting datacenter. Choose whichever is physically closest to you or your target audience
iv) No need to set a VPC or any of the additional options
v) For authentication a password is the simplest option and will work, but we highly recommend that you instead use an **SSH key**. This will enable you to connect to your server directly through your command line application, instead of through a mock console in a browser window, which has several limitations. To set up an SSH key, click **New SSH key** and follow the steps detailed in that window

When you’re done, click Create Droplet to have your server instanced
Give the server a minute to get instanced. When that’s done, congratulations, you’re the proud owner of a server!

To access the console of your server:

i) If you configured your server to use authentication through SSH, open a command line on your machine and paste the following command, using your server’s IP address.

ssh [email protected]<ip>

You can easily find your server’s IP displayed in DigitalOcean in the server’s settings, once instanced.

With the IP from the screenshot above, the command to open that server would be the following:

ssh [email protected]

You will then be asked to write the passphrase that you set up when creating the SSH key. You can read more about this process and troubleshooting in DigitalOcean’s own documentation: https://www.digitalocean.com/docs/droplets/how-to/connect-with-ssh/openssh/

ii) If you configured the server to be accessed via password, click the three dots on the far right of the server’s data and select Access Console to open a mock-console in a browser window.

Test your server:

Install nginx in the server’s console by running the following commands:

apt -y update
apt install -y nginx

Open a browser window and type your server’s IP as an http URL. For example http://64.225.45.232/
If everything works correctly you should see a page saying “Welcome to nginx!”

Obtain TLS Certificates

To use WebSockets securely, you need to use WSS (WebSockets secure), which is the equivalent to what HTTPS is to HTTP. Like with HTTPS, WSS requires that you obtain TLS certificates that are specific to your domain to use as encrypting keys. To handle this, we’re going to use Certbot, a popular service that automates a lot of the steps needed to obtain these certificates and makes it quite simple.

In your server’s console, run the following command to install Certbot:

apt install -y certbot python-certbot-nginx

Once installed, run the following command to create your certificates:

certbot --nginx

Certbot will be guide you through several steps and prompt you to answer several questions:

Provide your email.
Agree to the terms of service, you don’t need to agree to share.your email with EFF.
Certbot will ask you for a domain name. If you’re just following the steps in this tutorial, then you likely won’t have one, but there’s an easy workaround. We can rely on nip.io, a service that maps all possible IP addresses to their domains and routes to these IPs accordingly. You don’t need to subscribe or do anything on their platform, they should already have a domain matching your IP address. If your server’s IP is 64.225.45.232, then the following nip domain should route to your server: 64-225-45-232.nip.io. You can provide this domain to Certbot in this step.
When prompted about redirecting HTTPS traffic, select the option 2 to redirect all traffic to HTTPS.

Alternative 1: Download the server as a Docker image

Instead of writing your own code for the server, you can use a generic broadcast server we prepared. This pre-made WebSockets server broadcasts all the messages it receives to all other players that subscribe to the same room. You can subscribe a player to a specific room by appending a room name to the end of the URL when calling the server. This can be used, for example, to only share messages between players that can also see each other in the same realm, or you can use any other criteria you want to group players into rooms.

In many scenarios, this simple example server might be all you need. If you plan on building something custom, then you can skip over to the next section, where we show how you can build your own server code.

For reference, you can find the code used by the server in this github repo: github.com/decentraland-scenes/ws-broadcast

We will be installing the server as a Docker container. Docker containers are self-contained black boxes that have all the configuration and dependency installations ready to run something out of the box. They are the easiest way to set up something like this. For more complete instructions about how to install Docker, see Docker’s own documentation: https://docs.docker.com/engine/install/debian/

To install and run the generic WebSockets server app from Docker:

Install the following dependencies:

apt install apt-transport-https ca-certificates curl gnupg-agent software-properties-common

Install Curl and gnupg2

sudo apt install curl apt-get update && apt-get install -y gnupg2

Add docker’s official GPG key:

curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -

Configure apt to use the latest stable version of docker, then update your apt repositories:

add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable" apt update

Install the latest version of Docker Engine:

apt install -y docker-ce docker-ce-cli containerd.io

Once Docker Engine is installed, download and start running the container for our broadcast WebSockets server:

docker run -d -p 8080:8080 decentraland-scenes/ws-broadcast

>TIP: If you later need to stop running this server app, you can stop all active docker containers with the command `docker stop $(docker ps -aq)`.

The broadcast service is now running on our server, but we need to expose its endpoints through a path, which we can do using nginx. Open nginx’s configuration file on a console-based editor:
```
nano /etc/nginx/sites-enabled/default
```

Paste the following few lines into this document. These should go inside the second bracketed section that defines a server object. The first of these sections refers to port 80, which we don’t use here. We need the following to go in the second server object definition, which refers to the secure port 443.

location /broadcast {
    proxy_pass http://127.0.0.1:8080;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_set_header Host $host;
    add_header 'Access-Control-Allow-Origin' '*';
    add_header 'Access-Control-Allow-Methods' 'GET, OPTIONS';
    #
    #  Extra headers needed for certain browsers
    #
    add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,if-range';

    if ($request_method = 'OPTIONS') {
    #
            # Tell client that this pre-flight info is valid for 20 days
            #
    	add_header 'Access-Control-Max-Age' 1728000;
    	add_header 'Content-Type' 'text/plain; charset=utf-8';
    	add_header 'Content-Length' 0;
    	return 204;
    }
    if ($request_method = 'GET') {
            add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
    }
}

When you’re done, press Ctrl + X to exit the editor, saving your changes when asked.

Restart nginx so that it starts using the new configuration you just set.
```
systemctl reload nginx
```
Voila! You are now running a secure WebSocket server.

You can now connect a scene to the server via wss://yourIPwithDashs.nip.io/broadcast, in our case, that would be wss://64-225-45-232.nip.io/broadcast.

If we type this into a browser window, we won’t see any response, but you can use the following page to test it out: www.websocket.org. Just write the server’s wss address into the location field, click Connect, and then send any message, to see it responded back.

Alternative 2: Write custom code for the server

The previous section shows the quickest way to set up a WebSockets server from an existing Docker container. The downside of this approach is that the Docker container is a black box you can’t tamper with and you can’t change the code used by the server. If you instead want to be free to make changes to the server’s code to better suit your needs, this section describes an alternative path.

To build a well-rounded game you’ll likely need to add some custom logic to the server. For example, you might need to validate that a player isn’t cheating, or to handle what happens when players send conflicting information. You might also want to have the game loop and all of the logic run server-side rather than in the scene client-side.

Note: In this section we assume that you already have your server created, with nginx installed and have obtained TLS certificates for it, as described in the past sections.

Set up the following permissions and dependencies in the server:

Create a new user to run apps more securely:

adduser nodejs

Please use a strong password as it may be used to login to the server. Feel free to skip providing your full name and other personal information.

	Note: When using the server downloaded from Docker, we worked directly with the default `root` user because running things inside a docker container is less risky. But running the app directly while using the root user exposes vulnerabilities that malicious users could exploit, so it’s always best to use a non-root user that has less permissions to change things.

Switch to the new user you created:
```
su - nodejs
```
Your command line will now start with [email protected] rather than [email protected].

Install Curl and gnupg2

sudo apt install curl apt-get update && apt-get install -y gnupg2

Install NVM (Node Version Manager), this tool allows easy installation of different nodejs versions:
```
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
```

Once the installation is complete, either restart your terminal (don’t forget to switch the user back to nodejs) or run the following commands:

export NVM_DIR="$HOME/.nvm" [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm [ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion

Install the latest version of nodejs (v14):
```
nvm install 14
```
Verify that the installation worked as expected:
```
node -v
```
Install pm2, this is a tool to manage nodejs process that can restart our application automatically in case of crash:
```
```bash
npm install -g pm2
```
```
Install the pm2 Typescript transpiler, so we can run our typescript application directly with pm2
```
pm2 install typescript
```

We now have everything ready to run our server. As an example, we will download and run the same broadcast WebSockets server that the previous section uses through the Docker container, which can be found in the following repo: github.com/decentraland-scenes/ws-broadcast. You could fork this repository to make your own changes and then download that forked repo instead. Or you could build your own repo from scratch if you prefer.

Download the example WebSocket project from GitHub:
```
git clone https://github.com/decentraland-scenes/ws-broadcast.git
```
You’ll be asked to log in to your GitHub account while doing this.
Go into the folder and install the project’s dependencies:
```
cd ws-broadcast npm install
```
Start running the app using pm2, for better stability:
```
pm2 start src/index.ts --name ws-broadcast
```
The broadcast service is now running on our server, but we need to expose its endpoints through a path, which we can do using nginx. Open nginx’s configuration file on a console-based editor:
```
nano /etc/nginx/sites-enabled/default
```

location /broadcast {
    proxy_pass http://127.0.0.1:8080;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_set_header Host $host;
    add_header 'Access-Control-Allow-Origin' '*';
    add_header 'Access-Control-Allow-Methods' 'GET, OPTIONS';
    #
    #  Extra headers needed for certain browsers
    #
    add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,if-range';

    if ($request_method = 'OPTIONS') {
    #
            # Tell client that this pre-flight info is valid for 20 days
            #
    	add_header 'Access-Control-Max-Age' 1728000;
    	add_header 'Content-Type' 'text/plain; charset=utf-8';
    	add_header 'Content-Length' 0;
    	return 204;
    }
    if ($request_method = 'GET') {
            add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
    }
}

When you’re done, press Ctrl + X to save and exit the editor.

Restart nginx to use the new configuration you just set.
```
systemctl reload nginx
```
And voila! You are now running a secure WebSocket server.

While you are debugging your custom server code, you probably won’t want to go over the hassle of deploying a new version of the server every single time. You can run a copy of your server locally while you debug the scene. To do so, simply run npm start on the server folder. Then, while that process is active, you can direct your Decentraland scene to the local copy of the server with ws://localhost:8080 instead of the full URL.

Connecting a scene to the server

Let’s build a super simple scene to try out our server. You don’t need to import any special libraries to use WebSockets as Typescript provides the essential functionality out of the box. You can try this out on a clean Decentraland project after running dcl init. This example should be valid for whichever of the two methods you choose to follow: the Docker container or the custom code.

The bare minimum to initiate a connection to our WebSockets server is this line:

const socket = new WebSocket('wss://64-225-45-232.nip.io/broadcast`)

Or if you’re running a local copy of the server on your machine

const socket = new WebSocket(`ws://localhost:8080`)

However, if we want to keep players in different realms separate, we need to first get the player’s realm, and then then use the realm name as part of the url we send to the server. Since the fetching of the realm requires an await, we need to do that inside an async function:

import { getCurrentRealm } from '@decentraland/EnvironmentAPI'

let socket

joinSocketsServer()

export async function joinSocketsServer() {
  let realm = await getCurrentRealm()
  socket = new WebSocket(
    'wss://64-225-45-232.nip.io/wsecho/' + realm.displayName
  )
}

Once connected, we want to start listening for ws messages from the server, which we can do with a function like this:

socket.onmessage = function (event) {
  try {
    const parsed = JSON.parse(event.data)
    log(parsed)
    // DO SOMETHING WITH INPUT
  } catch (error) {
    log(error)
  }
}

Finally, we also want to send out messages to the server on certain events, which we can do with the following:

socket.send(
  JSON.stringify({
    //MESSAGE DATA
  })
)

And that’s all it really takes!

This very simple scene makes use of the snippets shared above to keep players in sync. Clicking on any of the cubes in this scene turns it green for all players.

github.com/decentraland-scenes/ws-example

Combining WS with physics

This scene where you can kick balls around uses the cannon.js physics library to calculate the ball’s movement. Whenever a player kicks a ball, the vector for that kick is shared via WebSockets with other players. The last player who has kicked the ball then starts sharing the position of all balls to other players every five seconds to make sure everyone is in sync.

Since this scene uses the broadcast WebSockets server that we created in this tutorial, it’s important that one single player is sharing synced positions at a time, as the server has no way to deal with conflicting information. In a more advanced example, you could have all players send their ball positions and the server would have some built-in logic to arrive at a single consensus.

TIP: To use the cannon.js library in this scene, it was necessary to install the Typescript types for this library via npm i "@types/cannon. It was also necessary to list node_modules/cannon/build/cannon.min.js as a scene file in the tsconfig.json file of the scene.

Here we’re calculating all of the physics on the client side, but there are also different approaches for handling physics in a multiplayer scenario. We could instead calculate all of the physics on the server-side, that would ensure perfect consistency and eliminate any risk of cheating, but would also lead to plenty of lag. Probably the best approach is to calculate the physics on both the client and server side, so that players see immediate response, but periodically adjust what the player sees to what the server has calculated to ensure consistency.

github.com/decentraland-scenes/websocket-bouncing-balls

Server-side logic

This scene includes a little more logic on the server side, organizing players into teams and only starting a match when there are players on both sides. Each scene runs its own calculations of the current remaining time and the scores of each team, but so does the server. In the end, the server has the final word about when the match is over and what is the final result.

github.com/decentraland-scenes/Land-Flipper-Game

Other examples

The following example server is capable of integrating a scene to Decentraland’s Discord chat using WebSockets, allowing you to read discord messages in-world, or to even write messages into the Discord server.

github.com/decentraland-scenes/discordWebsocket

This other example server connects a Decentraland scene to smart-lights out in the real world. It syncs to the server that handles these lights using WebSockets too: github.com/HPrivakos/yeelightDecentraland

A note about security

WebSockets provides more robust tools for ensuring that players don’t cheat, but you still need to do all the anti-cheating checks you’d do with a REST server. One advantage is that you only need to verify the player’s identity on the handshake stage, you don’t have to verify each request that the player sends. Another advantage is that once a session is open, you can be sure that all data coming in from that session is from the same source. For example, a cheating player wouldn’t be able to send fake requests to impersonate other players and mess with them. As with REST servers, the safest way to ensure that players don’t cheat is to have the game’s logic run server-side and make the game advance only through a specific sequence of actions that need to happen

This concludes our series about servers. We hope this has been a valuable learning experience for you and that you can bring these ideas into amazing projects in the near future. All of the examples linked throughout this series are open source and meant to be repurposed and remixed, don’t hesitate to use them as starting points for your own work.

If you have any questions or issues, the best place to find answers is in our support-sdk channel on Discord. Happy coding!

Jul 20, 2020 Tutorials

Servers Deep Dive Part 3: Using WebSockets

Master advanced tools for creating rich multiplayer experiences in Decentraland.

Create a server

Obtain TLS Certificates

Alternative 1: Download the server as a Docker image

Alternative 2: Write custom code for the server

Connecting a scene to the server

Combining WS with physics

Server-side logic

Other examples

A note about security

Related Articles

Servers Deep Dive Part 2: Launching a Server

Servers Deep Dive Part 1: Hit an API

Creating Genesis Plaza: Part 2

Servers Deep Dive Part 3: Using WebSockets

Master advanced tools for creating rich multiplayer experiences in Decentraland.

Create a server

Obtain TLS Certificates

Alternative 1: Download the server as a Docker image

Alternative 2: Write custom code for the server

Connecting a scene to the server

Combining WS with physics

Server-side logic

Other examples

A note about security

Related Articles

Servers Deep Dive Part 2: Launching a Server

Servers Deep Dive Part 1: Hit an API

Creating Genesis Plaza: Part 2

Developers newsletter

Thank you for subscribing!