Is setting up a VPS worth it?

Yes, if you want to save money and learn about networking on Linux along the way. Pieter Levels does it.

Horizontal scaling (more than one machine) is rarely needed. Vertical scaling (adding computing power to a machine) can go very far, millions-in-revenue far.

There are multiple platforms offering convenience when setting up a server, but free tiers are severely limited or nonexistent (as is now the case for Heroku). An economic environment of sane interest rates is likely to keep this as the standard. Comparatively, VPS’ can be radically affordable, with 1GB of RAM costing around $5/month at time of writing - and no limit on the number of servers you can run on it.

Personally, I also like the sense of ownership in setting up my own machine, and knowing exactly what it takes to make a website available to the world.

How much trouble is it?

Quite a lot. If you’re setting up a VPS for the first time, expect to spend a few days learning through trial and error.

There are lots of guides around for any thing you may need, but getting to the ground truth is surprisingly hard - configuration files are mystical and sometimes options may come out of nowhere with very slight explanation.

Take for example the systemd website, which has precisely 0 getting started guides or beginner friendly tutorials. It feels weird having to learn how to use these tools by averaging third party resources together.

I have been writing down the steps I’ve taken in a document, which is now at 200 lines. Some of the steps include:

Managing SSH connections
Setting up the ufw firewall
Setting up the NGINX reverse proxy server
Setting up HTTPS with the EFF’s certbot
Setting up a Python environment with pyenv and venv
Setting up systemd to manage the server as a service that boots with the OS

And more to come, such as log rotation (avoiding forever-growing files) & monitoring for errors, and looking into things like Cloudflare for additional security and caching.

Setting up a VPS

This guide is based on, among others:

Connecting

Initially use host web based terminal.

To connect from local machine, add SSH key to account, or if server is already live, might have to add to ~/.ssh/authorized_keys through web terminal.

Connect with ssh -i ~/.ssh/digital_ocean root@<ip>.

Start with apt update, apt upgrade, apt dist-upgrade.

Security

Logging in as root for now.

Setting up firewall:

ufw allow OpenSSH

ufw allow "Nginx Full" once NGINX is installed.

ufw enable

Disabling SSH password authentication:

nano /etc/ssh/sshd_config

Ensure PasswordAuthentication is set to no.

Install fail2ban:

sudo apt update sudo apt install fail2ban

Finally enable fail2ban so it runs automatically with systemctl enable fail2ban, then start it for the first time with systemctl start fail2ban.

Enabling alert emails

The below steps have not been written to the end. Digital Ocean blocks outgoing mail, so would need to get that resolved before testing this.

Enable alert emails:

cd /etc/fail2ban cp jail.conf jail.local nano jail.local

Change destemail to the destination email address, and action to %(action_mwl)s to get an email with a report and relevant logs.

This requires sendmail, so we run apt install sendmail (TODO: what next?)

NGINX

Run apt install nginx

Create file with nano /etc/nginx/sites-enabled/<app-name>. Replace <domain> with the actual domain:

server {
    listen 80;
    server_name <domain>;

    location /api {
        proxy_pass http://localhost:5000;

        # https://flask.palletsprojects.com/en/2.2.x/deploying/nginx/
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Prefix /;
    }
}

Disable NGINX default file: unlink /etc/nginx/sites-enabled/default
Reload NGINX configuration: nginx -s reload

HTTPS

Discord requires HTTPS for an application’s endpoint, and HTTPS is cool in general.

HTTPS must be set after NGINX is configured.

First open NGINX config with nano /etc/nginx/sites-enabled/<app name> and ensure server_name is set to the domain that will be included on the SSL certificate (such as app.example.com).

Also ensure that the port is set to 80 (listen 80), otherwise certbot will not configure NGINX to redirect HTTP to HTTPS.

Run nginx -t to check configuration file syntax.

Run systemctl reload nginx to reload NGINX.

Digital Ocean instructions appear to be outdated. Follow these instead.

snap install core; snap refresh core

snap install --classic certbot

certbot --nginx

NGINX config should now look like:

server {
    server_name app.example.com;

    location /api {
        proxy_pass http://localhost:5000;

        # https://flask.palletsprojects.com/en/2.2.x/deploying/nginx/
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Prefix /;
    }

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/app.example.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/app.example.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}
server {
    if ($host = app.example.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


        listen 80;
        server_name app.example.com;
    return 404; # managed by Certbot
}

Certbot should auto renew the certificate automatically. Test with certbot renew --dry-run.

Python

Ensure Python 3.11 or above is available, so that sqlite3.threadsafety works as intended (“Changed in version 3.11: Set threadsafety dynamically instead of hard-coding it to 1.”).

Using pyenv is recommended, since Ubuntu comes packaged with a specific version of Python that should not be interfered with.

Install pyenv following instructions in readme
pyenv install 3
pyenv global 3

App

apt install python3 python3-pip python3-venv -y
Generate SSH key to clone repo from GitHub: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent
Clone repo: git clone git@github.com:user/repo.git
cd into the repo directory
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
Create env file: nano .env. Make sure to set ENV=production and to not set PUBLIC_ADMIN_PAGE at all.
pip install gunicorn (instead of apt install to play well with venv)
Might have to rerun source venv/bin/activate before running gunicorn
Can start gunicorn now, but we’ll use systemd instead.

Systemd

We will use systemd to manage the gunicorn server as a service that starts alongside the OS.

Create nano /etc/systemd/system/discord-bot.service with the contents:

# Based on
# https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-gunicorn>

[Unit]
# Description=

[Service]
# User=root
WorkingDirectory=/srv/discord-bot
Environment="PATH=/srv/discord-bot/venv/bin"
ExecStart=/srv/discord-bot/venv/bin/gunicorn --workers 3 -b :5000 app:app
Restart=always

[Install]
WantedBy=multi-user.target

Gunicorn is started with number of workers equal to (number of cpu cores * 2) + 1.

Then to start the service run systemctl start discord-bot.

TODO

log rotation & monitoring for errors
cloudflare?

Update: automated & repeatable

There is a way to run these steps in an automated & repeatable way - by using cloud-init. I did this by creating a cloud-config.yaml.jinja that I paste into an input box in the Digital Ocean web UI when creating a new droplet.

The .jinja extension allows you to use templating so you don’t have to declare the same values more than once.

I include a sample, minimal file below.

Note that you have to be careful with the two first lines of the file. They have to be exactly in that order, and it took me some time to debug that.

## template: jinja
#cloud-config

# File used to initialize Digital Ocean droplet.

{# Set variables here #}

{# Sample variable #}
{% set myVar = "myValue" %}

# Disable SSH password auth
ssh_pwauth: false

users:
  - name: root
    # Ensure commands below run in bash shell, as that is what they were tested
    # on.
    shell: /bin/bash

# Specify which types of SSH key to generate (by default, all supported types
# are generated).
# A public key is required for SSH connections to work.
ssh_genkeytypes:
  - ed25519 # Best type
  - ecdsa # Required for Digital Ocean web based terminal :(

write_files:
  # Multiline variables indented by 6 spaces to avoid breaking yaml syntax.
  - path: /srv/{{ repo }}/.env
    content: |
      {{ env_file | indent(6) }}
  # Write commands into a bash script so that we can run them with bash
  # explicitly (by default /bin/sh points to dash and that's what `runcmd`
  # commands use).
  - path: /run/droplet-setup.sh
    permissions: "0700" # Executable
    content: | # Preserve newlines
      #!/bin/bash

      set -e # Exit on error
      set -u # Exit on unset variable
      set -o pipefail # Exit on pipe failure
      set -T # Inherit DEBUG and RETURN trap for functions
      set -C # Prevent file overwrite by > &> <>
      set -E # Inherit -e
      set -x # Log commands before running

      # Update packages
      printf "\n\n\n\n ========> Update packages \n\n\n\n"
      apt update -y
      # Same as apt upgrade but will add & remove packages as appropriate.
      apt dist-upgrade -y

      # You can add more commands here. My version of this file goes on to
      # execute every set up step.
runcmd:
  - "bash /run/droplet-setup.sh"
  - "rm /run/droplet-setup.sh"

What about you? Do you think it is worth it to go through the hassle of setting up a VPS? Let me know on X - I’m @voxelbased.

Marcos Pereira

Posts • About me • Notes • Concepts