Home Server Behind a NAT using a VPS Gateway with WireGuard VPN and nftables
2021 June 2
Okay, that's a cumbersome title, but I wanted to identify the specific technologies I'm using so people know straight out the gate whether this will be helpful.
This setup only handles IPv4. If you rely on IPv6 as well, some different configuration will be needed.
What are we doing here?
I don't have a static IP address, and my ISP doesn't let me accept incoming traffic, meaning I can establish connections with other servers, but no one can establish connections with mine. This is a problem when you, for example, want to run a server from your home.
Well, I run a server from my home, so this post describes how I work around that network restriction.
Overview
Since I can't accept incoming connections on my own server, I'm renting a VPS to accept those connections and forward them to my server. This has the added benefit of providing me with a static IP address.
Terminology
Throughout this post, I will refer to the computer in my home doing all the heavy lifting as my server, and I will refer to the VPS that just relays traffic as the VPS. Even though the VPS is also a server, the term "server" will be reserved for the computer in my home.
To provide networking between my server and the VPS, I use a WireGuard VPN. To forward the traffic, I set up nftables rules on the VPS.
Both my server and the VPS are running a SystemD-based GNU/Linux distro with Linux kernel >=5.6.
DNS
Point the DNS for your domain to the VPS, as that's the public-facing point of access to which people will connect.
WireGuard
My server and the VPS talk to each other using the WireGuard protocol. Fortunately, WireGuard is extremely easy to set up!
First, install WireGuard on both devices. WireGuard is included in the Linux kernel starting with version 5.6.
Now, we generate our private and public keys on each device. I'll describe the relevant commands, but we'll end up doing it all in one line.
wg genkey
This generates a private key and prints it to the console.
wg pubkey
This takes a private key from stdin and computes the public key from it. For example, if the private key is stored in a file called privatekey, we could use
cat privatekey | wg pubkey
Okay, let's do it! First, elevate to root if you're not root already.
If you use sudo, you can
sudo su
Otherwise, use su and type in the root password.
su
Next, cd into /etc/wireguard
cd /etc/wireguard
We want our new files to be accessible only by root, so we'll use umask.
umask 077
Now, we're ready to start generating our keys! We can do this in one line.
wg genkey | tee privkey | wg pubkey > pubkey
This
- generates a private key
- outputs the private key in the file privkey
- uses the private key to compute the public key
- outputs the public key in the file pubkey
We actually don't need either of these files, but we do need the two keys to write the config files, so for now it's good to have them easily accessible.
Now, we write the config files. On the server, write to the file wg0.conf:
[Interface] PrivateKey = <the server's private key> Address = 10.0.0.1 [Peer] PublicKey = <the VPS's public key> AllowedIPs = 10.0.0.2 Endpoint = <the VPS's IP address>:55107 PersistentKeepalive = 30
In this example, when they use the VPN connection, the VPS will identify the server as 10.0.0.1, and the server will identify the VPS as 10.0.0.2. They will connect over port 55107. These values aren't significant; you can choose other unreserved local addresses and an unused port. The VPS must be allowed to receive incoming traffic to that port, and the server must be allowed to make outgoing connections on that port.
Now, we'll write the VPS config. On the VPS, write to wg0.conf:
[Interface] PrivateKey = <the VPS's private key> ListenPort = 55107 Address = 10.0.0.2 [Peer] PublicKey = <the server's public key> AllowedIPs = 10.0.0.1
Again, you can change the IP addresses and port if you want.
That's it! Now, we just need to run both sides of the VPN. The VPS will listen for connections, and the server will try to make a connection to it. Once they're connected, they'll be able to send data back and forth.
Each device can start its side of the connection with
wg-quick up wg0
and stop it with
wg-quick down wg0
We can also use SystemD to start:
systemctl start wg-quick@wg0.service
and stop:
systemctl stop wg-quick@wg0.service
I'll assume we want this to run at startup on both devices so that the connection is always active when possible. Enable the SystemD service on both devices.
systemctl enable wg-quick@wg0.service
The two devices can now talk to each other using the IP addresses 10.0.0.1 (which identifies the server) and 10.0.0.2 (which identifies the VPS).
Enable IP forwarding with sysctl
Most systems have IP forwarding disabled by default. On the VPS, run
sysctl net.ipv4.ip_forward
to see. If it prints out "net.ipv4.ip_forward = 0" then you need to enable IP forwarding before forwarding will work.
Edit /etc/sysctl.conf and add the line
net.ipv4.ip_forward = 1
This will enable IPv4 forwarding when the VPS starts up. After changing this file, to load your new configuration for this boot, run
sysctl -p
nftables
Now we needed to set our firewall to forward traffic from the VPS to the server.
On the VPS, install nftables if it's not already installed.
Now, edit the file /etc/nftables.conf and make it look something like this:
#!/usr/sbin/nft -f flush ruleset define vps_ip = 10.0.0.2 define server_ip = 10.0.0.1 define relay_public_ip = <the VPS's IP address> define forward_tcp_ports = { <comma-separated list of ports you need for TCP> } define forward_udp_ports = { <comma-separated list of ports you need for UDP> } table inet filter { chain input { type filter hook input priority 0; # accept any localhost traffic iif lo accept # accept traffic originated from us ct state established,related accept # drop invalid packets ct state invalid counter drop # accept ssh tcp dport <the port you use for ssh to the VPS> accept # accept wireguard on 55107 tcp dport 55107 accept udp dport 55107 accept ## accept on other needed ports tcp dport $forward_tcp_ports accept udp dport $forward_udp_ports accept iif wg0 accept # drop any other traffic drop } chain forward { type filter hook forward priority 0; } chain output { type filter hook output priority 0; } } table ip nat { chain prerouting { type nat hook prerouting priority -100; iif <network interface> tcp dport $forward_tcp_ports dnat to $server_ip iif <network interface> udp dport $forward_udp_ports dnat to $server_ip } chain postrouting { type nat hook postrouting priority 100; masquerade } }
There's a lot going on here, so let's go through it...
At the top, we define some variables for convenience. In particular, forward_tcp_ports and forward_udp_ports are convenient so you don't have to type the ports repeatedly. Let's say you're running a web server (80/TCP and 443/tcp) and a Mumble server (64738/TCP and 64738/UDP). Your variables might look like this:
define forward_tcp_ports = { 80,443,64738 } define forward_udp_ports = { 64738 }
I prefer to run ssh on an alternative port to the default 22. It doesn't meaningfully improve security or anything, but it reduces the number of annoying bots trying to break into the server.
In the nat table, the prerouting chain has two lines (one for TCP and one for UDP) to forward traffic on $forward_tcp_ports and $forward_udp_ports to the server. The <network interface> will vary based on how your VPS's networking is set up, but you can find it, for example, with the command
ip addr
and figuring out which one is the network to the internet. (It will not be lo or wg0.) It might be something like eth0 or ens3. These lines say to forward traffic coming in on that interface (from the internet) on the specified ports to the server over the WireGuard VPN connection.
The postrouting chain does the opposite for replies from the server. (From what I understand, as I don't fully get all this yet) the "masquerade" command makes the response packet appear to come from the VPS, rather than the server, when it's sent to the client. In other words, the client just thinks it's talking to the VPS and doesn't need to know how the back-end (the actual server) is set up. (It turns out that if you just specify masquerade and not any more specific rules, nftables can just figure it out.)
Once your nftables rules are how you want them, restart nftables.
systemctl restart nftables.service
Now, hopefully, everything works!
(The server's firewall is independent of the VPS's firewall, and you will need to set it up separately. You can use nftables, iptables, ufw, firewalld, etc. Your choice. Right now, I'm using nftables with a configuration similar to the VPS's config, just without the NAT rules.)
Security Considerations
Why do all this? Why not just run everything on the VPS?
Well, for one thing, it ends up being cheaper for me. Since I already own hand-me-down hardware to use as a server, and the VPS only needs to relay traffic, I can pay for the lowest-specs-cheapest-cost VPS available. (Reusing hardware is also good practice as it reduces e-waste.)
That's not actually why I chose that setup, though. For me, it's about control. It's important to me to do my computing on hardware that I physically control. All else is untrusted.
How does this attitude fit with my setup? Well, my TLS private keys are stored on the server, and the VPS cannot decrypt the TLS connection between my server and the client; it simply forwards encrypted packets back and forth. (That said, it may still collect metadata on who connects, on which ports, when, and how much data is transferred.)
However, the VPS could provide its own response to clients, rather than forwarding traffic to the server. Since the DNS points to the VPS, it could even generate its own genuine, signed-by-a-trusted-CA TLS certificates for my domain.
But this seems to me like the best way to handle running services on an ICANN domain behind a NAT.
Thanks
I learned from various tutorials and tried some different techniques when I was setting this up. I don't remember which ones I used specifically, but I'm thankful to the authors and generally to everyone who writes tutorials and makes them publicly available on the internet.
haskal was a big help pointing me in the right direction and explaining how the pieces should fit together.