Hetzner Root-Server with Debian/KVM IPv4 and IPv6 Networking
Introduction
Since the IPv4 address space is almost completely used up, the future protocol is IPv6 which has been around for lots of years already and it is highly recommended to at least enable it for your machines and use it in dual-stack mode - so your service or website is available in the old IPv4 internet as well in its IPv6 counterpart.
I've been working with IPv6 for 6 years already - started with an IPv6 Tunnel from Hurricane Electric (They offer free IPv6 certification by the way!) over a SixXS Tunnel to native IPv6 with the latest KabelDeutschland modem firmware update. Also some years ago i started to use dual-stack IP for the first customer servers (mostly single purpose Root-servers or vServers from Hetzner or JiffyBox/Domainfactory). Later i'm going to migrate most smaller virtual customer machines to my larger Root-server machines which i can maintain better than any single vServer.
The largest problem i encountered was not setting up dual-stack for the Server itself - that was the easy part - but to enable it also for every single virtual machine, so that the customers servers had their own IPv4 and IPv6 address - globally routed of course.
This was the largest problem because i did not find any complete tutorials - so this is why i wrote this one. I think there are a lot more people around who are searching for the same answers. So here they are.
Limited to Debian and KVM?
Of course, not! You can use whatever distribution or virtualization technology you want. I chose Debian because i've been working with it since Woody and Etch and because i'm using KVM for my Root-Servers due to their damn good virtualization performance and good direct kernel integration.
Besides that you may need to adapt certain commands or configurations to fit your distribution or virtualization technology but i try to explain it as generic as possible.
What about using libvirt itself?
This is also possible and there is a very neat solution. Check out this german tutorial available here.
What do i normally get from Hetzner?
Normally you get a single IPv4 address and an /64 IPv6 Subnet with more than enough addresses. This is 18.446.744.073.709.551.616 host addresses - have fun using them all!
For this setup i chose another /29 IPv4 Subnet (with 6 host adresses when using it as a /29 subnet or with 8 host addresses when using all single addresses) to give every customer it's own IPv4 address.
For explanation here is a small list of what addresses we have now:
140.194.250.23 - this is the single IPv4 for the Root-server. 56.23.34.56/29 - this is the additional /29 Subnet or if you need to see all addresses it would read like this 56.23.34.56 - 56.23.34.63.
Besides that we have our single IPv6 Subnet. I explicitly made single bold because most tutorials on how to set up Hetzner IPv6 Networking is about ordering a second IPv6 Subnet - which is not necessary (anymore). The Subnet is this one 2a01:4f8:212:2122::/64. I won't explain how to read IPv6 addresses here - there are much better tutorials for this out there.
Setup all the networks!
Setup the KVM Host (or physical machine)
At first let us setup the Host. I will post the complete configuration here with inline comments and some explanation afterwards, so experienced sysadmins can just use my configuration without reading all the stuff afterwards.
Be sure to install the bridge-utils package beforehand, otherwise we cannot setup the VM-Bridge.
# Loopback device:
auto lo
iface lo inet loopback
# device: eth0
auto eth0
iface eth0 inet static
address 140.194.250.23 # Single Host IPv4 address
netmask 255.255.255.255
gateway 140.194.250.4 # The Hetzner Gateway IP
pointopoint 140.194.250.4
iface eth0 inet6 static
address 2a01:4f8:212:2122::2 # We gave our Host a single IPv6
netmask 128 # /128 because our Bridge has the whole /64 Subnet!
gateway fe80::1 # The Hetzner IPv6 Gateway
# VM-Bridge
auto br0
iface br0 inet static
address 140.194.250.23 # Our Host will be the Gateway for all VM's
netmask 255.255.255.255
bridge_stp off # Some bridge configuration
bridge_ports none
bridge_fd 0
bridge_maxwait 0
# Add all single IP's from your /29 subnet
up route add -host 56.23.34.56 dev br0
up route add -host 56.23.34.57 dev br0
up route add -host 56.23.34.58 dev br0
up route add -host 56.23.34.59 dev br0
up route add -host 56.23.34.60 dev br0
up route add -host 56.23.34.61 dev br0
up route add -host 56.23.34.62 dev br0
up route add -host 56.23.34.63 dev br0
iface br0 inet6 static
address 2a01:4f8:201:2122::2 # Our Host again as the Gateway
netmask 64 # Here we use the /64 subnet!What we did here besides setting up the loopback device is to configure the hosts eth0 device first because we want to be able to SSH into it later too. This configuration is straight forward and normally already been done by the installimage command from the rescue system. The only thing to mention here is the /32 netmask. Normally we are in a subnet with our neighbors from our rack - so mostly some kind of 255.255.255.192 or something. We don't want to talk to our neighbors so we set the /32 subnet. Also the pointopoint directive is important and tells our host to send all requests to the Hetzner IPv4 Gateway - even if we want to talk to our neighbors later.
The next configuration part is our static IPv6 config. This is quite easy. We just need to choose a single IPv6 address for our host. Important here is to choose a /128 subnet (it's like a 255.255.255.255). I'll explain it a bit later why. And also we add the Hetzner IPv6 Gateway here which is just a link-local address.
Now the interesting part. The VM-Bridge where we will add all our VM's later.
At first set up the IPv4 part by adding the usual bridge config. The bridge address will be our Host-IP so our VM's can send their stuff trough the Host to the internet. After some basic bridge config directives we are adding our ordered /29 subnet. We could just add it as a subnet in a single line but we would loose 2 IP addresses because a subnet always has a network address (56.23.34.56) and a broadcast address (56.23.34.63) and we would have only 6 usable host addresses. This is why we add every single IP as a single address and we can use all 8 IP's for customer machines. So just add all single routes to the bridge, so the bridge knows where to route the things to.
In the last step we add our globally routed(!) IPv6 Subnet - now as the desired /64 subnet. We did'nt have used /64 for our Host because our Host will be part of the IPv6 Network and it will not consume the whole subnet for itself. Both configurations are crucial for a working Guest-IPv6! Again our Host's IPv6 addresses is the address for the bridge and at the same time the gateway address for our virtual machines later.
Now we also need to change some kernel parameters for IP forwarding.
net.ipv4.ip_forward=1
net.ipv4.conf.eth0.send_redirects=0
net.ipv6.conf.all.forwarding=1You can set them on-the-fly like this sysctl -w net.ipv6.conf.all.forwarding=1 or add a config file in /etc/sysctl.d/99-networking.conf and add all three parameters in there by just copy and pasting them.
What we configure here is enabling IPv4 and IPv6 forwarding which is necessary for our host to act as the gateway for our guests. The .send_redirects=0 parameter tells our host not to send icmp redirect messages to our guests to tell them to directly use the Hetzner gateway. This is not possible because the Hetzner gateway doesn't know the MAC-adddresses of our guests. More about this later in Why do i need to choose my Host as the IPv4/v6 Gateway?
Now save this file and restart or save this file and set the parameters on-the-fly without rebooting.
Now let's continue with the guests - which is much easier.
Setting up the guests
Here is the complete guest configuration and the explanation afterwards.
# Loopback device:
auto lo
iface lo inet loopback
# device: eth0
auto eth0
iface eth0 inet static
address 56.23.34.56 # Any IP of our /29 subnet
netmask 255.255.255.255
gateway 140.194.250.23 # Our Host machine will do the job!
pointopoint 140.194.250.23
iface eth0 inet6 static
address 2a01:4f8:212:2122:56:23:34:56 # Any IPv6 of our /64 subnet
netmask 64 # The same subnet as the bridge!
gateway 2a01:4f8:212:2122::2 # Our Host is our gatewayYou can see that the guest config is much easier but you still have to take care of some things. At first, make sure that your guest's NIC is connected to the bridge br0 we configured on the host earlier.
Now we just configure IPv4 and IPv6 again. For IPv4 you can choose any IP of your /29 subnet and add the restrictive /32 subnet again. Remember that we don't like our neighbors that much!
For the gateway we will add our Host machine as well as for the route which is configured by the pointopoint directive (Take care of the single t in the middle!).
Now the IPv6 guest configuration is straight forward as well. Choose an IP out of those small number of possible host addresses. I prefer using the IPv4 as the second part of the IPv6 so i can remember it easier later but you can literally choose any IP you want as long as the prefix remains the same!
Then add the same /64 netmask and our Host's IPv6 address as the gateway.
Why do i need to choose my Host as the IPv4/v6 Gateway?
This is because Hetzner uses a fully routed IP network and assigns IP's to the known MAC-addresses of the physical machines. So while Hetzner will never know about our guests MAC-Addresses, we need to go this way.
Final words and references
I hope this post helps you setting up your machine. If there are any questions, leave them in the comments and i'll try to cover them up in an update.
For initial research i used those sites provided by Hetzner: