Setting Up CJDNS on pfSense

CJDNS is an IPv6 encrypted mesh networking technology, used in Hyperborea and other mesh networks around the world. I particularly like the way it’s conceived, with end-to-end encryption, distributed IP address allocation, DHT-based source routing and MPLS-like label switching.

It has several interesting use cases, but I wanted to set it up in my pfSense gateway VM so I can use it as a distributed VPN of sorts. This could be useful to be able to access my systems from anywhere while traveling, or where my team can share access to each other’s Cloud Foundry labs, particularly when we travel.

For folks with Linux or Macs is relatively easy to setup. However, pfSense 2.3 is based on FreeBSD, and it doesn’t get pkg packages from mainline, but rather from it’s own repo. Because of that, the only way to obtain a reliable binary was to setup a FreeBSD 10.3 environment to perform the compilation from code and then copy the binary.

Compiling on FreeBSD

The easiest way to compile the code on my Mac was to do it via a Vagrant box. Make sure you have the latest versions of Virtualbox and Vagrant installed, and then from your command line do:

cd ~/workspace
mkdir -p FreeBSD && cd FreeBSD
vagrant init freebsd/FreeBSD-10.3-RELEASE
vagrant up

In my case, I got an odd error of a missing MAC address in the package, but I executed vagrant up a second time and my VM was up.

SSH into your VM with ssh -p 2222 vagrant@127.0.0.1 and password vagrant.

Become root with sudo -i and update your packages with pkg update.

Next, create a working environment:

mkdir -p ~/workspace && cd ~/workspace
pkg install gmake node git bash
git clone https://github.com/cjdelisle/cjdns.git cjdns
cd cjdns
./do

This should produce the ./cjdroute executable, which is really all you need. You can test it works by just running its crypto benchmark included: ./cjdroute --bench. It should give you no errors.

You can now safely transfer that code to your pfSense system.

Since the filesystem is read-only by default, you first need to login as root (admin) and enable read-write with /etc/rc.conf_mount_rw before you can do anything.

I placed the binary in an internal fileserver and later copied it via scp from inside pfSense. /usr/local/bin is a good location, while /etc is a good place for our cjdroute.conf file. It would be safe to execute a similar benchmark in the pfSense system to make sure it works as expected. In my case, I’m using the same amd64 architecture and I didn’t see any problems, but if you do, you will probably want to cross compile it for your target architecture.

Configuring pfSense

Configuring the CJDNS Tunnel

The first step is to create the config file with /usr/local/bin/cjdroute --genconf > /etc/cjdroute.conf and modify the config file according to your needs. Protect it with chmod 600 /etc/cjdroute.conf.

Setting up the tunnel interface is not really complicated. At this point you just follow the directions found in GitHub, and share your public keys and IDs with your friends, which were autogenerated in the config file you created above.

Take note of the autogenerated IPv6 address and port, since we’ll use them later.

If you are going to use pfSense to receive incoming cjdns traffic, you need to poke a hole in the firewall to accept traffic in the incoming port on the WAN interface. You can use IPv4, IPv6 or both. You don’t need to do that if you are planning to connect to an external device, or to a VM instance in the cloud.

During the testing phase, you may want to enable logging to the console, to determine what’s going on. You can do that by uncommenting "logTo":"stdout" in the logging section of the config file. Bear in mind the log output is very chatty.

The next step is to test that this actually works. You can do that by executing /usr/local/bin/cjdroute --nobg < /etc/cjdroute.conf as root. Since the --nobg option was specifed, you may want to open another ssh session to the gateway. At this point, you should be able to see your tun0 interface if you do an ifconfig, and see the IPv6 address that you were assigned in the config file.

It would be a good idea you spend as much time needed in this point until you know you have a good network connection and you can establish a connection with some of your peers.

However, you must remember that you don’t have any firewall rules in that interface at this moment, so you need to be cautious. Don’t test on a real production system against networks like Hyperborea where you may not know all the peers.

You should also make sure you start CJDNS at boot time. There are more elegant ways to do it, but I just put the start sequence above in the /etc/rc right before the last line.

Configuring the Virtual interface

The tricky thing here is to be able to add firewall rules in a consistent manner with the way pfSense operates.

When you setup permanent OpenVPN connections with other end points, it’s actually quite simple to assign a virtual interface to it and create the proper firewall rules the same way you do it with regular Ethernet interfaces.

Unfortunately, pfSense doesn’t recognize tun0 as an interface you can apply a virtual interface to, probably because of the rather transient nature of tunnels in most VPN environments.

In CJDNS we expect this connection to be up and running at all times, so we need to trick the system to recognize the tun0 interface as valid.

First you want to force the tunnel to always use the tun0 interface. For that, we forcefully set the "tunDevice": "tun0" in the cjdroute.conf file under router -> interface.

Second, we need to edit the /etc/inc/util.inc file. Find the get_interface_list function and comment out (or delete) the line with 'tun', like this: /* 'tun', */. Save the file, and now you can go to your Web UI and navigate to interfaces -> (assign). There you will now see tun0 as an Available interface. Click on the Add button. You can change the name from OPT1 or something like CJDNS or similar. Don’t configure any parameter here. We don’t need to act on that interface in any way except from the firewall perspective.

At this point, all the manual configuration of pfSenses is done, so you need to re-enable read-only mode with /etc/rc.conf_mount_ro. Reboot the system and make sure things are the way you expect before going to the next step.

Static routing

In order to use the interface, we need to define it as a gateway in pfSense, and add a static route to it.

In the GUI, go to System -> Routing -> Gateways. Add a new gateway. Define it as IPv6 only, use a descriptive name (such as CJDNS) and set the IPv6 address of your gateway in the gateway IP field. This is the autogenerated IPv6 from your cjdroute.conf file.

Next, save it and go to Static Routes, the tab next to Gateways. Here, define a static route to fc00::/7 going through the gateway you just defined.

Conclusion

This is just a baseline setup, but the potential of CJDNS is incredible. It can be used in rural areas to connect p2p among residents, and to share one or few Internet links. If can be used in emergency situations to create quick ad-hoc mesh networks for first responders. It can send beacons in adjacent L2 networks for quick mesh setups without any special configuration.

You can run any IP service inside the CJDNS network with or without any gateways to the outside Internet, like DNS, web sites, Twitter-like services (based on open source technologies like pump.io). It provides strong encryption in motion and the endpoints are static and well-known, and it’s very fast.

It’s NOT Tor, nor it’s designed to be anonymous, but provides reasonably good privacy. It blurs the distinction with general purpose VPNs and can potentially simplify the setups when used for that purpose.

 
comments powered by Disqus