Tuesday, December 8, 2015

MPLS testbed on Ubuntu Linux with kernel 4.3

MPLS in the kernel

Linux 4.3 was released last month, and one of the long-awaited features was MPLS support in the kernel. There is still a the odd bug to iron out, but you can get a working MPLS testbed with the current kernel source (plus a single patch to fix a showstopper).

Building the kernel

  1. Download the source of kernel 4.3 from here: https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.3.tar.xz
  2. Unpack the tarball (tar -xf linux-4.3.tar.xz)
  3. Enter the newly-created linux-4.3 directory, run make menuconfig, and enable lwtunnel support, mpls-iptunnel support, mpls-gso support, and mpls-router support.
  4. Apply the patch from http://git.kernel.org/cgit/linux/kernel/git/davem/net.git/diff/?id=fe82b3300ec9c0dc4ba871f9a58b265aadf4e186 (this fixes a problem with sending MPLS packets)
  5. Build the kernel: make -j `getconf _NPROCESSORS_ONLN`
  6. Once this has finished, build the debian packages: make -j `getconf _NPROCESSORS_ONLN` deb-pkg LOCALVERSION=-mplsfix
  7. This will create a bunch of .deb files in the parent directory - copy both linux-image-4.3.0-mplsfix_amd64.deb and linux-headers-4.3.0-mplsfix_amd64.deb to the machine you want to install your new kernel on
  8. Install the kernel with dpkg -i [package name]
  9. Reboot, select Advanced options for booting Ubuntu, and choose your new kernel
  10. You are all ready to go!
edit: easier way with a docker container: https://github.com/samrussell/kernelbuilder

Enabling MPLS

The MPLS modules aren't loaded by default, so you'll need to load them yourself:

modprobe mpls_router
modprobe mpls_gso
modprobe mpls_iptunnel
sysctl -w net.mpls.conf.enp0s9.input=1
sysctl -w net.mpls.conf.lo.input=1
sysctl -w net.mpls.platform_labels=1048575

You'll need to set net.mpls.conf.[interface-name].input=1 for any other interfaces that you plan to receive MPLS packets on, otherwise the MPLS route table won't accept your routes.

Applying MPLS routes

The latest release of iproute2 isn't quite ready, so we'll need to live life on the bleeding edge and build this from source too

git clone git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git
cd iproute2
./configure
make
sudo make install

Once this is done, we can see that iproute2 has a few more options available for us - try ip route help and see what is available.

Some route examples:

Routing 10.10.10.10/32 to 192.168.1.2 with label 100: ip route add 10.10.10.10/32 encap mpls 100 via inet 192.168.1.2

Label swapping 100 for 200 and sent to 192.168.2.2: ip -f mpls route add 100 as 200 via inet 192.168.2.2

Decapsulating label 300 and delivering locally: ip -f mpls route add 300 dev lo

Testbed setup

We're going to make use of network namespaces here to set up a couple of hosts. The plan is as follows:
  • Base machine: has veth0 (plugs into veth1) and veth2 (plugs into veth3)
  • Host1: Has veth1 (plugs into veth0)
  • Host2: Has veth3 (plugs into veth2)
We will use label 111 for traffic from host1 to host2, and label 112 for traffic from host2 to host1. We will use penultimate hop popping here (as opposed to label swapping), but feel free to play with this and get different results.

Setup (all executed as root):

ip link add veth0 type veth peer name veth1
ip link add veth2 type veth peer name veth3
sysctl -w net.mpls.conf.veth0.input=1
sysctl -w net.mpls.conf.veth2.input=1
ifconfig veth0 10.3.3.1/24 up
ifconfig veth2 10.4.4.1/24 up
ip netns add host1
ip netns add host2
ip link set veth1 netns host1
ip link set veth3 netns host2
ip netns exec host1 ifconfig lo 10.10.10.1/32 up
ip netns exec host1 ifconfig veth1 10.3.3.2/24 up
ip netns exec host2 ifconfig lo 10.10.10.2/32 up
ip netns exec host2 ifconfig veth3 10.4.4.2/24 up
ip netns exec host1 ip route add 10.10.10.2/32 encap mpls 112 via inet 10.3.3.1
ip netns exec host2 ip route add 10.10.10.1/32 encap mpls 111 via inet 10.4.4.1
ip -f mpls route add 111 via inet 10.3.3.2
ip -f mpls route add 112 via inet 10.4.4.2

Testing (executed as root due to netns):

ip netns exec host2 ping 10.10.10.1 -I 10.10.10.2

Results:

tcpdump -envi veth0
tcpdump: listening on veth0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:14:14.687380 9a:08:f4:cf:aa:9c > 12:c7:db:9d:a5:25, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 53781, offset 0, flags [DF], proto ICMP (1), length 84)
    10.10.10.2 > 10.10.10.1: ICMP echo request, id 1359, seq 1, length 64
21:14:14.687404 12:c7:db:9d:a5:25 > 9a:08:f4:cf:aa:9c, ethertype MPLS unicast (0x8847), length 102: MPLS (label 112, exp 0, [S], ttl 64)
(tos 0x0, ttl 64, id 19009, offset 0, flags [none], proto ICMP (1), length 84)
    10.10.10.1 > 10.10.10.2: ICMP echo reply, id 1359, seq 1, length 64
21:14:15.701789 9a:08:f4:cf:aa:9c > 12:c7:db:9d:a5:25, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 53845, offset 0, flags [DF], proto ICMP (1), length 84)
    10.10.10.2 > 10.10.10.1: ICMP echo request, id 1359, seq 2, length 64
21:14:15.701810 12:c7:db:9d:a5:25 > 9a:08:f4:cf:aa:9c, ethertype MPLS unicast (0x8847), length 102: MPLS (label 112, exp 0, [S], ttl 64)
(tos 0x0, ttl 64, id 19246, offset 0, flags [none], proto ICMP (1), length 84)
    10.10.10.1 > 10.10.10.2: ICMP echo reply, id 1359, seq 2, length 64

tcpdump -envi veth2
tcpdump: listening on veth2, link-type EN10MB (Ethernet), capture size 262144 bytes
21:14:45.714220 8e:d5:9d:07:9a:5c > d6:8a:7c:5e:5b:0f, ethertype MPLS unicast (0x8847), length 102: MPLS (label 111, exp 0, [S], ttl 64)
(tos 0x0, ttl 64, id 55648, offset 0, flags [DF], proto ICMP (1), length 84)
    10.10.10.2 > 10.10.10.1: ICMP echo request, id 1363, seq 1, length 64
21:14:45.714251 d6:8a:7c:5e:5b:0f > 8e:d5:9d:07:9a:5c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 22394, offset 0, flags [none], proto ICMP (1), length 84)
    10.10.10.1 > 10.10.10.2: ICMP echo reply, id 1363, seq 1, length 64
21:14:46.717538 8e:d5:9d:07:9a:5c > d6:8a:7c:5e:5b:0f, ethertype MPLS unicast (0x8847), length 102: MPLS (label 111, exp 0, [S], ttl 64)
(tos 0x0, ttl 64, id 55848, offset 0, flags [DF], proto ICMP (1), length 84)
    10.10.10.2 > 10.10.10.1: ICMP echo request, id 1363, seq 2, length 64
21:14:46.717570 d6:8a:7c:5e:5b:0f > 8e:d5:9d:07:9a:5c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 22412, offset 0, flags [none], proto ICMP (1), length 84)
    10.10.10.1 > 10.10.10.2: ICMP echo reply, id 1363, seq 2, length 64

It works!

Next steps

We have software routers such as Quagga and BIRD, and these speak some of the more traditional protocols such as OSPF and BGP. We now need LDP daemons, and other linux software to stand up l2vpn and l3vpn.

Thanks to the team on the netdev mailing list, they have been super responsive and helpful.

Thursday, July 9, 2015

Deepdream: What do all the layers do?

I spent last night getting my computer prepped for some deep dreaming, and it left me thinking: What do all the different layers do? There's over a hundred to choose from, so why not iterate through them all and see what happens?

I used this as my starting point: https://github.com/Dhar/image-dreamer

My base picture is one I took from a plane out of Queenstown (munged version here), resized to 400px wide, and run through the layers as follows:


conv1/7x7_s2

pool1/3x3_s2

pool1/norm1
conv2/3x3_reduce
conv2/3x3
conv2/norm2
pool2/3x3_s2
inception_3a/1x1

inception_3a/3x3_reduce
inception_3a/3x3
inception_3a/5x5_reduce
inception_3a/5x5
inception_3a/pool
inception_3a/pool_proj
inception_3a/output
inception_3b/1x1
inception_3b/3x3_reduce
inception_3b/3x3
inception_3b/5x5_reduce
inception_3b/5x5
inception_3b/pool
inception_3b/pool_proj
inception_3b/output
pool3/3x3_s2
inception_4a/1x1
inception_4a/3x3_reduce
inception_4a/3x3
inception_4a/5x5_reduce
inception_4a/5x5
inception_4a/pool
inception_4a/pool_proj
inception_4a/output
inception_4b/1x1
inception_4b/3x3_reduce
inception_4b/3x3
inception_4b/5x5_reduce
inception_4b/5x5
inception_4b/pool
inception_4b/pool_proj
inception_4b/output
inception_4c/1x1
inception_4c/3x3_reduce
inception_4c/3x3
inception_4c/5x5_reduce
inception_4c/5x5
inception_4c/pool
inception_4c/pool_proj
inception_4c/output
inception_4d/1x1
inception_4d/3x3_reduce
inception_4d/3x3
inception_4d/5x5_reduce
inception_4d/5x5
inception_4d/pool
inception_4d/pool_proj
inception_4d/output
inception_4e/1x1
inception_4e/3x3_reduce
inception_4e/3x3
inception_4e/5x5_reduce
inception_4e/5x5
inception_4e/pool
inception_4e/pool_proj
inception_4e/output
pool4/3x3_s2
inception_5a/1x1
inception_5a/3x3_reduce
inception_5a/3x3
inception_5a/5x5_reduce
inception_5a/5x5
inception_5a/pool
inception_5a/pool_proj
inception_5a/output
inception_5b/1x1
inception_5b/3x3_reduce
inception_5b/3x3
inception_5b/5x5_reduce
inception_5b/5x5
inception_5b/pool
inception_5b/pool_proj
inception_5b/output