Migrating from the Unifi USG3P to the Unifi Dream Machine Pro

If the title wasn't clear enough, this one isn't HAM related.

I've been using a Unifi Security Gateway 3P - otherwise known as the USG3P for a couple of years. And it's been pretty good. 

For a glorified router it's done a reasonable job. I have a complex enough home network:

• 12 VLANs, each with their own subnet
• Around 6 WLANs, with different VLANs attached
• Multiple switches
• 3 Access Points (and more to come)
• An average of around 80 IP connected devices and VMs running at any time.

 It's not always been smooth sailing though: When I turned on IPv6 weird and wonderful things happened a lot between subnets. Especially when a VM would wind up with IPs from nearly every subnet. In hindsight, I probably should have given the IPv6 side some isolation rules too. Oops. 

The more frustrating issue was to do with the "Security" part. Turning on the IDS/IPS features that made it a "security" product took the throughput capability from 1Gbps down to an apparent 85Mbps. 

I say apparent because I'm on NBN Fixed Wireless, which these days runs around 85Mbps. It couldn't cope with it. Especially when I had IPv6 on. Turning it off certainly helped stop the CPU staying at 100%, but it still struggled. A device that happily manages to otherwise run for years at a time (I'm prone to having rather large UPSes), could barely last a day before it'd fall over if I did anything close to run the Internet connection at full speed. Something that I do, indeed do. Often.

In the end I turned off the "security" features and left it alone throughout the whole pandemic. And it went back to working as expected for the duration. 

It's been reliable enough to keep me interested in Unifi gear. There's always quirks, hurdles and risks when updating firmware, but it's an ecosystem that I like. Need a new access point? Connect. adopt, use. 

I decided recently to treat myself to the Unifi Dream Machine Pro - or UDM Pro. 

It's quite a bit more expensive than the USG3P, but has a lot more processing power. The IDS/IPS function will drop the WAN bandwidth down to 3.5Gbps.. from the 10Gbps capability it has otherwise. More than enough to handle.. well.. any residential Internet connection in Australia for the next few years. 

The non-Pro model of this is a weird looking white gizmo that sits on its side and looks a bit like its creator was staring at a portable sports speaker. Not a fan. 

This box has a pair of 10Gbps SPF+ ports - which is good given that I have a pair of servers that move the bulk of the data between them and this avoids me having secondary direct connections between the boxes for high bandwidth. 

It has a single 1Gbps "WAN" port, though one of the SFP+ ports can be configured as either a secondary WAN or the actual WAN port should you happen to be so fortunate enough to have such a fast connection. 

The 8 port switch is a bit of a misnomer. All 8 1Gbps ports were apparently designed with a single 1Gbps backplane between them. I've heard rumours that later revisions of the hardware may have been upgraded to 2.5Gbps. Either way, that's not terribly appealing. 

At the moment I'm using a single connection from that to my primary switch. At some point I'm going to redirect the cabling from my camera switch over to it to reduce the load on the link between the UDM Pro and the main switch. The data is going into a server connected through the SFP+ port anyway. 

The cutover:

I'd read beforehand that unlike any other Unifi product, that this one couldn't be adopted into my existing Unifi network controller - simply because it runs its own. Hmm. There's a lot of work in that configuration. 

I read some doco on how to export my configuration and import it into the UDM Pro. 

Once it arrived, I set it up. It's necessary to use the phone app initially, and to have the WAN port connected to the Internet. This will allow the app to find your device, associate it to your account, then allow you to configure it through Unifi's website. 

The fun starts now.

Firstly, the controller tells me that there is an outstanding CVE for older controller software and that I need to update. Not that it shows me how to do this. 

Elsewhere in the controller, it tells me that there is newer firmware, but pushing buttons to update it did.. nothing. 

I went around in circles for quite some time getting no where with this, eventually opting to download the latest firmware from their website, enabled SSH, used SCP to copy the update and manually updated. 

Huzzah. Finally the firmware is current.. as is the Network controller. Now that the controller version matched, I decided to import my configuration. 

Wednesday 19:45. Politely ask XYL to amuse herself somewhere away from Netflix for a few minutes. 

I reroute the cable from the NBN NTD to the WAN port on the UDM Pro so it has direct access, and move the LAN side of the UDM Pro to the uplink port on the switch. 

UDM Pro gets an IP address, but reports it can't see the internet. I can't see anything on my network that isn't on the same subnet as my PC. Hmm.. no routes. 

I note that the UDM could do with an IP address change so it has the same as the old USG3P. It won't change. I use my phone to do some research. Still won't change. 

I manually screw with my PC settings for a while, manually changing my gateway IP and using an external DNS and found that I could get Internet. That at least made it easier to keep fighting it. All of the network configuration is there. None of it works. 

Wednesday 21:30. I've been up since around 04:00, so I'm done. I start to roll back the network. Disconnect the UDM Pro, reconnect the USG3P. Restart the network switch. Fight a few other things so that everything will be working when I wake up tomorrow. 

Thursday 03:45. It's been eating at me all night, there's a few hours before I need the Internet working so I can work. 

I factory reset the UDM Pro and reconnect it to the network. I manually create every single network. Sadly, I can't recreate the Wireless networks until I have an access point connected. 

I start manually recreating all of the firewall rules that I need. 

Cut over time. Jump in, reset the IP address to the USG3's original address. Huzzah. I can still access the internet. And everything else that I need at the same time. 

All ready to go by around 07:30. 

Over the course of the day I got around to enabling the security features, and fighting with some quirks on one subnet that I reduced the size of where every Ubuntu box on it (including the physical ones) started getting weird. A clean netplan template copy and paste though seems to have made all the difference. It is further complicated because a number of virtual machines on that particular subnet are configured to point to a pfSense router that is setup to tunnel traffic into a VPN via another subnet, so that all took some working through. 

Friday 05:45.

The UDM Pro has a current uptime of 1day 2 hours. The internet connection has been running flat knack for around 12 hours and it's all looking fine. It's processed some 376GB in that time - and that's just internet traffic. The actual traffic through the device in that time is some 2.4.TB. Around 1TB of that has gone back to the network switch. I can at least deduce from that that 1.4TB is left, around 400GB was internet traffic, so the remaining 1TB is likely to have been LAN traffic between the two servers.

Given the sheer amount of video my network handles, the the numbers make sense. And I need to stop having the camera switch pushing data to the network switch. 

Overall, the UDM Pro has been an interesting journey so far. It's an expensive toy and I don't hate it. I'll probably never use features like Unifi Protect as I don't care to change all of my cameras to theirs. The tiny touch screen is useful surprisingly and gives a nice little throughput graph that I wish the controller's UI provided. 

It'll be a lot more interesting when I start turning IPv6 back on. 

In the mean time, there really is a lot of "little things" I need to do. Much like when I started configuring the USG3P, I really need to start jumping inside my various subnets and testing the firewall rules are working. The original design blocked everything between subnets then provided a set of rules around that to allow specific things. 

My where my PC sits, that's hard to see - simply because my PC is allowed everywhere, so I need to poke around and make sure that holes exist only where I want them. 

In case you don't get why I care about security features:

Seeing things like this make me fell better. That's just this morning's noise.