On docker stealing routes and breaking the internet

Docker allows you to "containerize" apps, running them in separate individual environments. This is really nice for repeatability: you can run the same container on a different system and expect it to still work. There are some good security arguments too, but for me having the docker service itself running as root raises a red flag. The problem is that at some point you need to connect to the app running in the container. Typically this is done through some web interface. In an attempt to make the UX seamless, docker routes IP address ranges from the real network into its virtual network(s) that bridge the containers. The kicker is that docker has no idea what IP ranges need to be routed so **it just routes everything!!!** 😡🤬 # What are the routes? ip route -n ... all the IPs -> docker Now in fairness, docker has no idea that you're about to keep using your PC like a regular PC after starting it /s. E.g. you want to connect to your work's VPN or start a webserver that expects connections not to be forwarded away. It is nice for first time users because docker can magically "just work". But that's no excuse for just funnelling every IP under the sun into its own network, which can "just break things". Ideally the default would be a tiny default range that is rarely used, or even have it make you configure the range. # E.g. ... loopback/ sudo ip route flush table 0 ip link set lo down ip link set lo up Failing to restart `lo` results in weird `bind: Cannot assign requested address` issues from local port forwarding or `Error: Failed to find a free local port for dynamic forwarding` from vscode. { ] }

Docker allows you to "containerize" apps, running them in separate individual environments. This is really nice for repeatability: you can run the same container on a different system and expect it to still work. There are some good security arguments too, but for me having the docker service itself running as root raises a red flag. The problem is that at some point you need to connect to the app running in the container. Typically this is done through some web interface. In an attempt to make the UX seamless, docker routes IP address ranges from the real network into its virtual network(s) that bridge the containers. The kicker is that docker has no idea what IP ranges need to be routed so **it just routes everything!!!** 😡🤬 # What are the routes? ip route -n ... all the IPs -> docker Now in fairness, docker has no idea that you're about to keep using your PC like a regular PC after starting it /s. E.g. you want to connect to your work's VPN or start a webserver that expects connections not to be forwarded away. It is nice for first time users because docker can magically "just work". But that's no excuse for just funnelling every IP under the sun into its own network, which can "just break things". Ideally the default would be a tiny default range that is rarely used, or even have it make you configure the range. # E.g. ... loopback/ sudo ip route flush table 0 ip link set lo down ip link set lo up Failing to restart `lo` results in weird `bind: Cannot assign requested address` issues from local port forwarding or `Error: Failed to find a free local port for dynamic forwarding` from vscode. { ] }

... r repeatability (you can run the same container on a different system and expect it to still work. There are some good security arguments too, but for me having the docker service itself running as root raises a red flag. The problem is that at some point you need to connect to the app running in the container. Typically this is done through some web interface. In an attempt to make the UX seamless, docker routes IP address ranges from the real network into its virtual network(s) that bridge the containers. The kicker is that docker has no idea what IP ranges need to be routed so **it just routes everything!!!** 😡🤬 # What are the routes? ip route -n ... all the IPs -> docker Now in fairness, docker has no idea that you're about to connect to your work's VPN and then start a webserver right after starting it (e.g. with `sudo service docker start`). It's also nice for first time users, because docker can magically "just work". But that's no excuse for just funnelling every IP under the sun into its own network, which can "just break things". Ideally the default would be a tiny default range, or even have it make you configure the range. Even worse is that clearing up the mess docker makes is hard. Docker doesn't remove routes that it adds when you stop the process. That is, you can break your system with `sudo service docker start` and stopping it won't fix anything. After stopping it you could be a chump and delete routes manually, but there can be a lot. sudo ip route del 172.19.0.0 br-5094d9589bea sudo route del -net 172.19.0.0 netmask 255.255.0.0 dev br-5094d9589bea You could type them in manually like a chump, but a shotgun approach is to delete all routes 💪. Be careful with this one as it really is all routes. You'll need to disconnect and reconnect your LAN/WiFi/VPN connection(s) right after for the regular default routes to be recreated. # Careful. This deletes everything! Don't run unless you're physically at the machine. sudo ip route flush table 0 How do you stop docker misbehaving? After cleaning up the mess it made with the above, configure it not to break things. Edit `/etc/docker/daemon.json` and add something like the following from [stack overflow](https://stackoverflow.com/questions/50730426/docker-bridge-connection-breaks-the-internet/67224830#67224830). I'm still pretty confused by what these actually do. I expect something here tells docker what default ranges to forward. Somewhere it needs a real IP to give its virtual interface to route IPs to. It also needs to assign virtual IPs within its internal network(s). Why does docker need so many bridge adapters 🤷. Reading the [docs](https://docs.docker.com/network/drivers/bridge/#use-the-default-bridge-network) might help. { "bip": "192.168.1.5/24", "fixed-cidr": "192.168.1.5/25", "default-address-pools":[ {"base":"192.168.2.5/24","size":28} ] }: you can run the same container on a different system and expect it to still work. There are some good security arguments too, but for me having the docker service itself running as root raises a red flag. The problem is that at some point you need to connect to the app running in the container. Typically this is done through some web interface. In an attempt to make the UX seamless, docker routes IP address ranges from the real network into its virtual network(s) that bridge the containers. The kicker is that docker has no idea what IP ranges need to be routed so **it just routes everything!!!** 😡🤬 # What are the routes? ip route -n ... all the IPs -> docker Now in fairness, docker has no idea that you're about to connect to your work's VPN and then start a webserver right after starting it (e.g. with `sudo service docker start`). It's also nice for first time users, because docker can magically "just work". But that's no excuse for just funnelling every IP under the sun into its own network, which can "just break things". Ideally the default would be a tiny default range, or even have it make you configure the range. Even worse is that clearing up the mess docker makes is hard. Docker doesn't remove routes that it adds when you stop the process. That is, you can break your system with `sudo service docker start` and stopping it won't fix anything. After stopping it you could be a chump and delete routes manually, but there can be a lot. sudo ip route del 172.19.0.0 br-5094d9589bea sudo route del -net 172.19.0.0 netmask 255.255.0.0 dev br-5094d9589bea You could type them in manually like a chump, but a shotgun approach is to delete all routes 💪. Be careful with this one as it really is all routes. You'll need to disconnect and reconnect your LAN/WiFi/VPN connection(s) right after for the regular default routes to be recreated. # Careful. This deletes everything! Don't run unless you're physically at the machine. sudo ip route flush table 0 How do you stop docker misbehaving? After cleaning up the mess it made with the above, configure it not to break things. Edit `/etc/docker/daemon.json` and add something like the following from [stack overflow](https://stackoverflow.com/questions/50730426/docker-bridge-connection-breaks-the-internet/67224830#67224830). I'm still pretty confused by what these actually do. I expect something here tells docker what default ranges to forward. Somewhere it needs a real IP to give its virtual interface to route IPs to. It also needs to assign virtual IPs within its internal network(s). Why does docker need so many bridge adapters 🤷. Reading the [docs](https://docs.docker.com/network/drivers/bridge/#use-the-default-bridge-network) might help. { "bip": "192.168.1.5/24", "fixed-cidr": "192.168.1.5/25", "default-address-pools":[ {"base":"192.168.2.5/24","size":28} ] } [Edit] maybe use `.2.5`? I hit yet another conflict with my home network and the above recently. See what I mean, re. frustrating. "bip": "192.168.2.5/24", "fixed-cidr": "192.168.2.5/25",

... r repeatability (you can run the same container on a different system and expect it to still work. There are some good security arguments too, but for me having the docker service itself running as root raises a red flag. The problem is that at some point you need to connect to the app running in the container. Typically this is done through some web interface. In an attempt to make the UX seamless, docker routes IP address ranges from the real network into its virtual network(s) that bridge the containers. The kicker is that docker has no idea what IP ranges need to be routed so **it just routes everything!!!** 😡🤬 # What are the routes? ip route -n ... all the IPs -> docker Now in fairness, docker has no idea that you're about to connect to your work's VPN and then start a webserver right after starting it (e.g. with `sudo service docker start`). It's also nice for first time users, because docker can magically "just work". But that's no excuse for just funnelling every IP under the sun into its own network, which can "just break things". Ideally the default would be a tiny default range, or even have it make you configure the range. Even worse is that clearing up the mess docker makes is hard. Docker doesn't remove routes that it adds when you stop the process. That is, you can break your system with `sudo service docker start` and stopping it won't fix anything. After stopping it you could be a chump and delete routes manually, but there can be a lot. sudo ip route del 172.19.0.0 br-5094d9589bea sudo route del -net 172.19.0.0 netmask 255.255.0.0 dev br-5094d9589bea You could type them in manually like a chump, but a shotgun approach is to delete all routes 💪. Be careful with this one as it really is all routes. You'll need to disconnect and reconnect your LAN/WiFi/VPN connection(s) right after for the regular default routes to be recreated. # Careful. This deletes everything! Don't run unless you're physically at the machine. sudo ip route flush table 0 How do you stop docker misbehaving? After cleaning up the mess it made with the above, configure it not to break things. Edit `/etc/docker/daemon.json` and add something like the following from [stack overflow](https://stackoverflow.com/questions/50730426/docker-bridge-connection-breaks-the-internet/67224830#67224830). I'm still pretty confused by what these actually do. I expect something here tells docker what default ranges to forward. Somewhere it needs a real IP to give its virtual interface to route IPs to. It also needs to assign virtual IPs within its internal network(s). Why does docker need so many bridge adapters 🤷. Reading the [docs](https://docs.docker.com/network/drivers/bridge/#use-the-default-bridge-network) might help. { "bip": "192.168.1.5/24", "fixed-cidr": "192.168.1.5/25", "default-address-pools":[ {"base":"192.168.2.5/24","size":28} ] }: you can run the same container on a different system and expect it to still work. There are some good security arguments too, but for me having the docker service itself running as root raises a red flag. The problem is that at some point you need to connect to the app running in the container. Typically this is done through some web interface. In an attempt to make the UX seamless, docker routes IP address ranges from the real network into its virtual network(s) that bridge the containers. The kicker is that docker has no idea what IP ranges need to be routed so **it just routes everything!!!** 😡🤬 # What are the routes? ip route -n ... all the IPs -> docker Now in fairness, docker has no idea that you're about to connect to your work's VPN and then start a webserver right after starting it (e.g. with `sudo service docker start`). It's also nice for first time users, because docker can magically "just work". But that's no excuse for just funnelling every IP under the sun into its own network, which can "just break things". Ideally the default would be a tiny default range, or even have it make you configure the range. Even worse is that clearing up the mess docker makes is hard. Docker doesn't remove routes that it adds when you stop the process. That is, you can break your system with `sudo service docker start` and stopping it won't fix anything. After stopping it you could be a chump and delete routes manually, but there can be a lot. sudo ip route del 172.19.0.0 br-5094d9589bea sudo route del -net 172.19.0.0 netmask 255.255.0.0 dev br-5094d9589bea You could type them in manually like a chump, but a shotgun approach is to delete all routes 💪. Be careful with this one as it really is all routes. You'll need to disconnect and reconnect your LAN/WiFi/VPN connection(s) right after for the regular default routes to be recreated. # Careful. This deletes everything! Don't run unless you're physically at the machine. sudo ip route flush table 0 How do you stop docker misbehaving? After cleaning up the mess it made with the above, configure it not to break things. Edit `/etc/docker/daemon.json` and add something like the following from [stack overflow](https://stackoverflow.com/questions/50730426/docker-bridge-connection-breaks-the-internet/67224830#67224830). I'm still pretty confused by what these actually do. I expect something here tells docker what default ranges to forward. Somewhere it needs a real IP to give its virtual interface to route IPs to. It also needs to assign virtual IPs within its internal network(s). Why does docker need so many bridge adapters 🤷. Reading the [docs](https://docs.docker.com/network/drivers/bridge/#use-the-default-bridge-network) might help. { "bip": "192.168.1.5/24", "fixed-cidr": "192.168.1.5/25", "default-address-pools":[ {"base":"192.168.2.5/24","size":28} ] } [Edit] maybe use `.2.5`? I hit yet another conflict with my home network and the above recently. See what I mean, re. frustrating. "bip": "192.168.2.5/24", "fixed-cidr": "192.168.2.5/25",

Boy this is frustrating. The internet just doesn't work with docker running and this is why... Docker allows you to "containerize" apps, running them in separate individual environments. This is really nice for repeatability (you can run the same container on a different system and expect it to still work. There are some good security arguments too, but for me having the docker service itself running as root raises a red flag. The problem is that at some point you need to connect to the app running in the container. Typically this is done through some web interface. In an attempt to make the UX seamless, docker routes IP address ranges from the real network into its virtual network(s) that bridge the containers. The kicker is that docker has no idea what IP ranges need to be routed so **it just routes everything!!!** 😡🤬 # What are the routes? ip route -n ... all the IPs -> docker Now in fairness, docker has no idea that you're about to connect to your work's VPN and then start a webserver right after starting it (e.g. with `sudo service docker start`). It's also nice for first time users, because docker can magically "just work". But that's no excuse for just funnelling every IP under the sun into its own network, which can "just break things". Ideally the default would be a tiny default range, or even have it make you configure the range. Even worse is that clearing up the mess docker makes is hard. Docker doesn't remove routes that it adds when you stop the process. That is, you can break your system with `sudo service docker start` and stopping it won't fix anything. After stopping it you could be a chump and delete routes manually, but there can be a lot. sudo ip route del 172.19.0.0 br-5094d9589bea sudo route del -net 172.19.0.0 netmask 255.255.0.0 dev br-5094d9589bea You could type them in manually like a chump, but a shotgun approach is to delete all routes 💪. Be careful with this one as it really is all routes. You'll need to disconnect and reconnect your LAN/WiFi/VPN connection(s) right after for the regular default routes to be recreated. # Careful. This deletes everything! Don't run unless you're physically at the machine. sudo ip route flush table 0 How do you stop docker misbehaving? After cleaning up the mess it made with the above, configure it not to break things. Edit `/etc/docker/daemon.json` and add something like the following from [stack overflow](https://stackoverflow.com/questions/50730426/docker-bridge-connection-breaks-the-internet/67224830#67224830). I'm still pretty confused by what these actually do. I expect something here tells docker what default ranges to forward. Somewhere it needs a real IP to give its virtual interface to route IPs to. It also needs to assign virtual IPs within its internal network(s). Why does docker need so many bridge adapters 🤷. Reading the [docs](https://docs.docker.com/network/drivers/bridge/#use-the-default-bridge-network) might help. { "bip": "192.168.1.5/24", "fixed-cidr": "192.168.1.5/25", "default-address-pools":[ {"base":"192.168.2.5/24","size":28} ] }

Boy this is frustrating. The internet just doesn't work with docker running and this is why... Docker allows you to "containerize" apps, running them in separate individual environments. This is really nice for repeatability (you can run the same container on a different system and expect it to still work. There are some good security arguments too, but for me having the docker service itself running as root raises a red flag. The problem is that at some point you need to connect to the app running in the container. Typically this is done through some web interface. In an attempt to make the UX seamless, docker routes IP address ranges from the real network into its virtual network(s) that bridge the containers. The kicker is that docker has no idea what IP ranges need to be routed so **it just routes everything!!!** 😡🤬 # What are the routes? ip route -n ... all the IPs -> docker Now in fairness, docker has no idea that you're about to connect to your work's VPN and then start a webserver right after starting it (e.g. with `sudo service docker start`). It's also nice for first time users, because docker can magically "just work". But that's no excuse for just funnelling every IP under the sun into its own network, which can "just break things". Ideally the default would be a tiny default range, or even have it make you configure the range. Even worse is that clearing up the mess docker makes is hard. Docker doesn't remove routes that it adds when you stop the process. That is, you can break your system with `sudo service docker start` and stopping it won't fix anything. After stopping it you could be a chump and delete routes manually, but there can be a lot. sudo ip route del 172.19.0.0 br-5094d9589bea sudo route del -net 172.19.0.0 netmask 255.255.0.0 dev br-5094d9589bea You could type them in manually like a chump, but a shotgun approach is to delete all routes 💪. Be careful with this one as it really is all routes. You'll need to disconnect and reconnect your LAN/WiFi/VPN connection(s) right after for the regular default routes to be recreated. # Careful. This deletes everything! Don't run unless you're physically at the machine. sudo ip route flush table 0 How do you stop docker misbehaving? After cleaning up the mess it made with the above, configure it not to break things. Edit `/etc/docker/daemon.json` and add something like the following from [stack overflow](https://stackoverflow.com/questions/50730426/docker-bridge-connection-breaks-the-internet/67224830#67224830). I'm still pretty confused by what these actually do. I expect something here tells docker what default ranges to forward. Somewhere it needs a real IP to give its virtual interface to route IPs to. It also needs to assign virtual IPs within its internal network(s). Why does docker need so many bridge adapters 🤷. Reading the [docs](https://docs.docker.com/network/drivers/bridge/#use-the-default-bridge-network) might help. { "bip": "192.168.1.5/24", "fixed-cidr": "192.168.1.5/25", "default-address-pools":[ {"base":"192.168.2.5/24","size":28} ] }