machine VM unreachable if the virtualbox host-only network conflicts with one of the machine host's network interfaces #3060
Description
Description of problem: If one of the host's network interfaces has the same subnet/broadcast as the host-only virtualbox network, there will be problems creating and accessing the docker-machine vm(s). They manifest themselves as "no route to host" errors during creation (visible with the --debug flag). The vm will actually launch and look like it is running, but then subsequent docker commands against it will get timeout errors.
I came across this because one of the wifi networks I use has 192.168.99.1/24 as its CIDR, which is the same subnet and broadcast that docker-machine uses by default for the virtualbox host-only network it uses.
It appears virtualbox will happily create a host-only network that conflicts with a host interface, and not print any errors. It just won't set up routing for it. They probably allow this because you can still set up a manual static route and get it to work - maybe for some advanced use-case.
For docker-machine, though, it just results in an unusable environment with cryptic errors.
docker version
:
Client:
Version: 1.10.0
API version: 1.22
Go version: go1.5.3
Git commit: 590d5108
Built: Thu Feb 4 18:18:11 2016
OS/Arch: darwin/amd64
Server:
Version: 1.10.0
API version: 1.22
Go version: go1.5.3
Git commit: 590d5108
Built: Thu Feb 4 19:55:25 2016
OS/Arch: linux/amd64
docker info
:
Containers: 7
Running: 0
Paused: 0
Stopped: 7
Images: 210
Server Version: 1.10.0
Storage Driver: aufs
Root Dir: /mnt/sda1/var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 238
Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Plugins:
Volume: local
Network: host bridge null
Kernel Version: 4.1.17-boot2docker
Operating System: Boot2Docker 1.10.0 (TCL 6.4.1); master : b09ed60 - Thu Feb 4 20:16:08 UTC 2016
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 7.79 GiB
Name: default
ID: GZCU:NH4C:QNFU:NQQB:KB55:ULRS:HKZW:AMMR:PSBI:FMFG:SMV4:S2JM
Debug mode (server): true
File Descriptors: 11
Goroutines: 22
System Time: 2016-02-16T18:01:35.563863286Z
EventsListeners: 0
Init SHA1:
Init Path: /usr/local/bin/docker
Docker Root Dir: /mnt/sda1/var/lib/docker
Labels:
provider=virtualbox
docker-machine version
:
docker-machine version 0.6.0, build e27fb87
uname -a
:
Darwin mac.local 15.3.0 Darwin Kernel Version 15.3.0: Thu Dec 10 18:40:58 PST 2015; root:xnu-3248.30.4~1/RELEASE_X86_64 x86_64
Environment details (AWS, VirtualBox, physical, etc.): physical laptop, macbook pro
How reproducible:
Basically put your host on a network with a subnet of 192.168.99, and try to create a dockermachine vm with the defaults. Or try to create a virtualbox machine vm with the --virtualbox-hostonly-cidr set to your router's CIDR.
Steps to Reproduce:
- determine the CIDR of your host's network. (use an online calculator, or whatever). Let's say it's 192.168.0.1/24
- run
docker-machine create --driver virtualbox --virtualbox-hostonly-cidr "192.168.0.1/24" cidrfail
Actual Results:
The create command will print the warning:
Error creating machine: Error checking the host: Error checking and/or regenerating the certs: There was an error validating certificates for host "192.168.0.100:2376": dial tcp 192.168.0.100:2376: connect: no route to host
N.B: with default values, you won't see this error because the quickstart terminal hides it.
docker-machine ls will print the following:
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
cidrfail - virtualbox Running tcp://192.168.0.100:2376 Unknown Unable to query docker version: Get https://192.168.0.100:2376/v1.15/version: dial tcp 192.168.0.100:2376: connect: host is down
default * virtualbox Running tcp://192.168.200.100:2376 v1.10.0
If you're using the default vboxnet0 CIDR, the URL will actually be tcp://192.168.99.100:2376
Additional Info:
For the default case, you can see the lack of route to the vboxnet0 interface in the routing table. It looks like virtualbox maybe creates a second route to 192.168.99.1, but assigns it to interface en0 rather than vboxnet0
Internet:
Destination Gateway Flags Refs Use Netif Expire
default 192.168.99.1 UGSc 46 0 en0
...
192.168.99 link#4 UCS 9 0 en0
192.168.99.1/32 link#4 UCS 2 0 en0
192.168.99.1 dc:9f:db:2b:89:39 UHLWIir 46 61 en0 1197
192.168.99.100 link#4 UHLWIi 1 3 en0
Activity
paul-callahan commentedon Feb 16, 2016
#dibs if accepted.
Proposed fix: raise an error if docker-machine attempts to create or start a vm that has a CIDR conflict with the host.
I've tested a bit of code that will will compare all the host's non-loopback, up-state interfaces with the CIDR docker-machine wishes to use. If the CIDR conflicts, it will raise an error before it attempts to create/launch the machine.
jnardone commentedon Mar 2, 2016
How do I change the default network for the host-only network? Is there any place I can globally override the use of 192.168.99.x?
paul-callahan commentedon Mar 2, 2016
@jnardone use --virtualbox-hostonly-cidr "10.10.10.1/24" (replace 10.10.10.1/24 with whatever you want for the subnet.)
Or after the vm has been created, bring up the virtual box UI and change it in the network preferences.