systemd and udev in their default configuration attempt to set a
persistent MAC address for network interfaces that don't have one
already [systemd-def-link]. We set the address only after creating the
interface, so there is a race between us and udev. There are several
outcomes (that actually occur, this race is very much not a theoretical
one):
* We set the address before udev gets to the networking rules, so udev
sees `/sys/devices/virtual/net/docker0/addr_assign_type = 3`
(NET_ADDR_SET). This means there's no need to assign a different
address and everything is fine.
* udev reads `/sys/devices/virtual/net/docker0/addr_assign_type` before
we set the address, gets `1` (NET_ADDR_RANDOM), and proceeds to
generate and set a persistent address.
Old versions of udev (pre-v242, i.e. without [udev-patch]) would then
fail to generate an address, spit out "Could not generate persistent
MAC address for docker0: No such file or directory" (see [udev-issue],
and everything would be probably fine as well.
Current version of udev (with [udev-patch]) will generate an address
just fine and then race us setting it. As udev does more work than we,
the most probable outcome is that udev will overwrite the address we
set and possibly cause some trouble later on.
On a clean Debian Buster (from Vagrant) VM with systemd/udev 242 from
Debian Experimental, `docker network create net1` up to `net7` resulted
in 3 bridges having a 02:42: address and 4 bridges having a seemingly
random (actually generated from interface name) address. With systemd
241, the result would be all bridges having a 02:42:, but some "Could
not generate persistent MAC address for" messages in the log.
The fix is to revert the MAC address setting fix from 6901ea51dc,
as it is no longer necessary with current netlink [netlink-addr-add],
and set the address atomically when creating the bridge interface, not
after that.
[systemd-def-link]: a166cd3aac/network/99-default.link
[udev-patch]: 6d36464065
[udev-issue]: https://github.com/systemd/systemd/issues/3374
[netlink-addr-add]: 7d9b424492
...
Do note that a similar race happens when creating veth devices as well.
I wasn't able to reproduce getting a wrong (non-02:42:) address,
possibly because the address is set by docker later, maybe only after
the interface is moved to another network namespace (but I'm just
guessing here). Still, different timings result in various error
messages being logged ("link_config: could not get ethtool features for
vethd9c938e" and the like) depending on when the interface disappears
from the primary network namespace. I'm not sure how to fix this and I
don't intend to dig deeper into this.
Signed-off-by: Tomas Janousek <tomi@nomi.cz>
- DisableBridgeCreation is misleading. change it to DefaultBridge
- Dont fail the init if localstore cannot be initialized
- added a convenience function to get endpoint for a container
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Replaced it with DisableBridgeCreation and it can be used ONLY in
a special case for docker0 bridge from docker, instead of calling it
from all other case.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
As seen in https://github.com/docker/docker/issues/14738 there is
general instability in the later kernels under race conditions when ioctl
calls are used in parallel with netlink calls for various operations.
(We are yet to narrow down to the exact root-cause on the kernel).
For those older kernels which doesnt support some of the netlink APIs,
we can fallback to using ioctl calls. Hence bringing back the original
code that used netlink (https://github.com/docker/libnetwork/pull/349).
Also, there was an existing bug in bridge creation using netlink which
was setting bridge mac during bridge creation. That operation is not
supported in the netlink library (and doesnt throw an error either).
Included a fix for that condition by setting the bridge mac after
creating the bridge.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
The netlink way of creating bridge has problems in older
kernels like the one used on RHEL 6 (which is a supported
one). So trying to use ioctl method to create bridge
so that it works on any version.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
- Also unexporting configuration structures in bridge
- Changes in dnet/network.go to set bridge name = network name
Signed-off-by: Alessandro Boch <aboch@docker.com>
types, except the naked error returns which were just prefixing
strings to previously returned error strings.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
- Added api enhancement to pass driver specific config
- Refactored simple bridge driver code for driver specific config
- Added an undocumented option to add non-default bridges without
manual pre-provisioning to help libnetwork testing
- Reenabled libnetwork test to do api testing
- Updated README.md
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
- Move ipallocator package into libnetwork
- Also ported network utility functions and their tests in libnetwork:
docker/daemon/networkdriver/utilg.go => libnetwork/utils.go
docker/daemon/networkdriver/network_test.go => libnetwork/utils_test.go
- Changed drivers/setup_device.go and setup_ipv4.go to reuse functions in
utils.go, instead of redefining internally.
- Modified utils to use vishvananda/netlink instead of libcontainer/netlink
Signed-off-by: Alessandro Boch <aboch@socketplane.io>
Tomas Janousek