Container network architecture and performance analysis of Macvlan and IPvlan

: Macvlan and IPvlan modes allow the host physical Network Interface Card (NIC) to create several virtual sub interfaces, which can meet the containers communication requirements across different network segment in complex application scenarios. The paper describes the architecture characteristics of these two modes in depth and builds container overlay network in single machine and multi machine environments. Around the key network indicators such as round-trip time (RTT), bandwidth and jitter, Ping and Iperf tools are used to test and analyse the end-to-end container communication performance. The experimental results show that IPvlan mode has better network performance and provides a comprehensive and rich application scenario for the container overlay network.


Introduction
Docker is the most popular lightweight virtualization container platform at present.Because Docker shares the host operating system, it greatly saves system resources such as CPU, memory and storage, and saves license and OS patch maintenance costs.Compared with traditional virtual machines, Docker has many advantages, such as less resources, fast startup and convenient migration.More and more applications are running in the form of container in the development, testing, and production environment [1][2].
However, Docker technology is not perfect, and container network has certain weaknesses.The background of modern application architecture is becoming more and more complex, and service provision often requires cooperation with multiple containers.How to build a secure and high-performance container network is the core problem to be solved in the industry.IPvlan and Macvlan are new modes of container overlay network [3], several virtual sub interfaces are allowed to be created from the host physical Network Interface Card (NIC) to interconnect the container, so that it is regarded as equal status of the container and the host in the physical network.Container can communicate with each other in Layer 2 and Layer 3 network between different hosts.
This paper constructs container overlay network environment for Macvlan and IPvlan within multi hosts, and uses Alpine and Iperf official container images to implement experimental testing and analysis of end-to-end container network transmission performance.IPvlan not only breaks through many limitations of Macvlan network in application, but also has excellent network transmission performance across different network segments, providing useful reference for building high-performance and production level container network architecture in the future.The remainder of this paper is organized as follows.Section 2 introduces the related work.Section 3 describes container network architecture of Macvlan and IPvlan.Section 4 shows the experimental performance test and result analysis of Macvlan and IPvlan.Section 5 summarizes our work.

Related work
Docker containers are used in large-scale deployment, elastic scaling and other application scenarios.The degree of dispersion of container networks is significantly higher than that of traditional virtual machine network [4].Management and control have become a difficult problem in large-scale container cluster deployment.Zhang weiqi [5] realized flexible adaptation and migration of container network based on Macvlan.Yang xin et al [6] pointed out that using the Macvlan network can solve the problem of docker network port conflict in the mesos cluster.Li wei et al [7] carried out experiments to verify that Macvlan container network does not cause significant loss of network transmission performance.Feng mingzhen [8] designed and implemented Docker container network system based on Macvlan, and the entire system can be seamlessly integrated with Docker daemon.However, Ipvlan is regarded as the upgrade iteration technology of Macvlan and there are few relevant references on IPvlan except the Docker official website.Therefore, research on the characteristics and performance of IPvlan and Macvlan can provide a better solution for container interconnection in complex networks.

Network architecture of IPvlan
Because the application scenarios of Macvlan container network are subject to many restrictions, the new mode IPvlan network is generated iteratively to improve the performance and security of container network communication.
IPvlan requires at least 4.2 Linux kernel, and allows the host NIC to create several virtual sub interfaces to connect internal container of IPvlan network.All containers have the same Mac address.IPvlan network does not receive multicast or broadcast messages, and all ARP processes or multicast messages on the network are completed at the host NIC [12].
The working principle of the Layer 2 mode of the IPvlan is similar to that of the Bridge mode of the Macvlan.The Layer 3 mode of the IPvlan supports the message forwarding across network segments within current host.If the routing rule to reach the IPvlan container is configured on the external router, the IPvlan network can realize communication between the container and the external network.4 Experimental performance test and analysis

Experimental environment and test scheme
The experimental environment involves three host nodes.Node1 and Node2 install the Docker engine and create new container overlay network of Macvlan and IPvlan.Node3 creates two sub interfaces of the host NIC as the corresponding gateway of the Macvlan network respectively, and simultaneously enables routing and forwarding function.The node configuration information is shown in Table 1.The measurement of network performance includes availability, round-trip time (RTT), utilization, throughput and bandwidth.Ping command is an Internet packet explorer used to test network connectivity.Iperf is an open source tool developed by the University of Illinois to test network performance.Client and server modes are enabled at both ends of the test, TCP connection is tested bandwidth and transmitted data, UDP connection is tested jitter and packet loss rate [13][14][15].For the two types of Macvlan and IPvlan container networks, our experimental test scheme is as follows: Ping command is used to test RTT, and Iperf tool is used to test bandwidth, throughput, jitter, packet loss rate and other network performance between end-to-end containers.
In our experimental test, the source container is set to send 100 ping request packets to the target container each time to get the RTT average, and the payload data size increases from 32 bytes to 32K bytes.The test parameters of Iperf are set as follows: the read/write buffer length option is the maximum allowed value, namely 63KB (UDP) and 1MB (TCP), Maximum Segment Size (MSS) of the transport Layer is set to 1460 bytes, and the numbers of concurrent threads is 1, 2, 4, 8, 16, and 32 in turn.One container runs the server side and the other container runs the client side.It outputs data communication statistics of the sending container and the receiving container within 10 seconds

Test results and analysis
For RTT, bandwidth, UDP jitter and other test dimensions, our experiment gives a comparison of the container network performance of Macvlan and IPvlan across different network segment as shown in Figure3-5.
The experimental results show that the transmission performance of IPvlan container network is significantly better than that of Macvlan container network.Macvlan relies on two sub interfaces of the third host to complete the route forwarding between containers across different network segments, which will inevitably increase the communication delay between end-to-end containers.In the Layer 3 mode of IPvlan, a router is added to current host to forward the routing message of each endpoint.As long as the parent interface is the    IPvlan is a new twist on the tried and true network virtualization technique.The Linux implementations are extremely lightweight because rather than using the traditional Linux bridge for isolation, they are associated to a Linux Ethernet interface or sub-interface to enforce separation between networks and connectivity to the physical network.
For the following container application scenarios in the production environment, IPvlan should be used instead of Macvlan: Physical host NIC works on IEEE 802.11 wireless network protocol; (4) Constructing complex network topology, etc.

Conclusion
Macvlan is the main mode for multi host container overlay network.Without the help of traditional Linux virtual bridge devices, Macvlan can achieve nearly the lossless transmission performance of the local host.However, Macvlan is limited by port binding, the numbers of Mac addresses, and the lack of support for wireless networks.New mode IPvlan can better solve such problems instead of Macvlan, enabling administrator to fully control Layer 2 network, and even Layer3 network and has excellent network performance test across different network segment.Deeply exploring the special function of IPvlan and carrying out innovative application research on multi container network to better realize the system integration of user network, which will be our future research direction.
Figure.2 shows that different IPvlan network container can communicate with each other within the current host.

Fig. 4 .
Fig. 4. Bandwidth test of container across different network segment.

( 1 )
The uplink switch of the host NIC has set the binding between the current port and the designed Mac address; (2) The uplink switch of the host NIC has limited the numbers of Mac addresses associated with current port; (3) https://doi.org/10.1051/shsconf/202316601072SHS Web of Conferences 166, 01072 (2023) EIMM 2022

Table 1 .
Node configuration information.