通过OpenvSwitch构建虚拟网络
目录
环境准备
VMware+centos7克隆多个虚拟机 : https://www.cnblogs.com/Lynette/p/9470800.html
使用VMware Workstation软件创建两个主机ovs1和ovs2,内存为2G,全部配置双网卡。
ovs1:
网卡1:10.0.0.150(NAT) ;网卡2:10.0.1.150(仅主机)
ovs2:
网卡1:10.0.0.151 (NAT);网卡2:10.0.1.151(仅主机)
进入主机配置启动网卡,用远程连接软件SecureCRT.exe连接虚拟机。查看两台主机的网卡信息。
openvswitch1第2块网卡:
TYPE=Ethernet BOOTPROTO=static NAME=eno33554960 #UUID=3c145aba-af74-4b62-bebe-7c2c5480248a DEVICE=eno33554960 ONBOOT=yes IPADDR=10.0.1.150 NETMASK=255.255.255.0 #GATEWAY=10.0.1.2
openvswitch2第2块网卡:
TYPE=Ethernet BOOTPROTO=static NAME=ens34 #UUID=3c145aba-af74-4b62-bebe-7c2c5480248a DEVICE=ens34 ONBOOT=yes IPADDR=10.0.1.151 NETMASK=255.255.255.0 #GATEWAY=10.0.1.2 HWADDR=00:50:56:2c:aa:f6
主机ovs1和主机ovs2两个网段互ping一下,看看能否ping通。
启动OpenvSwitch
主机ovs1
启动OpenvSwitch服务,并查看OpenvSwitch网络详情
[root@openvswitch1 ~]# ovs-vsctl show 6ae4acad-591f-4c10-beae-dcdf16e0ea30 ovs_version: "2.1.3"
主机ovs2
[root@openvswitch2 ~]# ovs-vsctl show 760a61c1-8794-4570-901e-c227e0828d7c ovs_version: "2.1.3"
此时一个软交换机已经诞生。他比brctl强大,多了一个VLan的功能。
绑定网桥
主机ovs1
在主机openvswitch1上创建网桥br0和br1,并将物理网卡eth0挂接到网桥br0上
[root@openvswitch1 ~]# ovs-vsctl add-br br0
[root@openvswitch1 ~]# ovs-vsctl list-br
br0
[root@openvswitch1 ~]# ovs-vsctl add-port br0 eno33554960
[root@ovs1 ~]# ovs-vsctl show
27727e54-922e-4735-93f4-0e59fef5d5fa Bridge "br0" Port "eno33554960" Interface "eno33554960" Port "br0" Interface "br0" type: internal ovs_version: "2.3.2"
创建br0的配置文件:
[root@ovs1 network-scripts]# ifconfig eno33554960 0
[root@ovs1 network-scripts]# ifconfig br0 10.0.1.150 netmask 255.255.255.0
[root@ovs1 network-scripts]# route add default gw 10.0.1.2 dev br0
[root@localhost ~]# cd /etc/sysconfig/network-scripts
[root@localhost network-scripts]# cp -rfv ifcfg-eno33554960 ifcfg-br0
‘ifcfg-eno33554960’ -> ‘ifcfg-br0’
[root@localhost network-scripts]# vi ifcfg-eno33554960
TYPE=Ethernet BOOTPROTO=none NAME=eno33554960 #UUID=3c145aba-af74-4b62-bebe-7c2c5480248a DEVICE=eno33554960 ONBOOT=yes #IPADDR=10.0.1.150 #NETMASK=255.255.255.0 #GATEWAY=10.0.1.2
修改配置文件ifcfg-br0
vi ifcfg-br0
TYPE=Bridge BOOTPROTO=static NAME=br0 #UUID=3c145aba-af74-4b62-bebe-7c2c5480248a DEVICE=br0 ONBOOT=yes IPADDR=10.0.1.150 NETMASK=255.255.255.0 #GATEWAY=10.0.1.2
~ [root@localhost network-scripts]# systemctl restart network
主机ovs2
在主机openvswitch2上创建网桥br0,并将物理网卡ens34挂接到网桥br0上
[root@openvswitch2 ~]# ovs-vsctl add-br br0
[root@openvswitch2 ~]# ovs-vsctl list-br
br0
[root@openvswitch2 ~]# ovs-vsctl add-port br0 ens34
[root@ovs2 ~]# ifconfig ens34 0
[root@ovs2 ~]# ifconfig br0 10.0.1.151 netmask 255.255.255.0
[root@ovs2 ~]# route add default gw 10.0.1.2 dev br0
[root@localhost ~]# cd /etc/sysconfig/network-scripts
[[root@ovs2 network-scripts]# cp -rfv ifcfg-ens34 ifcfg-br0
‘ifcfg-ens34’ -> ‘ifcfg-br0’
[root@ovs2 network-scripts]# vi ifcfg-ens34
TYPE=Ethernet BOOTPROTO=none NAME=ens34 #UUID=3c145aba-af74-4b62-bebe-7c2c5480248a DEVICE=ens34 ONBOOT=yes #IPADDR=10.0.1.151 #NETMASK=255.255.255.0 #GATEWAY=10.0.1.2 #HWADDR=00:50:56:2c:aa:f6
修改配置文件ifcfg-br0
vi ifcfg-br0
TYPE=Bridge BOOTPROTO=static NAME=br0 #UUID=3c145aba-af74-4b62-bebe-7c2c5480248a DEVICE=br0 ONBOOT=yes IPADDR=10.0.1.151 NETMASK=255.255.255.0 #GATEWAY=10.0.1.2 #HWADDR=00:50:56:2c:aa:f6
~ [root@localhost network-scripts]# systemctl restart network
绑定网卡后,主机ovs1和主机ovs2在br0的网段10.0.1.0上互ping一下,看看能否ping通。
建立GRE隧道
主机ovs1
[root@ovs1 ~]# ovs-vsctl show
27727e54-922e-4735-93f4-0e59fef5d5fa Bridge "br0" Port "eno33554960" Interface "eno33554960" Port "br0" Interface "br0" type: internal ovs_version: "2.3.2"
[root@ovs1 ~]# route add default gw 10.0.1.150
[root@ovs1 ~]# ovs-vsctl add-br br1
[root@ovs1 ~]# ifconfig br1 192.168.3.1/24 up
[root@ovs1 ~]# ovs-vsctl show
27727e54-922e-4735-93f4-0e59fef5d5fa Bridge "br0" Port "eno33554960" Interface "eno33554960" Port "br0" Interface "br0" type: internal Bridge "br1" Port "br1" Interface "br1" type: internal ovs_version: "2.3.2"
[root@ovs1 ~]# ovs-vsctl add-port br1 gre1 -- set interface gre1 type=gre option:remote_ip=10.0.1.151
[root@ovs1 ~]# ifconfig
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.1.150 netmask 255.255.255.0 broadcast 10.0.1.255 inet6 fe80::20c:29ff:fe9a:f531 prefixlen 64 scopeid 0x20<link> ether 00:0c:29:9a:f5:31 txqueuelen 0 (Ethernet) RX packets 1352 bytes 393758 (384.5 KiB) RX errors 0 dropped 171 overruns 0 frame 0 TX packets 81 bytes 4698 (4.5 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 br1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.3.1 netmask 255.255.255.0 broadcast 192.168.3.255 inet6 fe80::c14:79ff:fe78:1b4e prefixlen 64 scopeid 0x20<link> ether 0e:14:79:78:1b:4e txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 10 bytes 732 (732.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.0.150 netmask 255.255.255.0 broadcast 10.0.0.255 inet6 fe80::20c:29ff:fe9a:f527 prefixlen 64 scopeid 0x20<link> ether 00:0c:29:9a:f5:27 txqueuelen 1000 (Ethernet) RX packets 172445 bytes 234314866 (223.4 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 65468 bytes 5583309 (5.3 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eno33554960: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet6 fe80::20c:29ff:fe9a:f531 prefixlen 64 scopeid 0x20<link> ether 00:0c:29:9a:f5:31 txqueuelen 1000 (Ethernet) RX packets 2784 bytes 810207 (791.2 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 246 bytes 18372 (17.9 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 0 (Local Loopback) RX packets 119 bytes 11274 (11.0 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 119 bytes 11274 (11.0 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 inet 192.168.122.1 netmask 255.255.255.0 broadcast 192.168.122.255 ether 00:00:00:00:00:00 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@ovs1 ~]# ovs-vsctl show
27727e54-922e-4735-93f4-0e59fef5d5fa Bridge "br0" Port "eno33554960" Interface "eno33554960" Port "br0" Interface "br0" type: internal Bridge "br1" Port "gre1" Interface "gre1" type: gre options: {remote_ip="10.0.1.151"} Port "br1" Interface "br1" type: internal ovs_version: "2.3.2"
主机ovs2
[root@ovs2 ~]# ovs-vsctl show
27727e54-922e-4735-93f4-0e59fef5d5fa Bridge "br0" Port "ens34" Interface "ens34" Port "br0" Interface "br0" type: internal ovs_version: "2.3.2"
[root@ovs2 ~]# route add default gw 10.0.1.151
[root@ovs2 ~]# ovs-vsctl add-br br1
[root@ovs2 ~]# ifconfig br1 192.168.4.1/24 up
[root@ovs2 ~]# ovs-vsctl show
27727e54-922e-4735-93f4-0e59fef5d5fa Bridge "br1" Port "br1" Interface "br1" type: internal Bridge "br0" Port "ens34" Interface "ens34" Port "br0" Interface "br0" type: internal ovs_version: "2.3.2"
[root@ovs2 ~]# ovs-vsctl add-port br1 gre1 -- set interface gre1 type=gre option:remote_ip=10.0.1.150
[root@ovs2 ~]# ifconfig
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.1.151 netmask 255.255.255.0 broadcast 10.0.1.255 inet6 fe80::250:56ff:fe2c:aaf6 prefixlen 64 scopeid 0x20<link> ether 00:50:56:2c:aa:f6 txqueuelen 0 (Ethernet) RX packets 765 bytes 224487 (219.2 KiB) RX errors 0 dropped 3 overruns 0 frame 0 TX packets 120 bytes 6760 (6.6 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 br1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.4.1 netmask 255.255.255.0 broadcast 192.168.4.255 inet6 fe80::94eb:8dff:fe9c:3945 prefixlen 64 scopeid 0x20<link> ether 96:eb:8d:9c:39:45 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 10 bytes 732 (732.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.0.151 netmask 255.255.255.0 broadcast 10.0.0.255 inet6 fe80::250:56ff:fe2c:7eba prefixlen 64 scopeid 0x20<link> ether 00:50:56:2c:7e:ba txqueuelen 1000 (Ethernet) RX packets 8494 bytes 653792 (638.4 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 2996 bytes 542858 (530.1 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens34: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet6 fe80::250:56ff:fe2c:aaf6 prefixlen 64 scopeid 0x20<link> ether 00:50:56:2c:aa:f6 txqueuelen 1000 (Ethernet) RX packets 1482 bytes 388939 (379.8 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 545 bytes 81776 (79.8 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 0 (Local Loopback) RX packets 109 bytes 9832 (9.6 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 109 bytes 9832 (9.6 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 inet 192.168.122.1 netmask 255.255.255.0 broadcast 192.168.122.255 ether 00:00:00:00:00:00 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@ovs2 ~]# ovs-vsctl show
27727e54-922e-4735-93f4-0e59fef5d5fa Bridge "br1" Port "br1" Interface "br1" type: internal Port "gre1" Interface "gre1" type: gre options: {remote_ip="10.0.1.150"} Bridge "br0" Port "ens34" Interface "ens34" Port "br0" Interface "br0" type: internal ovs_version: "2.3.2"
联通测试
最后互ping可发现在设置隧道隧道以前只有两机的BR0可以互相通信而br1不行,但是创建了gre隧道之后就可以互相通信了:
[root@ovs1 ~]# ping 192.168.4.1
PING 192.168.4.1 (192.168.4.1) 56(84) bytes of data. 64 bytes from 192.168.4.1: icmp_seq=1 ttl=64 time=1.09 ms 64 bytes from 192.168.4.1: icmp_seq=2 ttl=64 time=0.595 ms 64 bytes from 192.168.4.1: icmp_seq=3 ttl=64 time=0.581 ms ^C --- 192.168.4.1 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 0.581/0.758/1.098/0.240 ms
[root@ovs2 ~]# ping 192.168.3.1
PING 192.168.3.1 (192.168.3.1) 56(84) bytes of data. 64 bytes from 192.168.3.1: icmp_seq=1 ttl=64 time=1.31 ms 64 bytes from 192.168.3.1: icmp_seq=2 ttl=64 time=0.307 ms 64 bytes from 192.168.3.1: icmp_seq=3 ttl=64 time=0.644 ms 64 bytes from 192.168.3.1: icmp_seq=4 ttl=64 time=0.875 ms 64 bytes from 192.168.3.1: icmp_seq=5 ttl=64 time=0.431 ms ^C --- 192.168.3.1 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4005ms rtt min/avg/max/mdev = 0.307/0.714/1.314/0.357 ms [root@ovs2 ~]#
如果网桥上接上主机那么这两主机就可以互相通信.
参考文档:
https://blog.csdn.net/a1164520408/article/details/81836077
安装kvm工具
主机ovs1
[root@openvswitch1 ~]# yum install qemu-kvm libvirt –y [root@openvswitch1 ~]#lsmod |grep kvm kvm_intel 162153 0 kvm 525259 1 kvm_intel [root@openvswitch1 ~]#service libvirtd start Redirecting to /bin/systemctl start libvirtd.service
主机ovs2
[root@openvswitch2 ~]# yum install qemu-kvm libvirt –y [root@openvswitch2 ~]#lsmod |grep kvm kvm_intel 162153 0 kvm 525259 1 kvm_intel [root@openvswitch2 ~]#service libvirtd start Redirecting to /bin/systemctl start libvirtd.service
创建虚拟机
参考文档: https://blog.csdn.net/Ghost_leader/article/details/71123484
整合Open vSwitch与DNSmasq为虚拟机提供DHCP功能: https://www.cnblogs.com/CasonChan/p/4604871.html
DNS:https://blog.csdn.net/yanghua1012/article/details/80555487
主机ovs1
在主机openvswitch1上创建一个KVM虚拟机kvm1,将虚拟机kvm1的网卡桥接在网卡br1上,将br1的子网网段设置为192.168.3.0/24
[root@ovs1 network-scripts]# vi /etc/ovs-ifup #!/bin/sh switch='br1' /sbin/ifconfig $1 up ovs-vsctl add-port ${switch} $1 [root@ovs1 network-scripts]# ip route add 192.168.4.0/24 via 10.0.1.151 dev br0 [root@ovs1 network-scripts]# systemctl stop firewalld [root@ovs1 network-scripts]# systemctl disable firewalld Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service. [root@ovs1 network-scripts]# systemctl disable iptables Failed to execute operation: No such file or directory [root@ovs1 network-scripts]# sed -i 's/SELINUX=.*/SELINUX=permissive/g' /etc/selinux/config [root@ovs1 network-scripts]# setenforce 0 [root@ovs1 network-scripts]# virsh -c qemu:///system list Id Name State ---------------------------------------------------- [root@ovs1 network-scripts]# ln -s /usr/libexec/qemu-kvm /usr/bin/qemu-kvm [root@ovs1 network-scripts]# chmod +x ovs-ifup [root@ovs1 network-scripts]# /usr/bin/qemu-kvm -m 256 -smp 1 -drive file=cirros-0.3.0-x86_64-disk.img,media=disk,if=virtio -net nic,model=virtio,macaddr=52:54:00:00:00:01 -net tap,ifname=vif0.0,script=/etc/ovs-ifup -nographic -name kvm1
虚拟机kvm1:
$ sudo ifconfig eth0 192.168.3.3 netmask 255.255.255.0 up $ ifconfig eth0 Link encap:Ethernet HWaddr 52:54:00:00:00:01 inet addr:192.168.3.3 Bcast:192.168.3.255 Mask:255.255.255.0 inet6 addr: fe80::5054:ff:fe00:1/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:7 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:578 (578.0 B) TX bytes:1364 (1.3 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) $ sudo route add default gw 192.168.3.1 $ route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface default 192.168.3.1 0.0.0.0 UG 0 0 0 eth0 192.168.3.0 * 255.255.255.0 U 0 0 0 eth0
</nowiki>
主机ovs2
在主机openvswitch2上创建一个KVM虚拟机kvm2,将虚拟机kvm2的网卡桥接在网桥br1上,将br1的子网网段设置为192.168.4.0/24
[root@ovs2 ~]# vi /etc/ovs-ifup #!/bin/sh switch='br1' /sbin/ifconfig $1 up ovs-vsctl add-port ${switch} $1 [root@ovs2 ~]# ip route add 192.168.3.0/24 via 10.0.1.150 dev br0 [root@ovs1 network-scripts]# systemctl stop firewalld [root@ovs1 network-scripts]# systemctl disable firewalld Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service. [root@ovs1 network-scripts]# systemctl disable iptables Failed to execute operation: No such file or directory [root@ovs1 network-scripts]# sed -i 's/SELINUX=.*/SELINUX=permissive/g' /etc/selinux/config [root@ovs1 network-scripts]# setenforce 0 [root@ovs1 network-scripts]# virsh -c qemu:///system list Id Name State ---------------------------------------------------- [root@ovs1 network-scripts]# ln -s /usr/libexec/qemu-kvm /usr/bin/qemu-kvm [root@ovs2 ~]# chmod +x /etc/ovs-ifup [root@ovs1 network-scripts]# /usr/bin/qemu-kvm -m 256 -smp 1 -drive file=cirros-0.3.0-x86_64-disk.img,media=disk,if=virtio -net nic,model=virtio,macaddr=52:54:00:00:00:02 -net tap,ifname=vif1.0,script=/etc/ovs-ifup -nographic -name kvm2
虚拟机kvm2:
$ sudo ifconfig eth0 192.168.4.3 netmask 255.255.255.0 up $ ifconfig eth0 Link encap:Ethernet HWaddr 52:54:00:00:00:02 inet addr:192.168.4.3 Bcast:192.168.4.255 Mask:255.255.255.0 inet6 addr: fe80::5054:ff:fe00:2/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:7 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:578 (578.0 B) TX bytes:1364 (1.3 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) $ sudo route add default gw 192.168.4.1 $ route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface default 192.168.4.1 0.0.0.0 UG 0 0 0 eth0 192.168.4.0 * 255.255.255.0 U 0 0 0 eth0
联通测试
[root@ openvswitch1 ~]# virsh console kvm1 [root@ localhost ~]#ping 192.168.4.3 -c 4 64 bytes from 192.168.4.3: icmp_seq=1 ttl=63 time=0.297 ms 64 bytes from 192.168.4.3: icmp_seq=2 ttl=63 time=0.225 ms 64 bytes from 192.168.4.3: icmp_seq=3 ttl=63 time=0.188 ms 64 bytes from 192.168.4.3: icmp_seq=4 ttl=63 time=0.208 ms