作者: 趙怡
一、VLAN aware VMs簡介
"VLAN aware VMs"有時也叫做"VM trunk ports", 主要是讓虛擬機收發的vlan tagged報文, 能夠被虛擬網絡所識別和處理。
二、VLAN aware VMs應用場景
該功能的主要使用場景有:
1. 有些應用程序, 需要連接很多neutron網絡。傳統的方法是虛擬機上連接很多neutron網絡, 但是這樣做顯然不好維護, 並不實用。比較好的辦法是在虛擬機上使用1個VIF網卡接口，創建多個VLAN接口, 並通過收發不同VLAN報文來區分不同網絡。
2. 雲計算網絡中的負載較爲動態, 有時需要動態的增刪虛擬機上的網絡接口。通過增刪vlan相對來說更加簡單和快捷。
3. 通過配置VLAN, 可以不用增刪VIF, 把VM從一個網絡挪到另一個網絡
4. 一個VM可能跑多個容器, 通過給容器分配VLAN來連接到不同的neutron網絡, 比給每個容器分配一個VM的VIF, 更加有效和更好的可擴展性。
5. 有些老的應用程序以及部分NFV, 要求能夠通過VLAN來連接網絡, 本功能能夠滿足這個需求。
三、VLAN aware VMs配置和使用方法
配置
需要在網絡節點的neutron.conf裏配置service_plugins爲: "trunk"。
|
service_plugins=...., trunk |
使用
例如, 我們將要創建一個虛擬機trunk-vm1, 虛擬機上希望有3個網絡接口: eth0, VLAN接口: eth0.1, VLAN接口: eth0.2。 其中：
nova創建虛擬機時創建的eth0, 主要用來收發VLANuntagged報文, 所屬net0網絡(子網爲:10.0.4.0/24)。 對應的neutron port被稱爲parent_port, 取名爲: trunkparent。
手動在VM裏創建的VLAN接口: eth0.1, 用來收發VLAN tag=1的報文, 所屬net1網絡(子網爲: 10.0.5.0/24)。對應的neutron port被稱爲subport, 取名爲: subport1。
手動在VM裏創建的VLAN接口: eth0.2, 用來收發VLAN tag=2的報文, 所屬net2網絡(子網爲: 10.0.6.0/24)。對應的neutron port被稱爲subport, 取名爲: subport2。
1. 創建網絡net0, net1, net2, 以及對應的子網: subnet0 (10.0.4.0/24),subnet1 (10.0.5.0/24), subnet2 (10.0.6.0/24)
|
# openstack network create net0 # openstack network create net1 # openstack network create net2 # openstack subnet create --network net0 --subnet-range 10.0.4.0/24 subnet0 # openstack subnet create --network net1 --subnet-range 10.0.5.0/24 subnet1 # openstack subnet create --network net2 --subnet-range 10.0.6.0/24 subnet2 |
2. 在net0上創建trunkparentport, 在net1上創建subport1, 在net2上創建subport2
|
# openstack port create --network net0 trunkparent # openstack port create --network net1 subport1 # openstack port create --network net2 subport2 |
3. 創建mytrunk, 並設置parent-port爲: trunkparent。subport 1的vlan_id=1。 subport2的vlan_id=2。
|
# openstack network trunk create --parent-port trunkparent \ --subport port=subport1,segmentation-type=vlan,segmentation-id=1 \ --subport port=subport2,segmentation-type=vlan,segmentation-id=2 mytrunk
+-----------------+-----------------------------------------------------------------------------------------------+ | Field | Value | +-----------------+-----------------------------------------------------------------------------------------------+ | admin_state_up | UP | | created_at | 2016-11-15T01:58:13Z | | description | | | id | a5f0be95-67bd-4dea-b321-b4cad753b91d | | name | mytrunk | | port_id | b7d4c968-afb2-4709-9f64-0e7853209673 | | revision_number | 1 | | status | DOWN | | sub_ports | port_id='c7dd9ac2-730a-4476-b93f-08a6f31367d4', segmentation_id='1', segmentation_type='vlan' | | | port_id='3ba8b2d5-fe03-4d2a-bcd7-abeee2121739', segmentation_id='2', segmentation_type='vlan' | | tenant_id | f332909670c841abb78c2e30e8c50c68 | | updated_at | 2016-11-15T01:58:13Z | +-----------------+-----------------------------------------------------------------------------------------------+ |
4. 檢查當前創建的trunk 以及相關subport信息
|
# openstack network trunk list +--------------------------------------+---------+--------------------------------------+-------------+ | ID | Name | Parent Port | Description | +--------------------------------------+---------+--------------------------------------+-------------+ | a5f0be95-67bd-4dea-b321-b4cad753b91d | mytrunk | b7d4c968-afb2-4709-9f64-0e7853209673 | | +--------------------------------------+---------+--------------------------------------+-------------+
# openstack network trunk show mytrunk +-----------------+-----------------------------------------------------------------------------------------------+ | Field | Value | +-----------------+-----------------------------------------------------------------------------------------------+ | admin_state_up | UP | | created_at | 2016-11-15T01:58:13Z | | description | | | id | a5f0be95-67bd-4dea-b321-b4cad753b91d | | name | mytrunk | | port_id | b7d4c968-afb2-4709-9f64-0e7853209673 | | revision_number | 1 | | status | DOWN | | sub_ports | port_id='c7dd9ac2-730a-4476-b93f-08a6f31367d4', segmentation_id='1', segmentation_type='vlan' | | | port_id='3ba8b2d5-fe03-4d2a-bcd7-abeee2121739', segmentation_id='2', segmentation_type='vlan' | | tenant_id | f332909670c841abb78c2e30e8c50c68 | | updated_at | 2016-11-15T01:58:13Z | +-----------------+-----------------------------------------------------------------------------------------------+ |
5. 查詢subport1的mac地址 (mac地址爲: fa:16:3e:cc:b9:27), 以及subport2的mac地址(mac地址爲: fa:16:3e:25:d2:c9)
|
# openstack port show subport1 +-----------------------+--------------------------------------+ | Field | Value | +-----------------------+--------------------------------------+ | id | c7dd9ac2-730a-4476-b93f-08a6f31367d4 | | mac_address | fa:16:3e:cc:b9:27 | | name | subport1 | ... +-----------------------+--------------------------------------+
# openstack port show subport2 +-----------------------+--------------------------------------+ | Field | Value | +-----------------------+--------------------------------------+ | id | 3ba8b2d5-fe03-4d2a-bcd7-abeee2121739 | | mac_address | fa:16:3e:25:d2:c9 | | name | subport2 | ... +-----------------------+--------------------------------------+ |
6. 創建虛擬機, 網卡設置爲parent_port:trunkparent
|
# openstack server create --image 5c43aefd-7256-4fd8-b641-72a5a0945666 --flavor 3--nic port-id=trunkparent trunk-vm1 |
虛擬機創建成功後, parent_port會綁定成功, 底層trunk設備也會被創建。
7. 虛擬機中配置vlan接口, 並根據subport1/subport2的mac地址, 配置VLAN接口。
subport1的vlan id爲1, mac地址爲: fa:16:3e:cc:b9:27
subport2的vlan id爲2, mac地址爲: fa:16:3e:25:d2:c9
|
VM# sudo ip link add link eth0 eth0.1 address fa:16:3e:cc:b9:27 broadcast ff:ff:ff:ff:ff:ff type vlan id 1 VM# sudo ip link add link eth0 eth0.2 address fa:16:3e:25:d2:c9 broadcast ff:ff:ff:ff:ff:ff type vlan id 2
VM# sudo ip link set eth0.1 up VM# sudo ip link set eth0.2 up |
8. 通過執行對接口eth0.1和eth0.2上執行dhclient, 可以使eth0.1, eth0.2得到對應虛擬網絡的ip地址。
|
VM# sudo dhclient eth0.1 VM# sudo dhclient eth0.2 |
9. VM中通過eth0.1 ping dhcp端口來驗證vlan tags。
|
VM# ping -I eth0.1 10.0.5.2
qvo上抓包, 能夠抓到報文帶vlan id=1: # tcpdump -en -i qvob7d4c968-af tcpdump: WARNING: qvob7d4c968-af: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on qvob7d4c968-af, link-type EN10MB (Ethernet), capture size 65535 bytes 17:38:56.411075 fa:16:3e:cc:b9:27 > fa:16:3e:cc:4b:83, ethertype 802.1Q (0x8100), length 102: vlan 1, p 0, ethertype IPv4, 10.0.5.7 > 10.0.5.2: ICMP echo request, id 15784, seq 180, length 64 17:38:56.411123 fa:16:3e:cc:4b:83 > fa:16:3e:cc:b9:27, ethertype 802.1Q (0x8100), length 102: vlan 1, p 0, ethertype IPv4, 10.0.5.2 > 10.0.5.7: ICMP echo reply, id 15784, seq 180, length 64 17:38:57.413020 fa:16:3e:cc:b9:27 > fa:16:3e:cc:4b:83, ethertype 802.1Q (0x8100), length 102: vlan 1, p 0, ethertype IPv4, 10.0.5.7 > 10.0.5.2: ICMP echo request, id 15784, seq 181, length 64 |
四、VLAN aware VMs底層原理
在使用VLAN aware VMs之前, 傳統Openstack VM vif和neutron端口, 網絡之間的關係如下,
虛擬機(openstack instance)通過eth0/eth1/eth2這類vNIC虛擬網卡和計算節點上的port相聯, port再添加到不同虛擬網絡中去進行2層互聯。
傳統Openstack VM vif和neutron端口、網絡之間的關係
在使用了vlan-aware-vms功能後, 虛擬機中的網絡接口將會配成vlan接口方式, 並通過1個vNIC連接到計算節點的parent_port上, parent_port將把不同tagged的報文發送給不同的vlan subport, 並和虛擬網絡相連接。其中parent_port可用來收發untagged VLAN報文, 而subports(child_ports)用來收發VLAN tagged的報文。
如下圖, 這樣可以省去在VM上添加多個網絡端口的動作:
使用了vlan-aware-vms功能後, VM vif和neutron端口、網絡之間的關係
目前, openstack newton中實現了ovs和linuxbridge 兩種trunk的驅動實現方式, 我們看一下OVS的驅動實現原理, linuxbridge實現方式類似。
OVS trunk驅動實現原理:
在使用trunk之前, 我們的ovs 2層網絡和報文走向是這樣子的(左圖): VM中eth0 -> tap -> qbr -> qvb-> qvo -> br-int -> br-tun ...
在使用trunk之後, qvo(trunkport)掛到了新的ovs bridge:tbr- 上, 我們的ovs 2層網絡和報文走向變成了這樣子(右圖)。
eth0(trunk port, 不帶vlan tag), 通過tbr-上的ovs port: tpt (不設置vlan tag), 和tpi (爲patch-port對)相連, tpi最終連接到了br-int上。
eth0.1(subport, 帶vlan:1), 通過tbr-上的ovs port: spi(配置vlan tag: 1), 和spi (爲patch-port對)相連, spi最終連接到了br-int上。
eth0.2同上。
其中ovs bridge: tbr-XXX中下的流表爲: 匹配所有報文,action=NORMAL, 即傳統的2層bridge模式。
qvo等ovs port默認爲trunk port(可收發帶vlan報文, 並且出去後不剝掉vlan tag), 而tpt/tpi爲access port(一般進來和出去爲untagged報文, 進到網橋/交換機後按照端口上的tag配置進行vlan劃分和處理)。
縮寫解釋:
tbr: trunk bridge
tpt: trunk port trunk-end (即和trunk bridge連接的端口)
tpi: trunk port integration-end(即和br-int連接的端口)
spt: subport trunk-end (即和trunk bridge連接的端口)
spi: subport integration end (即和br-int連接的端口)
使用ovs trunk前後, 底層設備和報文走向
Linux trunk驅動實現原理:
通過VM的tap口上創建vlan子接口實現。
例如: parent_port爲tap9965461c-91, subport1爲: tapca4c78e2-66@tap9965461c-91,subport2爲: tapbf54d831-fe@tap9965461c-91。
|
# ip -d link show | grep 9965 -C 6 39: tap9965461c-91: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65486 qdisc pfifo_fast master brq7412a74a-7b state UNKNOWN mode DEFAULT group default qlen 500 link/ether fe:16:3e:90:7a:ca brd ff:ff:ff:ff:ff:ff promiscuity 2 tun 40: tapbf54d831-fe@tap9965461c-91: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65486 qdisc noqueue state UP mode DEFAULT group default link/ether fa:16:3e:d5:55:a5 brd ff:ff:ff:ff:ff:ff promiscuity 0 vlan protocol 802.1Q id 2 <REORDER_HDR> 41: tapca4c78e2-66@tap9965461c-91: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65486 qdisc noqueue state UP mode DEFAULT group default link/ether fa:16:3e:b5:5b:a1 brd ff:ff:ff:ff:ff:ff promiscuity 0 vlan protocol 802.1Q id 1 <REORDER_HDR> |
parent_port所屬網絡爲net0 (uuid: 7412a74a-7b04-4c45-9491-b8929cf1fcf4), 則其對應的linux bridge爲: brq7412a74a-7b
|
# brctl show brq7412a74a-7b bridge name bridge id STP enabled interfaces brq7412a74a-7b 8000.5a63a569d91c no tap8f6a9f8b-8d tap9965461c-91 vxlan-100 |
使用linuxbridge trunk前後, 底層設備和報文走向
代碼:
trunk驅動支持ovs和linux bridge, 代碼主要在neutron/services/trunk目錄下:
trunk驅動
五、CLI命令行和API接口
|
CLI verb (as in the spec) |
CLI verb (openstack client) |
HTTP method |
URL |
|
trunk-create |
network trunk create |
POST |
/v2.0/trunks |
|
trunk-delete |
network trunk delete |
DELETE |
/v2.0/trunks/$trunk_id |
|
trunk-list |
network trunk list |
GET |
/v2.0/trunks |
|
trunk-show |
network trunk show |
GET |
/v2.0/trunks/$trunk_id |
|
trunk-subport-add |
network trunk set |
PUT |
/v2.0/trunks/$trunk_id/add_subports |
|
trunk-subport-delete |
network trunk unset |
PUT |
/v2.0/trunks/$trunk_id/remove_subports |
|
trunk-subport-list |
network subport list |
GET |
/v2.0/trunks/$trunk_id/get_subports |