OVS端口鏡像（十四）

OVS上實現端口鏡像的基本流程如下:

創建mirror，在mirror中指定鏡像數據源及鏡像目的地
將創建的mirror應用到bridge中

鏡像數據源可以通過下面幾個選項來指定:

select_all: 布爾值，設置爲true時，進出該mirror所生效的bridge上的每個數據包都將被鏡像
select_dst_port: 從該port離開虛擬交換機的數據包將會被鏡像，從Guest角度看是Guest網絡接口的流入方向
select_src_port: 從該port進入虛擬交換機的數據包將會被鏡像，從Guest角度看是Guest網絡接口的流出方向
select_vlan: 指定特定VLAN做爲數據源，整個VLAN的數據包都會鏡像到目的地

鏡像目的地可以用下面選項來指定:

output_port: 將數據包鏡像到特定的port
output_vlan: 將數據包鏡像到指定VLAN, 原始數據的VLAN tag會被剝掉。若鏡像多個VLAN到同一個VLAN，沒有辦法區分鏡像後的數據包來源於哪個VLAN。

下面我們通過實例來說明OVS上的鏡像機制。我們的第一個實驗拓樸結構如下圖，我們將流入tap1網絡接口的數據包鏡像到tap3中:

首先構造環境:

ovs-vsctl add-br br0
ovs-vsctl add-port br0 tap1 -- set interface tap1 type=internal
ovs-vsctl add-port br0 tap2 -- set interface tap2 type=internal
ovs-vsctl add-port br0 tap3 -- set interface tap3 type=internal
ip netns add ns1
ip netns add ns2
ip netns add ns3
ip link set dev tap1 netns ns1
ip link set dev tap2 netns ns2
ip link set dev tap3 netns ns3
ip netns exec ns1 ip addr add 10.10.10.11/24 dev tap1
ip netns exec ns1 ip link set up tap1
ip netns exec ns2 ip addr add 10.10.10.12/24 dev tap2
ip netns exec ns2 ip link set up tap2
ip netns exec ns3 ip link set up tap3

我們從ns1中PING ns2的IP:

[root@centos3 vagrant]# ip netns exec ns1 ping 10.10.10.12 -c 2
PING 10.10.10.12 (10.10.10.12) 56(84) bytes of data.
64 bytes from 10.10.10.12: icmp_seq=1 ttl=64 time=0.060 ms
64 bytes from 10.10.10.12: icmp_seq=2 ttl=64 time=0.106 ms

--- 10.10.10.12 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.060/0.083/0.106/0.023 ms

在ns3中運行tcpdump觀察是否能收到數據包, 可以看到此時ns3的tap3並不會收到tap1訪問tap2的數據包:

[root@centos3 vagrant]# ip netns exec ns3 tcpdump -i tap3 -e -nn icmp or arp
tcpdump: WARNING: tap3: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap3, link-type EN10MB (Ethernet), capture size 65535 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel

接下來我們創建了相應的mirror, 並將其應用到br0上:

ovs-vsctl -- --id=@tap1 get port tap1  \
          -- --id=@tap3 get port tap3  \
          -- --id=@m create mirror name=m0 select_dst_port=@tap1 output_port=@tap3 \
          -- set bridge br0 mirrors=@m

此時查看OVS上的mirror:

[root@centos3 vagrant]# ovs-vsctl list mirror
_uuid               : 98b89127-cf94-4926-8d5f-76145154b03c
external_ids        : {}
name                : "m0"
output_port         : 1dcde312-e33d-439f-b646-9db92416a586
output_vlan         : []
select_all          : false
select_dst_port     : [16db4055-5b5b-411d-8e98-87813ba14eff]
select_src_port     : []
select_vlan         : []
statistics          : {tx_bytes=0, tx_packets=0}

再次進行PING實驗:

[root@centos3 vagrant]# ip netns exec ns1 ping 10.10.10.12 -c 2
PING 10.10.10.12 (10.10.10.12) 56(84) bytes of data.
64 bytes from 10.10.10.12: icmp_seq=1 ttl=64 time=0.274 ms
64 bytes from 10.10.10.12: icmp_seq=2 ttl=64 time=0.341 ms

--- 10.10.10.12 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.274/0.307/0.341/0.037 ms

在ns3上抓包可以看到成功獲得tap2迴應tap1的ICMP響應數據包:

[root@centos3 vagrant]# ip netns exec ns3 tcpdump -i tap3 -e -nn icmp or arp
tcpdump: WARNING: tap3: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap3, link-type EN10MB (Ethernet), capture size 65535 bytes
23:22:11.919448 26:1e:74:67:6c:cc > 16:fe:12:ad:f0:4f, ethertype IPv4 (0x0800), length 98: 10.10.10.12 > 10.10.10.11: ICMP echo reply, id 4411, seq 1, length 64
23:22:12.919823 26:1e:74:67:6c:cc > 16:fe:12:ad:f0:4f, ethertype IPv4 (0x0800), length 98: 10.10.10.12 > 10.10.10.11: ICMP echo reply, id 4411, seq 2, length 64
23:22:16.929503 26:1e:74:67:6c:cc > 16:fe:12:ad:f0:4f, ethertype ARP (0x0806), length 42: Request who-has 10.10.10.11 tell 10.10.10.12, length 28
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel

這種方式對應了Cisco的SPAN方式, 下面我們來實驗RSPAN方式。我們將拓樸修改爲如下圖所示，我們首先在br0上將tap1的數據包鏡像到物定VLAN: 111, 在br1上再從VLAN: 111中將數據包鏡像到tap4:

準備拓樸:

ip link add p0 type veth peer name p1
ovs-vsctl add-port br0 p0
ovs-vsctl add-port br1 p1
ovs-vsctl add-port br1 tap4 -- set interface tap4 type=internal
ip netns add ns4
ip link set tap4 netns ns4
ip netns exec ns4 ip link set up tap4

關閉VLAN:111的MAC學習功能，避免影響正常網絡轉發:

ovs-vsctl set bridge br0 flood_vlans=111
ovs-vsctl set bridge br1 flood_vlans=111

首先創建一個mirror將tap1的數據包鏡像到VLAN:111:

ovs-vsctl -- --id=@tap1 get port tap1  \
          -- --id=@m create mirror name=m1 select_src_port=@tap1 output_vlan=111 \
          -- set bridge br0 mirrors=@m

查看OVS上的mirror:

[root@centos3 vagrant]# ovs-vsctl list mirror
_uuid               : fd39fdb2-ab69-47c1-bde9-01b7b40dd4d3
external_ids        : {}
name                : "m1"
output_port         : []
output_vlan         : 111
select_all          : false
select_dst_port     : []
select_src_port     : [16db4055-5b5b-411d-8e98-87813ba14eff]
select_vlan         : []
statistics          : {tx_bytes=13426, tx_packets=157}

再次從tap1發送PING包到tap2, 我們在tap4上抓包, 可以看到tap4上可以收到鏡像的數據包:

[root@centos3 vagrant]# ip netns exec ns4 tcpdump -i tap4 -e -nn icmp or arp
tcpdump: WARNING: tap4: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap4, link-type EN10MB (Ethernet), capture size 65535 bytes
14:45:58.762260 16:fe:12:ad:f0:4f > 26:1e:74:67:6c:cc, ethertype 802.1Q (0x8100), length 102: vlan 111, p 0, ethertype IPv4, 10.10.10.11 > 10.10.10.12: ICMP echo request, id 9722, seq 1, length 64
14:45:59.806203 16:fe:12:ad:f0:4f > 26:1e:74:67:6c:cc, ethertype 802.1Q (0x8100), length 102: vlan 111, p 0, ethertype IPv4, 10.10.10.11 > 10.10.10.12: ICMP echo request, id 9722, seq 2, length 64
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel

此時，tap4爲trunk模式，所有VLAN的數據包都可以收到。我們將tap4的tag設置爲111:

ovs-vsctl set port tap4 tap=111

此時，再次重複PING訪問，tap4上依然可以收到鏡像的數據包, 只不過VLAN TAG已經被剝除:

[root@centos3 vagrant]# ip netns exec ns4 tcpdump -i tap4 -e -nn icmp or arp
tcpdump: WARNING: tap4: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap4, link-type EN10MB (Ethernet), capture size 65535 bytes
14:47:24.060144 16:fe:12:ad:f0:4f > 26:1e:74:67:6c:cc, ethertype IPv4 (0x0800), length 98: 10.10.10.11 > 10.10.10.12: ICMP echo request, id 9816, seq 1, length 64
14:47:25.117508 16:fe:12:ad:f0:4f > 26:1e:74:67:6c:cc, ethertype IPv4 (0x0800), length 98: 10.10.10.11 > 10.10.10.12: ICMP echo request, id 9816, seq 2, length 64
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel

我們再次將tag設置爲110，再次從tap4抓包則收不到數據包。

我們可以添加另一個鏡像規則，將鏡像VLAN的數據包鏡像到tap4:

ovs-vsctl -- --id=@tap4 get port tap4  \
          -- --id=@m create mirror name=m2 select_vlan=111 select_all=true output_port=@tap4 \
          -- add bridge br1 mirrors @m

查看OVS上的mirror:

 [root@centos3 vagrant]# ovs-vsctl list mirror
_uuid               : e758d8ae-408d-4e4f-a1c4-eaa8dd99bce5
external_ids        : {}
name                : "m2"
output_port         : e6a7d9b6-1032-4f9a-84f1-54b69f6db315
output_vlan         : []
select_all          : true
select_dst_port     : []
select_src_port     : []
select_vlan         : [111]
statistics          : {tx_bytes=0, tx_packets=0}

_uuid               : fd39fdb2-ab69-47c1-bde9-01b7b40dd4d3
external_ids        : {}
name                : "m1"
output_port         : []
output_vlan         : 111
select_all          : false
select_dst_port     : []
select_src_port     : [16db4055-5b5b-411d-8e98-87813ba14eff]
select_vlan         : []
statistics          : {tx_bytes=14378, tx_packets=169}

在tap4上抓包, 得到的數據包已經被剝掉VLAN TAG:

[root@centos3 vagrant]# ip netns exec ns4 tcpdump -i tap4 -e -nn icmp or arp
tcpdump: WARNING: tap4: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap4, link-type EN10MB (Ethernet), capture size 65535 bytes
14:50:37.842958 16:fe:12:ad:f0:4f > 26:1e:74:67:6c:cc, ethertype IPv4 (0x0800), length 98: 10.10.10.11 > 10.10.10.12: ICMP echo request, id 10044, seq 1, length 64
14:50:38.904391 16:fe:12:ad:f0:4f > 26:1e:74:67:6c:cc, ethertype IPv4 (0x0800), length 98: 10.10.10.11 > 10.10.10.12: ICMP echo request, id 10044, seq 2, length 64
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel

使用鏡像到物定VLAN的方式會導致原始的VLAN TAG信息丟失，如果要鏡像多個VLAN的數據到同一目的地則會造成混亂。這種場景下，最好將數據包鏡像到一個GRE port，基於這種方式可以實現ERSPAN模式。

清除mirror設置:

ovs-vsctl clear bridge br0 mirrors
ovs-vsctl clear bridge br1 mirrors

添加一個GRE端口:

ovs-vsctl add-port br0 gre0 -- set interface gre0 type=gre options:key=0x1111 options:remote_ip=10.95.30.43

創建mirror將tap1的數據包鏡像至GRE端口:

ovs-vsctl -- --id=@tap1 get port tap1  \
          -- --id=@gre0 get port gre0  \
          -- --id=@m create mirror name=m3 select_src_port=@tap1 output_port=@gre0 \
          -- set bridge br0 mirrors=@m

查看mirror:

[root@centos3 vagrant]# ovs-vsctl list mirror
_uuid               : 3c63e590-d42c-454a-a7f9-baa3e0ebb77f
external_ids        : {}
name                : "m3"
output_port         : e71af801-289f-466b-9f66-d1e5cf396233
output_vlan         : []
select_all          : false
select_dst_port     : []
select_src_port     : [16db4055-5b5b-411d-8e98-87813ba14eff]
select_vlan         : []
statistics          : {tx_bytes=0, tx_packets=0}

我們在外網出口eth0上抓包，可以看到，GRE數據包已經發送:

[root@centos3 vagrant]# tcpdump -ieth0 -nn -e proto gre
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
15:46:50.236791 08:00:27:6b:57:88 > 52:54:00:12:35:02, ethertype IPv4 (0x0800), length 140: 10.0.2.15 > 10.95.30.43: GREv0, key=0x1111, proto TEB (0x6558), length 106: 86:5e:7a:ba:4a:5f > e2:ee:06:10:cb:70, ethertype IPv4 (0x0800), length 98: 10.10.10.11 > 10.10.10.12: ICMP echo request, id 1941, seq 1, length 64
15:46:51.237935 08:00:27:6b:57:88 > 52:54:00:12:35:02, ethertype IPv4 (0x0800), length 140: 10.0.2.15 > 10.95.30.43: GREv0, key=0x1111, proto TEB (0x6558), length 106: 86:5e:7a:ba:4a:5f > e2:ee:06:10:cb:70, ethertype IPv4 (0x0800), length 98: 10.10.10.11 > 10.10.10.12: ICMP echo request, id 1941, seq 2, length 64
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel

最後需要注意一點，我們的實驗是在VirtualBox虛擬機中完成的，VirtualBox的底層虛擬網絡基礎設施不支持GRE協議，因而實際的GRE數據包並不能到達宿主機併發送到目的地，因而我們選擇在本地出口網卡處來抓包。

原文鏈接：http://just4coding.com/2017/12/02/ovs-mirror/

OVS端口鏡像（十四）

2024年DataOps趨勢預測：AI不會取代數據工程師

雲原生週刊：K8s 中的服務和網絡｜ 2024.4.29

通過Http鏈接地址爬取有贊微信商城商品信息及下載至EXCEL

多人同時導出 Excel 幹崩服務器！新來的阿里大佬給出的解決方案太優雅了！

[轉帖]cpupower

今天，昨天，近七天，近30天，近90天，js封裝

華爲云云原生FinOps解決方案，釋放雲原生最大價值

linux虛擬網絡設備--虛擬機網卡和linux bridge上tap設備的關係（七）

linux用戶態驅動--VFIO（一）

KVM virtio_net之NAPI機制（十七）

OVS端口鏡像（十五）

KVM Vhost-net 和 Virtio-net代碼詳解（十八）

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結