SDN控制器Floodlight源碼學習(八)--轉發模塊(Forwarding)

很久沒更新了，今天來學習下Forwarding模塊,先看看官網上對這個模塊的說明:

Description
Forwarding will forward packets between two devices. The source and destination devices will be classified by the IDeviceService.
How it works
Since Floodlight is designed to work in networks that contain both OpenFlow and non-OpenFlow switches Forwarding has to take this into account. The algorithm will find all OpenFlow islands that have device attachment points for both the source and destination devices. FlowMods will then be installed along the shortest path for the flow. If a PacketIn is received on an island and there is no attachment point for the device on that island the packet will be flooded.

官網上主要就是說，Forwarding模塊能夠找到源和目的設備之間的最短路徑，並通過下發流表項的方式讓交換機來對數據包進行轉發。下面我們深入的學習下該模塊.

Forwarding作爲一個實現了IOFMessageListener的模塊，表示它處於進入控制器的數據包的處理隊列中，我們先來看receive函數的功能:

@Override
    public Command receive(IOFSwitch sw, OFMessage msg, FloodlightContext cntx) {
    switch (msg.getType()) {
        case PACKET_IN:
            IRoutingDecision decision = null;
            if (cntx != null) {
                decision = RoutingDecision.rtStore.get(cntx, IRoutingDecision.CONTEXT_DECISION);
            }

            return this.processPacketInMessage(sw, (OFPacketIn) msg, decision, cntx);
        default:
            break;
        }
        return Command.CONTINUE;
    }

通過代碼可以看出，Forwarding模塊只處理packet_in消息，首先Forwarding模塊會從RoutingDecision模塊的上線文環境中獲取路由策略，並傳入processPacketInMessage函數中進行處理，然後我們看看processPacketInMessage模塊:

@Override
    public Command processPacketInMessage(IOFSwitch sw, OFPacketIn pi, IRoutingDecision decision, FloodlightContext cntx) {
        Ethernet eth = IFloodlightProviderService.bcStore.get(cntx, IFloodlightProviderService.CONTEXT_PI_PAYLOAD);
        // We found a routing decision (i.e. Firewall is enabled... it's the only thing that makes RoutingDecisions)
        if (decision != null) {
            if (log.isTraceEnabled()) {
                log.trace("Forwarding decision={} was made for PacketIn={}", decision.getRoutingAction().toString(), pi);
            }
            switch(decision.getRoutingAction()) {
            case NONE:
                // don't do anything
                return Command.CONTINUE;
            case FORWARD_OR_FLOOD:
            case FORWARD:
                doForwardFlow(sw, pi, decision, cntx, false);
                return Command.CONTINUE;
            case MULTICAST:
                // treat as broadcast
                doFlood(sw, pi, decision, cntx);
                return Command.CONTINUE;
            case DROP:
                doDropFlow(sw, pi, decision, cntx);
                return Command.CONTINUE;
            default:
                log.error("Unexpected decision made for this packet-in={}", pi, decision.getRoutingAction());
                return Command.CONTINUE;
            }
        } else { // No routing decision was found. Forward to destination or flood if bcast or mcast.
            if (log.isTraceEnabled()) {
                log.trace("No decision was made for PacketIn={}, forwarding", pi);
            }

            if (eth.isBroadcast() || eth.isMulticast()) {
                doFlood(sw, pi, decision, cntx);
            } else {
                doForwardFlow(sw, pi, decision, cntx, false);
            }
        }

        return Command.CONTINUE;
    }

1.進入函數以後，首先判斷是否存在路由策略，如果存在，則根據路由策略的類型（NONE、FORWARD_OR_FLOOD、FORWARD、MULTICAST、DROP）來對數據包進行處理。
2.如果不存在路由策略，則判斷二層報文是否爲廣播和多播，如果是廣播和多播則進行doFlood，如果爲單播則進行doForwardFlow。
以上處理的流程可以通過下圖體現:

下面我們進一步看看每個具體的處理代碼
1.doForwardFlow 轉發函數

protected void doForwardFlow(IOFSwitch sw, OFPacketIn pi, IRoutingDecision decision, FloodlightContext cntx, boolean requestFlowRemovedNotifn) {
        //獲取源端口
        OFPort srcPort = OFMessageUtils.getInPort(pi);
        //獲取源IP地址
        DatapathId srcSw = sw.getId();

        //獲取目標設備
        IDevice dstDevice = IDeviceService.fcStore.get(cntx, IDeviceService.CONTEXT_DST_DEVICE);
        //獲取源設備
        IDevice srcDevice = IDeviceService.fcStore.get(cntx, IDeviceService.CONTEXT_SRC_DEVICE);

        //如果沒有目標設備，則廣播 
        if (dstDevice == null) {
            log.debug("Destination device unknown. Flooding packet");
            doFlood(sw, pi, decision, cntx);
            return;
        }

        //如果源設備沒有，直接返回
        if (srcDevice == null) {
            log.error("No device entry found for source device. Is the device manager running? If so, report bug.");
            return;
        }

        //如果配置爲允許ARP廣播同時以太網是ARP類型，則廣播
        /* Some physical switches partially support or do not support ARP flows */
        if (FLOOD_ALL_ARP_PACKETS && 
                IFloodlightProviderService.bcStore.get(cntx, IFloodlightProviderService.CONTEXT_PI_PAYLOAD).getEtherType() 
                == EthType.ARP) {
            log.debug("ARP flows disabled in Forwarding. Flooding ARP packet");
            doFlood(sw, pi, decision, cntx);
            return;
        }

        //如果發送pack_in消息的端口不是節點端口，那麼只做轉發
        /* This packet-in is from a switch in the path before its flow was installed along the path */
        if (!topologyService.isEdge(srcSw, srcPort)) {  
            log.debug("Packet destination is known, but packet was not received on an edge port (rx on {}/{}). Flooding packet", srcSw, srcPort);
            doFlood(sw, pi, decision, cntx);
            return; 
        }   

        //獲取目標設備所關聯的端口        
        for (SwitchPort ap : dstDevice.getAttachmentPoints()) {
            if (topologyService.isEdge(ap.getNodeId(), ap.getPortId())) {
                dstAp = ap;
                break;
            }
        }   

        if (dstAp == null) {
            log.debug("Could not locate edge attachment point for destination device {}. Flooding packet");
            doFlood(sw, pi, decision, cntx);
            return; 
        }

        //查看目標端口和源端口是不是同一個端口
        if (sw.getId().equals(dstAp.getNodeId()) && srcPort.equals(dstAp.getPortId())) {
            log.info("Both source and destination are on the same switch/port {}/{}. Dropping packet", sw.toString(), srcPort);
            return;
        }           

        U64 flowSetId = flowSetIdRegistry.generateFlowSetId();
        U64 cookie = makeForwardingCookie(decision, flowSetId);
        //找到從源到目標的路徑
        Path path = routingEngineService.getPath(srcSw, 
                srcPort,
                dstAp.getNodeId(),
                dstAp.getPortId());

        Match m = createMatchFromPacket(sw, srcPort, pi, cntx);

        if (! path.getPath().isEmpty()) {
            if (log.isDebugEnabled()) {
                log.debug("pushRoute inPort={} route={} " +
                        "destination={}:{}",
                        new Object[] { srcPort, path,
                                dstAp.getNodeId(),
                                dstAp.getPortId()});
                log.debug("Creating flow rules on the route, match rule: {}", m);
            }

            //設置流表
            pushRoute(path, m, pi, sw.getId(), cookie, 
                    cntx, requestFlowRemovedNotifn,
                    OFFlowModCommand.ADD);  

            for (NodePortTuple npt : path.getPath()) {
                flowSetIdRegistry.registerFlowSetId(npt, flowSetId);
            }
        } /* else no path was found */
    }

2.doFlood 泛洪函數

protected void doFlood(IOFSwitch sw, OFPacketIn pi, IRoutingDecision decision, FloodlightContext cntx) {
        OFPort inPort = OFMessageUtils.getInPort(pi);
        //構建packout消息
        OFPacketOut.Builder pob = sw.getOFFactory().buildPacketOut();
        List<OFAction> actions = new ArrayList<OFAction>();
        //獲取交換機的廣播域端口
        Set<OFPort> broadcastPorts = this.topologyService.getSwitchBroadcastPorts(sw.getId());

        if (broadcastPorts.isEmpty()) {
            log.debug("No broadcast ports found. Using FLOOD output action");
            broadcastPorts = Collections.singleton(OFPort.FLOOD);
        }

        //設置action
        for (OFPort p : broadcastPorts) {
            if (p.equals(inPort)) continue;
            actions.add(sw.getOFFactory().actions().output(p, Integer.MAX_VALUE));
        }
        pob.setActions(actions);
        // log.info("actions {}",actions);
        // set buffer-id, in-port and packet-data based on packet-in
        pob.setBufferId(OFBufferId.NO_BUFFER);
        OFMessageUtils.setInPort(pob, inPort);
        pob.setData(pi.getData());

        if (log.isTraceEnabled()) {
            log.trace("Writing flood PacketOut switch={} packet-in={} packet-out={}",
                    new Object[] {sw, pi, pob.build()});
        }
        messageDamper.write(sw, pob.build());

        return;
    }

3.doDropFlow 丟掉流函數

protected void doDropFlow(IOFSwitch sw, OFPacketIn pi, IRoutingDecision decision, FloodlightContext cntx) {
        OFPort inPort = OFMessageUtils.getInPort(pi);
        Match m = createMatchFromPacket(sw, inPort, pi, cntx);
        OFFlowMod.Builder fmb = sw.getOFFactory().buildFlowAdd();
        List<OFAction> actions = new ArrayList<OFAction>(); // set no action to drop
        U64 flowSetId = flowSetIdRegistry.generateFlowSetId();
        U64 cookie = makeForwardingCookie(decision, flowSetId); 

        /* If link goes down, we'll remember to remove this flow */
        if (! m.isFullyWildcarded(MatchField.IN_PORT)) {
            flowSetIdRegistry.registerFlowSetId(new NodePortTuple(sw.getId(), m.get(MatchField.IN_PORT)), flowSetId);
        }

        log.info("Dropping");
        fmb.setCookie(cookie)
        .setHardTimeout(FLOWMOD_DEFAULT_HARD_TIMEOUT)
        .setIdleTimeout(FLOWMOD_DEFAULT_IDLE_TIMEOUT)
        .setBufferId(OFBufferId.NO_BUFFER) 
        .setMatch(m)
        .setPriority(FLOWMOD_DEFAULT_PRIORITY);

        FlowModUtils.setActions(fmb, actions, sw);

        /* Configure for particular switch pipeline */
        if (sw.getOFFactory().getVersion().compareTo(OFVersion.OF_10) != 0) {
            fmb.setTableId(FLOWMOD_DEFAULT_TABLE_ID);
        }

        if (log.isDebugEnabled()) {
            log.debug("write drop flow-mod sw={} match={} flow-mod={}",
                    new Object[] { sw, m, fmb.build() });
        }
        boolean dampened = messageDamper.write(sw, fmb.build());
        log.debug("OFMessage dampened: {}", dampened);
    }

以上爲Forwarding 模塊大致的功能，那麼下面通過一個實驗來熟悉這個模塊,實驗的用例爲下圖:

實驗環境說明:
一臺控制器(192.168.56.1)，三臺OF交換機，S1下有兩臺主機，IP地址分別爲192.168.1.1(h1)、192.168.1.3(h3)。S2下有一臺主機，IP地址爲192.168.1.2(h2)。
實驗步驟
1.取消Forwarding模塊加載,觀察聯通情況

從實驗可以看出,再控制器不加載Forwarding模塊的情況下，ping 192.168.1.2直接返回Destination Host Unreachable,再通過arp -a命令看到h1沒有得到h2的MAC地址。證明OF交換機並沒有進行數據包的轉發。
2.恢復Forwarding模塊的加載，觀察網絡情況
恢復Forwarding模塊加載以後，同樣我們在h1上ping h2,可以看到h1已經能夠獲取h2的mac地址.

同樣我在每個接口進行抓包，抓包的情況如下:
s1-eth1接口抓包

s1-eth2接口抓包

s1-eth3接口抓包

s2-eth2接口抓包

OF交換機到控制器接口抓包

從這些抓包的情況可以分析出h1 ping h2的工作情況:
step1:h1查看本地mac地址表，沒有找到h2的mac,則發送ARP消息到s1交換機。
step2:s1收到ARP消息以後，發現流表中不存在對應項，則通過packet_in消息將ARP消息發送到控制器.
step3:控制器收到packet_in消息後，通過Forwarding模塊進行處理，發現爲ARP消息，則通過doFlood函數進行處理，形成packet_out消息，並下發至s1。
step4:s1收到packet_out消息後，將arp通過除入端口之外的其它端口進行轉發。
step5:ARP數據包進入s3,s3則重複step2-step3的操作，將數據包轉發到s2.
step6:s2重複step2-step3的操作，將ARP包發送至h2 .
step7:h2收到arp後，產生響應，數據包逆向返回到h1（與ARP過程相反）.
注意：在控制器通過packet_out消息控制OF交換機轉發數據包以後，還會下發FLOW_MOD消息，在相應的交換機的流表中添加流表項，這樣下次數據包到來的時候，OF交換機就可以通過流表進行數據轉發，而不需要告訴控制器了,下圖爲s1-s3交換機中的流表項:

這樣我們基本上了解了Forwarding模塊的工作原理.