Apache Storm流處理有序性探究

本文假設讀者已經對Storm的基礎結構有了全面理解,並知道Nimbus與supervisor在集羣之中所扮演的角色。之所以要理解Storm集羣的並行機制,是爲了能夠對數據流中數據地處理順序有一個深入地理解,這樣才能更有信心地使用工具。


首先是需要了解一些與Storm集羣並行機制相關地概念:

工作進程(worker process,就是一個JVM進程,通過在supervisor服務器上執行jps命令可以看到一個或多個的worker)的隔離性

一個Storm集羣可以運行多個Topologies,但是對於一個worker來說,它只爲一個Topology工作,這在一定程度上提供了Topologies之間的隔離性。


執行器(executor)

在工作進程JVM的的內部,可以啓動多個線程來一起工作,每個線程就是一個執行器(executor),一個執行器只能執行拓撲中的一個組件,要麼是一個Spout,要麼是一個Bolt。這裏,我們看到一個工作進程內部,某些線程可以執行Spout任務,而另外一些線程可以執行Bolt任務。


任務(tasks)

任務是Storm集羣中最細粒度的劃分,每個任務要麼是一個Spout任務,要麼是一個Bolt任務,再結合上面對於執行器的說明,我們可以看到這些任務被分配到一些執行器中去執行。我們在定義拓撲的時候,會定義好每個Spout和每個Bolt任務的數量,一旦定義好之後,是不能在拓撲運行的過程中動態變更相應組件的任務數的。


要理解Storm的並行機制,只需要理解Storm數據流中元組的處理順序即可。爲了更與生產環境接近,使用Maven作爲處理包依賴的處理工具,並通過其maven-shade-plugin插件,將需要的jar包依賴共同打包到最終的Storm任務jar包中,而不用對Storm的運行環境做環境變量做處理,這確實帶來了非常大的便利。首先,需要通過配置POM文件明確各種包的依賴:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.example</groupId>
  <artifactId>istorm</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>istorm</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
    
    <dependency>
		<groupId>org.apache.storm</groupId>
		<artifactId>storm-kafka</artifactId>
		<version>0.9.5</version>
		<scope>compile</scope>
	</dependency>
	
	<dependency>
		<groupId>org.apache.storm</groupId>
		<artifactId>storm-core</artifactId>
		<version>0.9.5</version>
		<scope>provided</scope>
	</dependency>

	<dependency>
		<groupId>redis.clients</groupId>
		<artifactId>jedis</artifactId>
		<version>2.7.3</version>
	</dependency>
	
	<dependency>
		<groupId>org.apache.kafka</groupId>
		<artifactId>kafka_2.11</artifactId>
		<version>0.8.2.1</version>
		<scope>compile</scope>
		<exclusions>
	        <exclusion>
	            <groupId>org.slf4j</groupId>
	            <artifactId>slf4j-log4j12</artifactId>
	        </exclusion>
	        <exclusion>
	            <groupId>log4j</groupId>
	            <artifactId>log4j</artifactId>
	        </exclusion>
	    </exclusions>
	</dependency>
  </dependencies>
  
  <build>
	<plugins>
		<plugin>
			<groupId>org.apache.maven.plugins</groupId>
			<artifactId>maven-shade-plugin</artifactId>
			<executions>
				<execution>
					<phase>package</phase>
					<goals>
						<goal>shade</goal>
					</goals>
				</execution>
			</executions>
			<configuration>
				<finalName>withdepends-${artifactId}-${version}</finalName>
			</configuration>
		</plugin>
	</plugins>
  </build>  
</project>

在剛剛建立好Maven任務之後,需要下載所有的依賴包,這可能需要一些時間。有了這些依賴之後,下一步就要規劃工程結構。爲了更能夠與生產環境接近,我們將Storm的數據源設定爲Kafka,Storm和Kafka相關的依賴包已經在上面的POM文件中有所體現了,至於Kafka集羣的搭建和使用,在此不做細緻的說明。整個工程的結構是這樣的:我們通過消費者向Kafka集羣的一個TOPIC發送一系列的順序的數字,1,2,3,4...N,而Storm(KafkaSpout)會讀取這些數據,將其交付給SequenceBolt,它負責爲序列數字添加上一些線程與任務相關的信息後,將其發送給PrinterBolt,這個PrinterBolt負責將收集到的數據發送給Kafka集羣中的一個結果隊列,我們能夠從終端中讀取這個這個隊列中的結果,以觀察最終的序列被處理的順序。爲了更能說明問題,我們的Topology能夠通過參數來控制總的worker數量以及各個組件的tasks的數量,這在下面的代碼示例中都會有所體現。


首先,是SequenceBolt的實現代碼:

package com.example.istorm;

import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.Map;
import java.util.Random;

import org.apache.commons.lang.StringUtils;

import com.google.common.collect.ImmutableList;

import backtype.storm.task.TopologyContext;
import backtype.storm.topology.BasicOutputCollector;
import backtype.storm.topology.OutputFieldsDeclarer;
import backtype.storm.topology.base.BaseBasicBolt;
import backtype.storm.tuple.Fields;
import backtype.storm.tuple.Tuple;

public class SequenceBolt extends BaseBasicBolt {
	
	private static final long serialVersionUID = -5735813963113079832L;
	
	private int taskId;
	private Random rand;
	
	@Override
	public void prepare(Map stormConf, TopologyContext context) {
		this.taskId = context.getThisTaskId();
		this.rand = new Random();
	}
	
	public void execute(Tuple input, BasicOutputCollector collector) {
		String data = input.getString(0);
		String timestamp = new Date().toString();
		List<String> dataList = new ArrayList<String>();
		dataList.add("sequenceBoltThreadID:" + Thread.currentThread().getId() + "@" + timestamp + "@sequnceTaskID:" + this.taskId);
		dataList.add(data);
		// simulate the work latency
		try {
			Thread.sleep(this.rand.nextInt(47));
		} catch (InterruptedException e) {
			e.printStackTrace();
		}
		collector.emit(ImmutableList.of((Object) StringUtils.join(dataList, '\t')));
	}
	
	public void declareOutputFields(OutputFieldsDeclarer declarer) {
		declarer.declare(new Fields("sequence"));
	}
	
	public static void main(String[] args) throws InterruptedException{
		List<String> dataList = new ArrayList<String>();
		dataList.add("100");
		dataList.add("200");
		System.out.println(StringUtils.join(dataList, '\t'));
		System.out.println(ImmutableList.of((Object) StringUtils.join(dataList, '\t')));
		System.out.println("Sleep start");
		Random rand = new Random();
		Thread.sleep(rand.nextInt(1000));
		System.out.println("Sleep end");
	}
}

這個Bolt的邏輯非常簡單,僅僅是在接收到序列數據之後,在其上加上了事件戳、線程信息、任務信息之後,直接提交這個數據。這麼簡單的邏輯,相信理解起來非常簡單。下面是更簡單的PrinterBolt,它僅僅負責將過來的數據發送到Kafka的結果隊列之中:

package com.example.istorm;

import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.Map;

import org.apache.commons.lang.StringUtils;

import com.example.utils.KafkaUtils;

import backtype.storm.task.TopologyContext;
import backtype.storm.topology.BasicOutputCollector;
import backtype.storm.topology.OutputFieldsDeclarer;
import backtype.storm.topology.base.BaseBasicBolt;
import backtype.storm.tuple.Tuple;

public class PrinterBolt extends BaseBasicBolt {
	private static final long serialVersionUID = -6508964903665304388L;
	
	// Every printer bolt task will hold a kafkaUtils object that will be used to
	// post the result to the Kafka servers
	private KafkaUtils kafkaUtils = null;
	private int taskId;
	
	@Override
	public void prepare(Map stormConf, TopologyContext context) {
		this.kafkaUtils = new KafkaUtils("resulttopic"); // we will send result to this topic
		this.taskId = context.getThisTaskId();
    }
	
    @Override
    public void cleanup() {
    	if(this.kafkaUtils != null){
    		this.kafkaUtils.close();
    	}
    }    	

	public void execute(Tuple input, BasicOutputCollector collector) {
		String sequence = input.getStringByField("sequence");
		String timestamp = new Date().toString();
		
		List<String> dataList = new ArrayList<String>();
		dataList.add("printerBoltThreadID:" + Thread.currentThread().getId() + "@" + timestamp + "@printerTaskID:" + this.taskId);
		dataList.add(sequence);
		this.kafkaUtils.publishMessage(StringUtils.join(dataList, '\t'));
	}

	public void declareOutputFields(OutputFieldsDeclarer arg0) {
		// we don't emit anything
	}

}

在這個程序中的prepare方法中,創建了Kafka的工具類,用來向Kafka的某個TOPIC發送消息,這個工具類的代碼也只有簡單幾行而已:

package com.example.utils;

import java.util.Properties;

import kafka.javaapi.producer.Producer;
import kafka.producer.KeyedMessage;
import kafka.producer.ProducerConfig;

public class KafkaUtils {

        private Producer<String, String> producer;
        private String topic;

        public KafkaUtils(String topic){
                Properties props = new Properties();
                props.put("metadata.broker.list", "a00:9092,a01:9092,a02:9092");
                props.put("serializer.class", "kafka.serializer.StringEncoder");
                props.put("partitioner.class", "com.example.utils.SimplePartitioner");
                props.put("request.required_acks", "1");
                ProducerConfig config = new ProducerConfig(props);
                this.producer = new Producer<String, String>(config);
                this.topic = topic;
        }

        public void publishMessage(String msg){
                KeyedMessage<String, String> data = new KeyedMessage<String, String>(this.topic, msg, msg);
                this.producer.send(data);
        }
        
        public void close(){
        	this.producer.close();
        }
        
        public static void main(String[] args){
            String topic = "datasrctopic";
            KafkaUtils sp = new KafkaUtils(topic);
            
            for(int i=0; i < 100; i++){
            	sp.publishMessage(Integer.toString(i));
            }
            sp.close();
        }
}

其中的main方法,實現了向Kafka中發送數據的功能,我們以後的測試數據就是通過執行這個方法獲取的。爲了更能說明問題,我實現了一個Kafka簡單的分區函數,其實現也是非常簡單的,其代碼也是最簡短的:

package com.example.utils;

import kafka.producer.Partitioner;
import kafka.utils.VerifiableProperties;

public class SimplePartitioner implements Partitioner {

        public SimplePartitioner(VerifiableProperties props){

        }

        public int partition(Object key, int a_numPartitions) {
                String partitionKey = (String) key;
                return partitionKey.hashCode() % a_numPartitions;
        }

}

這個代碼,我們期望能夠將消息分散到Kafka相應TOPIC中不同的分區中。最後,是將整個Topology初始化:


package com.example.istorm;

import backtype.storm.spout.SchemeAsMultiScheme;
import backtype.storm.topology.TopologyBuilder;

import java.util.ArrayList;
import java.util.List;

import backtype.storm.Config;
//import backtype.storm.LocalCluster;
import backtype.storm.StormSubmitter;
import backtype.storm.generated.AlreadyAliveException;
import backtype.storm.generated.InvalidTopologyException;
import storm.kafka.KafkaSpout;
import storm.kafka.SpoutConfig;
import storm.kafka.StringScheme;
import storm.kafka.ZkHosts;

public class KafkaTopology {
	public static void main(String[] args) throws AlreadyAliveException, InvalidTopologyException{
		// Zookeeper hosts for the Kafka cluster
		ZkHosts zkHosts = new ZkHosts("a00:2181,a01:2181,a02:2181");
		// Create the KafkaSpout configration
		// Second argument is the topic name
		// Third argument is the Zookeeper root for Kafka
		// Fourth argument is consumer group id
		SpoutConfig kafkaConfig = new SpoutConfig(zkHosts, "datasrctopic","/tmp", "id7");
		
		int workerNum = Integer.parseInt(args[0]);
		int kafkaSpoutNum = Integer.parseInt(args[1]);
		int kafkaSpoutTaskNum = Integer.parseInt(args[2]);
		int sequenceBoltNum = Integer.parseInt(args[3]);
		int sequenceBoltTaskNum = Integer.parseInt(args[4]);
		int printerBoltNum = Integer.parseInt(args[5]);
		int printerBoltTaskNum = Integer.parseInt(args[6]);
		
//		final List<String> zkServerAddresses = new ArrayList<String>();
//		zkServerAddresses.add("a00");
//		zkServerAddresses.add("a01");
//		zkServerAddresses.add("a02");
//		kafkaConfig.zkServers = zkServerAddresses;
//		kafkaConfig.zkPort = 2181;
		
		// Specify that the kafka messages are String
		kafkaConfig.scheme = new SchemeAsMultiScheme(new StringScheme());
		// We want to consume all the first messages in
		// the topic every time we run the topology to
		// help in debugging. In production, this property should be false
		kafkaConfig.forceFromStart = false;
		// Now we create the topology
		TopologyBuilder builder = new TopologyBuilder();
		// Set the kafka spout class
		builder.setSpout("KafkaSpout", new KafkaSpout(kafkaConfig), kafkaSpoutNum)
		.setNumTasks(kafkaSpoutTaskNum);
		// Configure the bolts
		builder.setBolt("SequenceBolt", new SequenceBolt(), sequenceBoltNum)
		.setNumTasks(sequenceBoltTaskNum)
		.globalGrouping("KafkaSpout");
		
		builder.setBolt("PrinterBolt", new PrinterBolt(), printerBoltNum)
		.setNumTasks(printerBoltTaskNum)
		.globalGrouping("SequenceBolt");
		
		Config conf = new Config();
		conf.setNumWorkers(workerNum);
		StormSubmitter.submitTopology("KafkaTopology", conf, builder.createTopology());
		
		/*
		// Create an instance of LocalCluster class
		// for executing in local mode.
		LocalCluster cluster = new LocalCluster();
		
		// Submit topology for execution
		cluster.submitTopology("KafkaTopology", conf, builder.createTopology());
		try{
			// Wait for some time before exiting
			Thread.sleep(100000);
		} catch(Exception exception){
			System.out.println("Thread interrupted exception: " + exception);
		}
		// Kill the KafkaTopology
		cluster.killTopology("KafkaTopology");
		// Shutdown the storm test cluster
		cluster.shutdown();
		*/
	}
}


這裏需要注意的一點是,如果Storm是運行在LocalCluster的模式下,那麼我們必須顯式指定如下的代碼,才能讓Storm-Kafka記住上次的偏移量到Zookeeper服務中:

		final List<String> zkServerAddresses = new ArrayList<String>();
		zkServerAddresses.add("a00");
		zkServerAddresses.add("a01");
		zkServerAddresses.add("a02");
		kafkaConfig.zkServers = zkServerAddresses;
		kafkaConfig.zkPort = 2181;

當然,如果是在集羣模式下運行,就不需要這幾行代碼了。在我們的Topology的配置中,我們使用了globalGrouping作爲分發數據的方法,這種分發數據流的方法並不會將數據流分區,而是將整個數據流完整地分發到具有最小任務ID地BOLT中,有點兒類似於MapReduce中的Reduce階段的數據流。

場景1:數據源的順序確定。

對於數據源的順序,我們經常會忽視。對於像Kafka這樣的數據源,我們就不得不考慮其分區特性對於Storm數據源輸入順序的影響:Kafka在每個分區上,通過順序偏移量的機制爲所有消息提供了強的順序保證,但是在不同的分區之間並沒有這種順序的保證。爲了驗證這個想法,我們在Kafka的數據源隊列的TOPIC創建時,這樣創建:

bin/kafka-topics.sh --create --zookeeper a00:2181,a01:2181,a02:2181 --replication-factor 1 --partitions 10 --topic datasrctopic

數據源TOPIC創建時,指定其有10個分區,我們期待Storm在讀取這個TOPIC的數據時,總體上是無序的(雖然每個分區內的消息時有序的)。而最終的結果TOPIC,爲了消除多分區對輸出順序的影響,我們僅僅指定其有1個分區:

bin/kafka-topics.sh --create --zookeeper a00:2181,a01:2181,a02:2181 --replication-factor 1 --partitions 1 --topic resulttopic

在有了如上的準備之後,我們就可以提交Topology到Storm集羣上了:

bin/storm jar /tmp/withdepends-istorm-0.0.1-SNAPSHOT.jar com.example.istorm.KafkaTopology 1 1 1 1 1 1 1

爲了消除其它影響,我們將worker數量以及各個組件的執行器數量和任務數量都設置爲1,這樣整體的順序就取決於數據源的順序了。這個時候,我們去執行一下KafkaUtils中的main函數,我們從結果隊列中得到如下的輸出:

printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	0
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	2
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	1
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	4
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	7
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	5
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	8
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	11
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	3
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	12
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	13
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	15
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	18
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	6
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	17
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	14
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	16
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	9
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	10
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	21
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	19
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	20
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	22
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	24
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	23
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	25
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	28
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	27
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	26
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	29
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	30
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	31
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	33
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	36
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	35
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	32
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	34
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	38
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	37
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	39
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	40
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	42
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	45
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	44
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	41
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	43
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	47
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	46
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	48
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	50
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	49
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	51
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	52
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	53
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	55
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	54
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	56
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	57
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	58
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	59
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	60
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	61
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	63
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	62
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	65
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	67
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	64
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	66
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	68
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	71
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	69
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	70
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	73
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	75
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	72
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	74
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	76
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	77
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	79
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	78
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	82
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	81
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	80
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	83
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	84
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	87
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	85
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	86
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	88
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	90
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	89
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	91
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	92
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	93
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	94
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	95
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	97
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	96
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	98
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	99

明顯看到,整個輸出時無序的。這個時候的無序完全是由於Kafka不能在多個分區上保持消息的有序性造成的,那麼如果要讓消息有序有兩種方法,一種方法就是修改Kafka的分區函數,將其全部消息發向同一個分區;第二種方法,就是在創建TOPIC的時候,指定只有1個分區。當然,作爲測試,我們採用第二種方法,在實際應用下,可能要利用第一種方法,採取某種Kafka的分區策略,比如按照用戶的uid分區到Kafka的各個分區上,保證每個用戶產生的消息是有序的。採用第二種方法就是簡單地重建TOPIC:

bin/kafka-topics.sh --delete --zookeeper a00:2181,a01:2181,a02:2181 --topic datasrctopic
bin/kafka-topics.sh --create --zookeeper a00:2181,a01:2181,a02:2181 --replication-factor 1 --partitions 1 --topic datasrctopic

在做好如上地處理之後,我們再執行先前地程序,最終地輸出就成爲:

printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	0
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	1
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	2
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	3
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	4
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	5
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	6
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	7
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	8
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	9
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	10
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	11
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	12
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	13
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	14
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	15
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	16
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	17
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	18
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	19
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	20
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	21
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	22
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	23
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	24
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	25
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	26
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	27
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	28
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	29
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	30
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	31
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	32
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	33
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	34
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	35
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	36
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	37
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	38
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	39
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	40
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	41
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	42
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	43
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	44
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	45
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	46
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	47
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	48
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	49
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	50
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	51
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	52
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	53
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	54
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	55
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	56
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	57
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	58
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	59
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	60
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	61
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	62
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	63
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	64
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	65
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	66
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	67
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	68
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	69
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	70
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	71
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	72
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	73
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	74
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	75
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	76
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	77
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	78
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	79
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	80
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	81
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	82
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	83
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	84
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	85
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	86
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	87
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	88
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	89
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	90
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	91
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	92
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	93
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	94
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	95
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	96
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	97
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	98
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	99

正如,我們地預期,輸出確實變成了順序輸出。這樣就提醒我們,在需要順序處理某些數據地時候,一定不能忽視數據源的順序,具體到Kafka,就是採用合適的分區策略,利用單個分區的有序特性來達成自己的目的。


場景2:探究Global Grouping的特性。

Global Grouping不會對數據流做分區處理,而是將整個數據流完整發送到具有最小任務ID的BOLT任務上,也就是說,不論一個BOLT開幾個任務,只有一個任務會獲取數據流,也就是說,在我們的示例中,不論我們開啓幾個BOLT任務,我們得到也總是有序的輸出。爲測試,我們的想法,我們將以如下的方式來提交Topology到Storm集羣中:

bin/storm jar /tmp/withdepends-istorm-0.0.1-SNAPSHOT.jar com.example.istorm.KafkaTopology 2 2 2 2 2 2 2

這樣之後,我們向數據源TOPIC中發送數據,得到的輸出仍舊是有序的(爲了避免篇幅冗餘,不再貼具體輸出)。多個worker去讀取數據的時候,也是要參照Zookeeper中存儲的偏移量的,所以即使是多個worker去讀Kafka隊列(因爲只有一個分區)中的數據,得到的數據也是有序的,並不會在數據讀取時發生順序上的變動。


場景3:探究Shuffle Grouping的特性。

Shuffle Grouping的分發方式是把數據流中的元組隨機地在任務之間分發,既然這裏提到了隨機,那麼就有可能出現失序地問題。但是,這裏必須十分注意執行器和任務之間地關係。執行器就是一個獨立地線程,而任務就是一個Bolt地實例對象,而一個執行器可以執行多個任務。如果每個Bolt不管有多少個任務,但是隻有一個執行器來執行它們,那麼理論上是不會失序的,那麼實際上是怎樣的呢?爲了驗證,我們理論上的想法,可以在啓動Topology的時候,將每個Bolt的執行器都設置成1,而任務設置成多個,然後再做測試(不要忘了將Grouping改成Shuffle Grouping,並重新打包):

bin/storm jar /tmp/withdepends-istorm-0.0.1-SNAPSHOT.jar com.example.istorm.KafkaTopology 2 2 2 1 10 1 10

輸出如下:

printerBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:13	0
printerBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:21	1
printerBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:15	2
printerBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:17	3
printerBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:14	4
printerBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:19	5
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:22	6
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:16	7
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:18	8
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:20	9
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:13	10
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:21	11
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:19	12
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:18	13
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:17	14
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:16	15
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:20	16
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:15	17
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:22	18
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:14	19
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:17	20
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:22	21
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:13	22
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:15	23
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:19	24
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:20	25
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:21	26
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:14	27
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:16	28
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:18	29
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:16	30
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:13	31
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:22	32
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:18	33
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:14	34
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:21	35
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:20	36
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:15	37
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:19	38
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:17	39
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:14	40
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:16	41
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:13	42
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:20	43
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:17	44
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:18	45
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:22	46
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:19	47
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:21	48
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:15	49
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:17	50
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:21	51
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:16	52
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:22	53
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:13	54
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:15	55
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:20	56
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:19	57
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:18	58
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:14	59
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:22	60
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:16	61
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:19	62
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:18	63
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:17	64
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:13	65
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:15	66
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:20	67
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:21	68
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:14	69
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:17	70
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:13	71
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:20	72
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:18	73
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:19	74
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:21	75
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:14	76
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:15	77
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:16	78
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:22	79
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:14	80
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:21	81
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:13	82
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:22	83
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:16	84
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:18	85
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:20	86
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:15	87
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:19	88
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:17	89
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:17	90
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:20	91
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:19	92
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:15	93
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:16	94
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:18	95
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:21	96
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:22	97
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:14	98
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:13	99

可以看到,數據流中的元組被平均分配到了不同的任務中,但是輸出依舊是有序的。那麼繼續我們的思路,如果要讓輸出失序,只要將其中一個BOLT的執行器設置成2個即可:

bin/storm jar /tmp/withdepends-istorm-0.0.1-SNAPSHOT.jar com.example.istorm.KafkaTopology 2 2 2 2 10 1 10

這裏,我們將SequenceBolt的執行器設置成了2個,其對應的輸出是:

printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:18	2
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:20	3
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:21	4
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:16	0
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:14	1
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:19	7
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:15	5
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:13	6
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:22	8
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:17	9
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:22	10
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:19	12
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:13	11
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:20	15
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:18	17
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:16	13
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:15	14
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:21	18
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:14	16
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:19	20
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:22	21
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:17	19
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:18	24
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:21	26
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:13	22
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:20	29
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:14	23
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:15	25
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:19	30
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:20	32
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:16	27
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:21	34
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:17	28
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:22	37
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:18	38
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:13	31
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:21	40
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:15	33
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:17	35
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:22	42
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:16	36
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:18	43
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:19	47
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:14	39
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:13	41
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:20	49
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:18	51
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:14	44
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:17	45
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:16	46
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:21	52
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:15	48
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:22	53
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:16	50
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:14	54
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:20	55
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:19	58
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:13	56
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:17	57
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:19	63
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:20	64
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:15	59
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:13	60
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:22	65
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:14	61
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:18	67
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:15	62
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:21	69
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:17	66
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:16	68
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:14	70
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:16	73
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:21	71
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:15	75
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:22	72
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:13	76
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:20	74
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:18	77
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:19	78
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:17	79
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:17	80
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:22	82
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:14	81
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:20	83
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:13	85
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:18	84
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:15	86
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:21	88
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:16	87
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:19	89
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:22	92
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:19	95
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:15	90
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:17	91
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:20	96
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:18	97
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:14	93
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:21	98
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:16	94
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:13	99

可以看到,在任務流中的元組被隨機分配到不同的任務中的同時,其最終的總體順序也被打亂了,這僅僅是因爲我們爲SequenceBolt分配了兩個執行器。同樣,如果我們只爲PrinterBolt分配1個以上的執行器,也會造成輸出的無序。


總結:希望通過這幾個例子,能夠有助於更深入理解Storm並行的各個組件,並在業務對流處理有序性有要求的時候能夠採取正確的處理方案。類似Kafka這樣的輸入源,一定要考慮其自身特點,其分區內有序,分區外無序的特性,使得使用個性化的Kafka分區策略是必要的。而要保證Storm處理流程中的有序性,可以使用Global Grouping來完成。


與Kafka分區機制類似的是,Storm的Fields Grouping,比如你可以指定按照用戶的uid字段進行Grouping,那麼同一個用戶的所有的數據都會進入到同一個任務中,這樣的話你在一個任務中才能統計到某一個用戶的完整信息。


當然,從另一個角度來說,要保證順序就可能要做對應的性能犧牲,最好能夠在業務設計的時候,儘量避免這種一致性的要求,但是如果不能繞過,那也是可以滿足需求的,只不過可能要犧牲一些性能來妥協了。


發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章