Sort Integers (Map Reduce)

Sort integers by Map Reduce framework.
In the mapper, key is the document id which can be ignored, value is the integers.

In the reducer, your can specify what the key / value is (this depends on how you implement your mapper class). For the output of the reducer class, the key can be anything you want, the value should be ordered. (the order is depending on when you output it)

Example

Example 1:

Input:
1: [14,7,9]
2: [10,1]
3: [2,5,6,3]
4: []
Output:
[1,2,3,5,6,7,9,10,14]

Example 2:

Input:
1: [14,7]
Output:
[7,14]

思路:其實就是一個merge k sorted list,注意後面判斷時候還有元素的時候,要+1 進行判斷:

node.y + 1 < lists.get(node.x).size()

/**
 * Definition of OutputCollector:
 * class OutputCollector<K, V> {
 *     public void collect(K key, V value);
 *         // Adds a key/value pair to the output buffer
 * }
 */
public class SortIntegers {

    
    public static class Map {
        private int id = 0;
        public void map(int _, List<Integer> value,
                        OutputCollector<String, List<Integer>> output) {
            // Write your code here
            // Output the results into output buffer.
            // Ps. output.collect(String key, List<Integer> value);
            Collections.sort(value);
            if(value.size() > 0){
                ++id;
                output.collect(String.valueOf(id), value);
            }
        }
    }
        
    public static class Reduce {
        public void reduce(String key, List<List<Integer>> values,
                           OutputCollector<String, List<Integer>> output) {
            // Write your code here
            // Output the results into output buffer.
            // Ps. output.collect(String key, List<Integer> value);
            List<Integer> res = mergeKSortedList(values);
            output.collect(key, res);
        }
        
        private class Node {
            public int x;
            public int y;
            public int value;
            public Node(int x, int y, int value) {
                this.x = x;
                this.y = y;
                this.value = value;
            }
        }
        
        private class NodeComparator implements Comparator<Node> {
            @Override
            public int compare(Node a, Node b) {
                return a.value - b.value;
            }
        }
        
        private List<Integer> mergeKSortedList(List<List<Integer>> lists) {
            List<Integer> res = new ArrayList<Integer>();
            int k = lists.size();
            PriorityQueue<Node> pq = new PriorityQueue<Node>(k, new NodeComparator());
            for(int i = 0; i < k; i++) {
                pq.offer(new Node(i, 0, lists.get(i).get(0)));
            }
            
            while(!pq.isEmpty()) {
                Node node = pq.poll();
                res.add(node.value);
                if(node.y + 1 < lists.get(node.x).size()){
                    pq.offer(new Node( node.x, node.y + 1, lists.get(node.x).get(node.y + 1)));
                }
            }
            return res;
        }
    }
}

 

發佈了619 篇原創文章 · 獲贊 13 · 訪問量 17萬+
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章