自己搭建的hadoop集羣一般默認不具備機架感知能力,爲了提高hadoop集羣的網絡性能,我們常常需要配置機架感知。下面是以下步驟:
一、修改配置文件core-site.xml,添加配置屬性:
<property>
<name>topology.script.file.name</name>
<value>/home/r203/hadoop-0.20.2/rackaware.py</value>
</property>
二、編寫自己的腳本rackaware.py:
#!/usr/bin/python
#-*-coding:UTF-8 -*-
import sys
rack = {
"14a":"rack1","14b":"rack1","14c":"rack1","14d":"rack1","14e":"rack1","14f":"rack1","14g":"rack1","14h":"rack1",
"14i":"rack1","14j":"rack1","14k":"rack1","14l":"rack1","14m":"rack1","14n":"rack1","14o":"rack1","14p":"rack1",
"14q":"rack1","14r":"rack1","14s":"rack1","14t":"rack1","14u":"rack1","14v":"rack1","14w":"rack1","14x":"rack1",
"15a":"rack2","15b":"rack2","15c":"rack2","15d":"rack2","15e":"rack2","15f":"rack2","15g":"rack2","15h":"rack2",
"15i":"rack2","15j":"rack2","15k":"rack2","15l":"rack2","15m":"rack2","15n":"rack2","15o":"rack2","15p":"rack2",
"15q":"rack2","15r":"rack2","15s":"rack2","15t":"rack2","15u":"rack2","15v":"rack2","15w":"rack2","15x":"rack2",
"16a":"rack3","16b":"rack3","16c":"rack3","16d":"rack3","16e":"rack3","16f":"rack3","16g":"rack3","16h":"rack3",
"16i":"rack3","16j":"rack3","16k":"rack3","16l":"rack3","16m":"rack3","16n":"rack3","16o":"rack3","16p":"rack3",
"16q":"rack3","16r":"rack3","16s":"rack3","16t":"rack3","16u":"rack3","16v":"rack3","16w":"rack3","16x":"rack3",
}
if __name__=="__main__":
print "/" + rack.get(sys.argv[1],"rack0")
注:腳本中可以寫主機名也可以寫ip地址。
三、重啓namenode。