[雪峯磁針石博客]大數據Hadoop工具python教程2-python訪問HDFS

https://pypi.org/project/hdfs3 已經不維護
PyArrow
https://pypi.org/project/hdfs/
https://pypi.org/project/snakebite/ python2中比較好,對python3支持不好。

hdfs和PyArrow比較常用,這裏以hdfs爲例:

 快速入門

from hdfs import InsecureClient
client = InsecureClient('http://localhost:50070', user='hduser_')

fs_folders_list = client.list("/")
print(fs_folders_list)
with client.read('/user/hduser/input.txt', encoding='utf-8') as reader:
    for line in reader:
        print(line)

執行結果:

['user']
https://china-testing.github.io/

https://diogoalexandrefranco.github.io/interacting-with-hdfs-from-pyspark/
http://wesmckinney.com/blog/python-hdfs-interfaces/
https://www.thomashenson.com/hadoop-python-example/
https://blog.cloudera.com/blog/2013/01/a-guide-to-python-frameworks-for-hadoop/

https://community.hortonworks.com/articles/92321/interacting-with-hadoop-hdfs-using-python-codes.html
http://yizhanggou.top/python%E8%AE%BF%E9%97%AEhdfs%E7%9A%84%E5%87%A0%E7%A7%8D%E6%96%B9%E5%BC%8F/
https://blog.csdn.net/Gamer_gyt/article/details/52446757

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章