使用shodan/censys API搜索免費的elasticsearch/kibana資源

記得以前我在elastic社區分享過一個主題《ITDS如何玩轉elastic》(這裏的ITDS是IT屌絲的意思)，裏面提到過如何用elasticsearch做數字貨幣的交易數據分析，回測數據分析等。其中一個問題就是，我們如何把分析結果進行共享。如果我們有一個私人的公網資源最好，沒有的話，我們可以用shodan等搜索引擎，搜索免費的資源，然後掛載數據和dashboard template進行共享。

這裏，分享以下shodan的API，方便我們隨時檢測失效的es資源，然後重新獲取。

注意，本文章是針對python的

註冊shodan並獲取API

• 在shodan的頁面上註冊一下。（免費用戶只能看100個搜索結果，並且不能用條件搜索，坑吧。。。）
• 註冊完之後，又上角又一個Show API Key，點擊即可看到你的key
  
  這裏，假設我們的key是SkVS0RAbiTQpzzEsahqnq2Hv6SwjUfs3(別人的，網上找的，無效 ?)

安裝shodan

pip3 install shodan

安裝完成之後，在你的系統上會又一個shodan CLI

$ shodan
Usage: shodan [OPTIONS] COMMAND [ARGS]...

Options:
  -h, --help  Show this message and exit.

Commands:
  alert       Manage the network alerts for your account
  convert     Convert the given input data file into a...
  count       Returns the number of results for a search
  data        Bulk data access to Shodan
  domain      View all available information for a domain
  download    Download search results and save them in a...
  honeyscore  Check whether the IP is a honeypot or not.
  host        View all available information for an IP...
  info        Shows general information about your account
  init        Initialize the Shodan command-line
  myip        Print your external IP address
  org         Manage your organization's access to Shodan
  parse       Extract information out of compressed JSON...
  radar       Real-Time Map of some results as Shodan finds...
  scan        Scan an IP/ netblock using Shodan.
  search      Search the Shodan database
  stats       Provide summary information about a search...
  stream      Stream data in real-time.

當然，只是以它作爲參考，最終我們是要在python腳本里面循環的檢查es等資源的可用性，並尋找新的資源。
先試一下：

$ shodan init SkVS0RAbiTQpzzEsahqnq2Hv6SwjUfs3
$ shodan count elasticsearch
4036

再搜索一下：

$ shodan search --fields ip_str,port,org,hostnames elasticsearch
54.36.102.199   8090    OVH SAS         
45.199.54.112   80      DXTL Tseung Kwan O Service              
159.89.28.238   8880    Digital Ocean   porn2gether.com 
154.88.111.89   80      Multacom Corporation            
154.88.111.83   80      Multacom Corporation            
13.53.217.15    8181    Amazon.com      ec2-13-53-217-15.eu-north-1.compute.amazonaws.com       
62.60.215.71    80      Iranian Research Organization for Science & Techno      mx70.myflightmode.com   
154.218.48.52   80      CloudInnovation infrastructure          
154.197.186.30  80      CloudInnovation infrastructure          
154.218.48.38   80      CloudInnovation infrastructure          
35.231.19.58    9200    Google Cloud    58.19.231.35.bc.googleusercontent.com   
62.60.214.79    80      Iranian Research Organization for Science & Techno              
193.58.133.117  80      QuickPacket LLC         
54.161.255.217  80      Amazon  ec2-54-161-255-217.compute-1.amazonaws.com      
45.199.54.106   80      DXTL Tseung Kwan O Service              
115.29.9.88     80      Hangzhou Alibaba Advertising Co.,Ltd.           
154.88.111.75   80      Multacom Corporation            
174.138.57.191  8080    Digital Ocean           
35.157.60.25    80      A100 ROW GmbH   ec2-35-157-60-25.eu-central-1.compute.amazonaws.com     
154.214.154.169 80      CloudInnovation infrastructure          
52.178.185.148  9200    Microsoft Azure         
154.93.68.123   80      Multacom Corporation            
5.196.225.154   9200    OVH SAS 154.ip-5-196-225.eu     
154.208.94.212  80      CloudInnovation infrastructure          
62.60.214.72    80      Iranian Research Organization for Science & Techno              
154.218.44.52   80      CloudInnovation infrastructure          
154.93.70.100   80      Multacom Corporation            
54.204.112.182  80      Amazon.com      ec2-54-204-112-182.compute-1.amazonaws.com      
13.81.108.180   80      Microsoft Azure         
154.197.184.153 80      CloudInnovation infrastructure          
156.235.150.206 80      DXTL Tseung Kwan O Service              
45.199.54.70    80      DXTL Tseung Kwan O Service              
62.60.208.11    80      Iranian Research Organization for Science & Techno              
154.93.69.122   80      Multacom Corporation            
101.200.169.192 9200    Hangzhou Alibaba Advertising Co.,Ltd.           
193.58.133.103  80      QuickPacket LLC         

但因爲只有付費的API才能做條件搜索，再此放棄
但還好，我們還有censys

註冊censys並獲取API

安裝censys

pip3 install censys

用censys查詢可用elasticsearch資源

import censys.certificates
import censys.ipv4
from elasticsearch import Elasticsearch

UID = "YOUR ID"
SECRET = "YOUR SECRECT"
c = censys.ipv4.CensysIPv4(UID, SECRET)

i = 0

available_es = []
for result in c.search("9200.elasticsearch.banner.system_info.version.number: 6.8.0 AND location.country: China", max_records=10):
    print(result['ip'])
    available_es.append(result['ip'])

for es_ip in available_es:
    es = Elasticsearch(hosts=[es_ip+':9200'])
    print(es.cluster.health())