抓取第一批北京積分落戶名單並做簡要分析抓取分析&作圖

結論：想要通過積分落戶的方式獲得北京戶口，就要加入一家互聯網公司，然後努力工作，最有機會在35歲到50歲之間成功。

---

抓取

公示只有幾天，幸虧我在接口關閉之前把數據抓了（應該是官方怕泄露個人信息，關的很早）。

# 抓取北京市2018年積分落戶公示名單
# http://www.bjrbj.gov.cn/integralpublic/settlePerson

import csv
import json
import requests

fw = open('luohu.csv', 'w')
writer = csv.writer(fw)
writer.writerow(['id','name','birthday','company','score'])

def get_publicity(page_number):
    url = 'http://www.bjrbj.gov.cn/integralpublic/settlePerson/settlePersonJson?sort=pxid&order=asc&limit=100&offset=0&name=&rows=100&page={}'.format(page_number*100)

    header = {
            'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 '
                          '(KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36'
        }

    response = requests.get(url,
                            headers=header,
                            timeout=5
                            )

    result = json.loads(response.text)

    for item in result['rows']:
        id = item['pxid']
        name = item['name']
        birthday = item['csrq']
        company = item['unit']
        score = item['score']
        print(id,name,birthday,company,score)
        writer.writerow([id, name, birthday, company, score])

def main():
    for i in range(0,61):
        get_publicity(i)


if __name__ == '__main__':
    main()

---

分析&作圖

import pandas as pd
import matplotlib.pyplot as plt

%matplotlib inline

用describe()展示一些基本信息

df_luohu = pd.read_csv('luohu.csv')
print(df_luohu.describe())

                id        score
count  6019.000000  6019.000000
mean   3010.000000    95.654552
std    1737.679967     4.354445
min       1.000000    90.750000
25%    1505.500000    92.330000
50%    3010.000000    94.460000
75%    4514.500000    97.750000
max    6019.000000   122.590000

找到前20名落戶人數最多的公司

看看自己的公司在不在榜上

df_luohu_grouped = df_luohu.groupby('company', as_index=False, sort=False).count()[['company','id']]
df_luohu_grouped_sorted = df_luohu_grouped.sort_values('id', ascending=False)
df_luohu_grouped_sorted.head(20)

落戶積分分數分佈

bins = [90,95,100,105,110,115,120,125]
bins = pd.cut(df_luohu['score'], bins)

def get_stats(group):
    return {'count': group.count()}

grouped = df_luohu['score'].groupby(bins)
bin_counts = grouped.apply(get_stats).unstack()

bin_counts.index = ['90~95', '95~100', '100~105', '105~110', '110~115', '115~120', '120~125']
bin_counts.index.name = 'score'
bin_counts.plot(kind='bar', alpha=1, rot=0)

前面的describe信息裏可以看到，落戶積分的中位數是94.46分（最低90.75，最高122.59）。評分差不多呈幾何級數下降。

年齡分佈

df_luohu['age'] = ((pd.to_datetime('2018-11') - pd.to_datetime(df_luohu['birthday'])) / pd.Timedelta('365 days'))

bins = [25,30,35,40,45,50,55,60,65,70]
bins = pd.cut(df_luohu['age'], bins)

def get_stats(group):
    return {'count': group.count()}

grouped = df_luohu['age'].groupby(bins)
bin_counts = grouped.apply(get_stats).unstack()

bin_counts.index = ['25~30',  '30~35', '35~40', '40~45', '45~50', '50~55', '55~60', '60~65', '65~70',]
bin_counts.index.name = 'age'
bin_counts.plot(kind='bar', alpha=1, rot=0)

基本上是人到中年纔拿到了北京戶口。