簡介
Beautiful Soup是可以輕鬆從網頁上抓取信息的庫。 它位於HTML或XML解析器的頂部，提供Pythonic方式迭代，搜索和修改解析樹。
安裝
pip install -U beautifulsoup4
快速入門
>>> from bs4 import BeautifulSoup
>>> soup = BeautifulSoup("<p>Some<b>bad<i>HTML", 'html.parser')
>>> print(soup.prettify())
<p>
Some
<b>
bad
<i>
HTML
</i>
</b>
</p>
>>> soup.find(text="bad")
'bad'
>>> soup.i
<i>HTML</i>
>>> soup = BeautifulSoup("<tag1>Some<tag2/>bad<tag3>XML", "xml")
>>> print(soup.prettify())
<?xml version="1.0" encoding="utf-8"?>
<tag1>
Some
<tag2/>
bad
<tag3>
XML
</tag3>
</tag1>
>>>
參考資料：https://pypi.org/project/beautifulsoup4/
參考資料
- python測試開發項目實戰-目錄
- python工具書籍下載-持續更新
- python 3.7極速入門教程 - 目錄
- 討論qq羣630011153 144081101
- 原文地址
- 本文涉及的python測試開發庫 謝謝點贊！
- [本文相關海量書籍下載](https://github.com/china-testing/python-api-tesing/blob/master/books.md
- 本文持續更新，最新版本：https://www.jianshu.com/p/fbc416635987
稍稍深入
Beautiful Soup 可從HTML或XML文件中提取數據的Python庫。通過你喜歡的轉換器實現文檔導航,查找,修改。.Beautiful Soup會幫你節省數小時或天的時間.
- 加載文檔
本節示例文檔來自愛麗絲夢遊仙境故事：
>>> from bs4 import BeautifulSoup
>>> soup = BeautifulSoup(html_doc, 'html.parser')
>>> print(soup.prettify())
<html>
<head>
<title>
The Dormouse's story
</title>
</head>
<body>
<p class="title">
<b>
The Dormouse's story
</b>
</p>
<p class="story">
Once upon a time there were three little sisters; and their names were
<a class="sister" href="http://example.com/elsie" id="link1">
Elsie
</a>
,
<a class="sister" href="http://example.com/lacie" id="link2">
Lacie
</a>
and
<a class="sister" href="http://example.com/tillie" id="link3">
Tillie
</a>
;
and they lived at the bottom of a well.
</p>
<p class="story">
...
</p>
</body>
</html>
>>>
html代碼的形式：
<html><head><title>The Dormouse's story</title></head>
<body>
<p class="title"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>
<p class="story">...</p>
瀏覽器打開的效果：
- 導航該數據結構的簡單方法：
>>> soup.title
<title>The Dormouse's story</title>
>>> soup.title.name
'title'
>>> soup.title.string
"The Dormouse's story"
>>> soup.title.parent.name
'head'
>>> soup.p
<p class="title"><b>The Dormouse's story</b></p>
>>> soup.p['class']
['title']
>>> soup.a
<a class="sister" href="http://example.com/elsie" id="link1">Elsie</a>
>>> soup.find_all('a')
[<a class="sister" href="http://example.com/elsie" id="link1">Elsie</a>, <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>, <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>]
>>> soup.find(id="link3")
<a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>
常用操作：1,通過<a>標籤找到所有鏈接。2,從頁面中提取所有文本
>>> for link in soup.find_all('a'):
... print(link.get('href'))
...
http://example.com/elsie
http://example.com/lacie
http://example.com/tillie
>>> print(soup.get_text())
The Dormouse's story
The Dormouse's story
Once upon a time there were three little sisters; and their names were
Elsie,
Lacie and
Tillie;
and they lived at the bottom of a well.
...
>>>