前言：本文是學習網易微專業的《python全棧工程師》 中的《服務器運維開發工程師》專題的課程筆記，歡迎學習交流。同時感謝老師們的精彩傳授！
一、課程目標
- 內連接
- 外連接
- 子查詢
- 記錄聯合查詢
二、詳情解讀
2.1.數據基礎
2.1.1.表結構
準備一個用戶表與文章表
執行下面程序批量生成數據：
from mysql.connector import pooling
import random
from datetime import datetime, timedelta
from concurrent.futures import ThreadPoolExecutor
from pymysql import escape_string
cnxpool =pooling.MySQLConnectionPool(pool_name="mypool", pool_size=30,
user='root', password='root',
host='localhost',database='mycms')
################
# 文章內容
article_template = '''
What’s New In Python 3.8
Editor
Raymond Hettinger
This article explains the new features in Python 3.8, compared to 3.7. For full details, see the changelog.
Python 3.8 was released on October 14th, 2019.
Summary – Release highlights
New Features
Assignment expressions
There is new syntax := that assigns values to variables as part of a larger expression. It is affectionately known as “the walrus operator” due to its resemblance to the eyes and tusks of a walrus.
In this example, the assignment expression helps avoid calling len() twice:
if (n := len(a)) > 10: print(f"List is too long ({n} elements, expected <= 10)")
A similar benefit arises during regular expression matching where match objects are needed twice, once to test whether a match occurred and another to extract a subgroup:
discount = 0.0
if (mo := re.search(r'(\d+)% discount', advertisement)):
discount = float(mo.group(1)) / 100.0
The operator is also useful with while-loops that compute a value to test loop termination and then need that same value again in the body of the loop:
# Loop over fixed length blocks
while (block := f.read(256)) != '':process(block)
Another motivating use case arises in list comprehensions where a value computed in a filtering condition is also needed in the expression body:
[clean_name.title() for name in names
if (clean_name := normalize('NFC', name)) in allowed_names]
Try to limit use of the walrus operator to clean cases that reduce complexity and improve readability.
See PEP 572 for a full description.
(Contributed by Emily Morehouse in bpo-35224.)
Positional-only parameters
There is a new function parameter syntax / to indicate that some function parameters must be specified positionally and cannot be used as keyword arguments. This is the same notation shown by help() for C functions annotated with Larry Hastings’ Argument Clinic tool.
In the following example, parameters a and b are positional-only, while c or d can be positional or keyword, and e or f are required to be keywords:
def f(a, b, /, c, d, *, e, f):print(a, b, c, d, e, f)
The following is a valid call:
f(10, 20, 30, d=40, e=50, f=60)
However, these are invalid calls:
f(10, b=20, c=30, d=40, e=50, f=60) # b cannot be a keyword argument
f(10, 20, 30, 40, 50, f=60) # e must be a keyword argument
One use case for this notation is that it allows pure Python functions to fully emulate behaviors of existing C coded functions. For example, the built-in pow() function does not accept keyword arguments:
def pow(x, y, z=None, /):
"Emulate the built in pow() function"r = x ** y
return r if z is None else r%z
Another use case is to preclude keyword arguments when the parameter name is not helpful. For example, the builtin len() function has the signature len(obj, /). This precludes awkward calls such as:
len(obj='hello') # The "obj" keyword argument impairs readability
A further benefit of marking a parameter as positional-only is that it allows the parameter name to be changed in the future without risk of breaking client code. For example, in the statistics module, the parameter name dist may be changed in the future. This was made possible with the following function specification:
def quantiles(dist, /, *, n=4, method='exclusive')
Since the parameters to the left of / are not exposed as possible keywords, the parameters names remain available for use in **kwargs:
This greatly simplifies the implementation of functions and methods that need to accept arbitrary keyword arguments. For example, here is an excerpt from code in the collections module:
Parallel filesystem cache for compiled bytecode files
The new PYTHONPYCACHEPREFIX setting (also available as -X pycache_prefix) configures the implicit bytecode cache to use a separate parallel filesystem tree, rather than the default __pycache__ subdirectories within each source directory.
The location of the cache is reported in sys.pycache_prefix (None indicates the default location in __pycache__ subdirectories).
(Contributed by Carl Meyer in bpo-33499.)
Debug build uses the same ABI as release build
Python now uses the same ABI whether it’s built in release or debug mode. On Unix, when Python is built in debug mode, it is now possible to load C extensions built in release mode and C extensions built using the stable ABI.
Release builds and debug builds are now ABI compatible: defining the Py_DEBUG macro no longer implies the Py_TRACE_REFS macro, which introduces the only ABI incompatibility. The Py_TRACE_REFS macro, which adds the sys.getobjects() function and the PYTHONDUMPREFS environment variable, can be set using the new ./configure --with-trace-refs build option. (Contributed by Victor Stinner in bpo-36465.)
On Unix, C extensions are no longer linked to libpython except on Android and Cygwin. It is now possible for a statically linked Python to load a C extension built using a shared library Python. (Contributed by Victor Stinner in bpo-21536.)
On Unix, when Python is built in debug mode, import now also looks for C extensions compiled in release mode and for C extensions compiled with the stable ABI. (Contributed by Victor Stinner in bpo-36722.)
To embed Python into an application, a new --embed option must be passed to python3-config --libs --embed to get -lpython3.8 (link the application to libpython). To support both 3.8 and older, try python3-config --libs --embed first and fallback to python3-config --libs (without --embed) if the previous command fails.
Add a pkg-config python-3.8-embed module to embed Python into an application: pkg-config python-3.8-embed --libs includes -lpython3.8. To support both 3.8 and older, try pkg-config python-X.Y-embed --libs first and fallback to pkg-config python-X.Y --libs (without --embed) if the previous command fails (replace X.Y with the Python version).
On the other hand, pkg-config python3.8 --libs no longer contains -lpython3.8. C extensions must not be linked to libpython (except on Android and Cygwin, whose cases are handled by the script); this change is backward incompatible on purpose. (Contributed by Victor Stinner in bpo-36721.)
'''
titles = article_template.splitlines()[1:]
###
# articls表
create_artilces='''
CREATE TABLE `articles2` (
`article_id` int(11) NOT NULL AUTO_INCREMENT,
`article_type` int(11) NOT NULL,
`title` char(255) NOT NULL,
`content` text ,
`author` int(11) DEFAULT NULL,
`pub_date` datetime DEFAULT NULL,
`edit_date` datetime DEFAULT NULL,
PRIMARY KEY (`article_id`),
KEY `book_type_index` (`article_type`),
KEY `author_idx` (`author`),
CONSTRAINT `author` FOREIGN KEY (`author`) REFERENCES `users` (`user_id`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
'''
# 取出用戶id，用於隨機設定作者
cnx = cnxpool.get_connection()
cursor = cnx.cursor()
cursor.execute(create_artilces)
cnx.commit()
user_ids_sql = "select user_id from users";
cursor.execute(user_ids_sql)
user_ids = cursor.fetchall()
cursor.close()
cnx.close()
def createBatchArticles():
try:
cnx = cnxpool.get_connection()
cursor = cnx.cursor()
except:
print("wait...")
return
sql_list = []
sql = "INSERT INTO `mycms`.`articles` VALUES "
# 批量創建1萬條數據
for i in range(0,10):
article = {
"title" :escape_string(random.choice(titles))[0:200],
"content" :escape_string("\r\n".join(random.choices(titles, k=random.randint(10,20)))),
"author": random.choice(user_ids)[0],
"pub_date": datetime.now()- timedelta(days=random.randint(0,300)),
"edit_date": datetime.now()
}
values = "(null, 1, '{title}', '{content}', '{author}', '{pub_date}','{edit_date}')".format(**article)
sql_list.append(values)
try:
sql += ",".join(sql_list)
# print(sql)
cursor.execute(sql)
cnx.commit()
except Exception as e:
print("error:",e)
finally:
cursor.close()
cnx.close()
pool = ThreadPoolExecutor(10)
# 每循環一次，生成10條文件
for i in range(10000):
pool.submit(createBatchArticles)
2.2.內連接查詢
內連接查詢，關聯表之間必須有相互匹配的記錄。
隱式語法：
select * from users, articles where users.user_id=articles.author
顯式語法：
select * from users inner join articles on users.user_id=articles.author
示例一：查詢 user_id 爲14820 發佈的文章
SELECT * FROM users, articles where users.user_id=articles.author and user_id=14820;
運行結果：
示例二：查詢 user_id 在100~200之間的會員發佈的文章
SELECT * FROM users.articles where users.user_id=articles.author and user_id > 100 and user_id < 200;
查詢結果：
示例三：在內連接查詢基礎上做統計(實際查詢不用這麼做， 這裏只是演示說可以在查詢時做統計)
select user_id, count(*) as num from (select users.*, articles.article_id from users, articles where users.user_id=articles.author and user_id > 100 and user_id < 200) as temp_table group by user_id;
查詢結果：
示例四：使用 inner join
select * from users inner join articles on users.user_id=articles.author where user_id=14820;
查詢結果：
示例五：可以只查詢某些字段（結果只包含username，title）
SELECT users.username, articles.title FROM users, articles where users.user_id=articles.author and user_id > 100 and user_id < 200;
查詢結果：
示例六：可以使用別名
SELECT u.username as un, ac.title as act FROM users as u, articles as ac where u.user_id=ac.author user_id > 100 and user_id < 200;
查詢結果：
查詢語法 ：
SELECT * FROM articles, category WHERE article.cate_id=category.category_id;
articles表和categroy表結構分別如下：
內連接查詢結果中爲在關聯表中樸素匹配的記錄。
2.3.外連接查詢
外連接查詢，關聯表之間不需要相互匹配，分左連接、右連接。
左連接查詢語法：
select * from users left join articles on users.users_id=article.author;
左連接，就是包含左邊的這張表的查詢記錄，不管右邊的表有沒有對應的記錄。
右連接查詢語法：
select * from users right join articles on users.user_id=articles.author;
右連接 ，就是包含右邊的這張表的查詢記錄，不管左邊的表有沒有對應的記錄。
users表和articles表結構如下：
示例一：常規查詢（查找出沒有發佈文章的用戶）
select * from articles where author in (100, 110, 120, 130, 140);
示例二：內連接（因爲文章爲空，無法相互匹配，所以查詢結果也爲空）
select * from users, articles where users.user_id=articles.author and user_id in (100, 110, 120, 130, 140);
示例三：外連接中的左連接
select * from users left join articles on users.user_id=articles.author where user_id in (100, 110, 120, 130, 140);
上面的左連接以users表爲準，先查詢用戶，再查找對應的用戶有沒有發佈文章。因爲都是沒有發佈文章的用戶，所以文章的字段都是null值出現的。
查詢結果：
當查詢一個有發佈文章的用戶時，文章部分的字段就會有值
select * from users left join articles on users.user_id=articles.author where user_id in (89, 100, 110, 120, 130, 140);
查詢結果：
示例四：外連接中的右連接
select * from users right join articles on users.user_id=articles.author where user_id in (89, 100, 110, 120, 130, 140);
上面的歷連接以articles表爲基準，首先得有文章，然後纔會顯示相應的會員信息。
查詢結果：
2.4.子查詢
一條查詢語句的結果作爲另一條查詢的條件
查詢語法：
select * from table where id in (select id from table)
子查詢了除了可以用in之外，還可以 not in，!=，=，exits，not exists。
比如，對於如下的article表和user_rank表，查詢排名靠前的用戶文章：
查詢語法：
select * from article where user_id in (select user_id from user_rank;
示例：
select user_id from users where province='江蘇';
select * from articles where author in (select user_id from users where province='江蘇');
查詢結果：
2.5.記錄聯合
union all(多個查詢結構拼接，必須字段數量一致)：
查詢語法：
select * from table where condition
union all
select * from table where condition;
union與union all不同的是去重複。
union只保留唯一的值，會去掉重複的值。
union all保留所有的值，包括重複的值。
示例一：
select user_id, username from users where user_id < 10;
查詢結果：
select user_id, username from users where user_id < 15;
查詢結果：
合併查詢後：
select user_id, username from users where user_id < 10
union all
select user_id, username from users where user_id < 15;
查詢結果：下圖可以看到，查詢包括重複值。
如果是下面的查詢，用union，則不包括重複值
select user_id, username from users where user_id < 10
union
select user_id, username from users where user_id < 15;
查詢結果：
示例二：
select user_id, username from users where user_id < 10
union
select author, title from articles where article_id > 100 and article_id < 1000;
查詢結果：
上圖說明：
1）、合併之後的查詢結果，列名是第一個表的查詢字段名。
2）、使用union時，兩個表的查詢字段數量必須一致，否則查詢會失敗。
2.6.查詢練習
1、根據用戶名查詢某個會員發表的文章。
2、查找某個城市的會員發表的文章。
3、查找某個id的文章會員信息。
4、查找某個區間內的用戶id的用戶文章發佈情況（比如id在100~200之間的用戶）。
三、課程小結
- 01 內連接查詢
- 02 左連接查詢
- 03 右連接查詢
- 04 子查詢