Python就業班——MongoDB數據庫

1、mongodb命令啓動與停止服務

>net start "mongodb server"
>net stop "mongodb server"

 

2、命令行客戶端

>mongo --host localhost --port 27017
>use school
>show dbs
>db.student.save({name: "Scott", age: 99, sex: "male", city: "Beijing"})
>show dbs

 

3、mongoDB內置角色

以下四個角色的權限僅限於某個邏輯庫，不能管理其他邏輯庫

Read：允許用戶讀取指定邏輯庫

readWrite：允許用戶讀寫指定邏輯庫

dbAdmin：可以管理指定的邏輯庫

userAdmin：可以管理指定邏輯庫的用戶

以下四個角色只能創建在admin邏輯庫中，可以管理其他邏輯庫

readAnyDatabase：允許讀取任何邏輯庫

readWriteAnyDatabase：允許讀寫任何邏輯庫

dbAdminAnyDatabase：允許管理任何邏輯庫

userAdminAnyDatabase：允許管理任何邏輯庫用戶

以下兩個角色必須創建在admin邏輯庫，root角色權限最大

clusterAdmin：允許管理MongoDB集羣

root：超級管理員，擁有最高權限

 

4、設置登錄帳戶

use admin
db.createUser({
    user: "admin",
    pwd: "admin",
    roles: [{role: "root", db: "admin"}),
   mechanisms: ["SCRAM-SHA-1"] 
})

 

5、開啓登錄驗證功能

創建MongoDB配置文件mongo.cnf

dbpath=C:\Program Files\MongoDB\Server\4.0\data
logpath=C:\Program Files\MongoDB\Server\4.0\logs\mongod.log
auth=true

爲了讓MongoDB服務加載mongoDB.cnf文件，需要重新安裝MongoDB服務(PowerShell執行下面命令)

mongod --config "C:\Program Files\MongoDB\Server\4.0\mongoDB.cnf" --reinstall

 

6、登錄MongoDB

use admin
db.auth("admin", "admin")

 

7、操作MongoDB

創建/切換邏輯庫：use test

查看邏輯庫：show dbs

刪除邏輯庫：db.dropDatabase()

創建集合：db.createCollection("student")

查看集合：show collections

刪除集合：db.student.drop()

查看集合記錄數量：db.student.count()

查看數據空間容量：db.student.dataSize()

重命名集合：db.student.renameCollection("stu")

插入數據：

db.student.save([
{name: "AA", age: 18, sex: "male"},
{name: "BB", age: 18, sex: "male"},
{name: "VV", age: 18, sex: "male"},
{name: "CC", age: 18, sex: "male"}
])

時間戳：MongoDB存儲日期會自動轉換成格林尼治時區

ObjectId("1f5c6ce09080f119f2fd1ce0").getTimestamp()

查詢記錄：

db.student.find()
db.student.find({name: "AA", sex: "male"})
db.student.find({sex: "male", age: {$gte: 20}})
db.student.findOne()

表達式：

$lt	小於
$gt	大於
$lte	小於等於
$gte	大於等於
$in	包括
$nin	不包括
$ne	不等於

$all	全部
$not	取反
$or	或關係
$exists	含有字段

use school
db.student.find({
    age: {$gte: 20, $lte: 38}
})
// 插入一條數據
db.teacher.save({name: "Jim", role: ["班主任", "年級主任", "校長"]})
//查詢身份是班主任和年級主任的教師
db.teacher.find({role: {$all: ["班主任", "年級主任"]}})
// 查詢年齡不在28~30之間的學生
db.student.find({age: {$not: {$gte: 18, $lte: 30}}})
// 查詢30歲以下的男學生，或者25歲以下的女學生
db.student.find({
    $or: [
        {age: {$lt: 30}, sex: "男"},
        {age: {$lt: 25}, sex: "女"}
    ]
})
// 查詢含有age字段的學生
db.student.find({age: {$exists: 1}})

 正則表達式：

db.student.find({name: /^李/})
db.student.find({name: /^[a-zA-Z]{2,10}$/})

 分頁查詢：

db.student.find().limit(20);
db.student.find().skip(20).limit(10);
db.student.find({name:/^[a-zA-Z1-9]{2,10}$/}).skip(1).limit(2)

數據排序：(1代表升序，-1代表降序)

db.student.find().sort({name: 1})
db.student.find().sort({name:  -1}).skip(10).limit(10)
db.student.find({name:/^[a-zA-Z1-9]{2,10}$/}).sort({name: -1}).skip(1).limit(2)

排除重複：distinct()函數代替find()函數查找不重複的記錄

db.student.distinct("name")
db.student.distinct("name").sort(function() { return -1 })
db.student.distinct("name").slice(0, 5)
db.student.distinct("name").sort(()=>{return 1}).slice(0, 2)

刪除、修改記錄：

update：修改符合條件的第一條記錄

updateMany：修改符合條件的所有記錄

db.collection.update({condition}, {$set: {data}})
db.collection.updateMany({condition}, {$set: {data}})

// 把李強年齡修改成26歲，班級修改爲2-6
db.collection.update({name: "李強"}, {$set: {age: 26, classno: "2-6"}})
// 把26歲以上的男同學班級修改爲2-6
db.collection.updateMany({sex: "男", age: {$gte: 25}}, {$set: {classno: "2-6"}})

$unset：刪除記錄中的字段

$inc：對某個字段值都做加法運算

db.student.updateMany({}, {$unset: {city: 1, tel: 1}})

db.student.updateMany({},  {$inc: {age:2}})

$push：向數組屬性添加元素

$pull：刪除數組屬性元素

db.teacher.update({name: "Jim"}, {$push: {role: "教務主任"}})
db.teacher.update({name: "Jim"}, {$pull: {role: "副校長"}})

remove()：刪除記錄

db.student.remove({})
db.student.remove({class: "6-2", sex: "男"})

創建索引：

db.collection.createIndex({key: 1}, options)

// 1升序，-1降序
db.student.createIndex({name: 1})
// 刪除指定索引
db.student.dropIndex("index_name")
// 刪除所有索引
db.student.dropIndexes()
// 創建索引會阻塞MongoDB，影響增刪改查操作，在參數添加background，讓索引在空閒的時候創建；name設置索引名；
db.student.createIndex({name: 1}, {background: true, name: "name"})
// 獲取所有索引
db.student.getIndexes()

唯一性索引：只能創建在每個記錄都含有的公共字段上，在非公共字段上是不能創建唯一性索引的

// 創建唯一性索引
db.student.createIndex({sid: 1}, {background: true, unique: true, name: "sid_index"})

 創建索引的原則：

_1.數據量很大的集合必須創建索引，相反則不需要創建索引

_2.集合的數據讀取過多寫入，則需要創建索引

_3.給經常被當做查詢條件的字段設置索引

————————————————————————————

use school
// db.student.save({name:"jack", age: 19})
// db.dropDatabase()
// db.createCollection("teacher")
// show collections
// db.teacher.drop()
// db.student.count()
// db.student.dataSize()
// db.student.renameCollection("stu")
// show collections
// db.student.save([
// {name: "AA", age: 18, sex: "male"},
// {name: "BB", age: 18, sex: "male"},
// {name: "VV", age: 18, sex: "male"},
// {name: "CC", age: 18, sex: "male"}
// ])
// ObjectId("1f5c6ce09080f119f2fd1ce0").getTimestamp()
// db.student.find()
// db.student.find({name: "BB", sex: "male"})
// db.student.find({sex: "male", age: {$gte: 20}})

/*
db.student.find({
    age: {$gte: 20, $lte: 38}
})
// 插入一條數據
db.teacher.save({name: "Jim", role: ["班主任", "年級主任", "校長"]})
//查詢身份是班主任和年級主任的教師
db.teacher.find({role: {$all: ["班主任", "年級主任"]}})
// 查詢年齡不在28~30之間的學生
db.student.find({age: {$not: {$gte: 18, $lte: 30}}})
// 查詢30歲以下的男學生，或者25歲以下的女學生
db.student.find({
    $or: [
        {age: {$lt: 30}, sex: "男"},
        {age: {$lt: 25}, sex: "女"}
    ]
})
// 查詢含有age字段的學生
db.student.find({age: {$exists: 1}})
*/
// db.student.find().limit(2)
// db.student.find().skip(1).limit(3)

// db.student.distinct("sex")
// db.student.distinct("sex").sort(()=>{return -1})
// db.student.distinct("sex").slice(0, 1)

8、數據導出導入

_1.mongoexport導出集合數據（csv、json）

mongoexport --host=localhost --port=27017 -u admin -p admin --authenticationDatabase=admin -d school -c student [-f "_id,name,sex,age"] -o D:/student.json

_2.mongoimport導入集合的數據

mongoimport --host=localhost --port=27017 -u admin -p admin --authenticationDatabase=admin -d test -c student --file=D:/student.json

_3.mongodump導出邏輯庫的數據

--dumpDbUsersAndRoles參數可以備份隸屬於邏輯庫的用戶

mongodump --host=localhost --port=27017 -u admin -p admin --authenticationDatabase=admin [-d school] -o D:/

_4.mongorestore導入邏輯庫數據

--drop刪除已有的數據在導入

mongorestore --host=localhost --port=27017 -u admin -p admin -authenticationDatabase=admin [--drop] -d school D:/school

 9、安裝pymongo

pip install pymongo

10、創建連接、基本操作

MongoClient客戶端代理對象用來執行增刪改查操作，內置連接池

from pymongo import MongoClient

client = MongoClient(host="localhost", port=27017)
client.admin.authenticate("admin", "admin")

_1. 數據寫入：insert_one、insert_many

#!/usr/bin/env python3
# coding=utf-8
# Version:python3.6.1
# Project:pythonMongoDBDemo
# File:1、數據寫入.py
# Data:2020/10/18 14:37
# Author:LGSP_Harold
from mongo_db import client

client.school.teacher.insert_one({"name": "Jim"})
client.school.teacher.insert_many([
    {"name": "Lucifer"},
    {"name": "Harold"},
    {"name": "Adolph"},
    {"name": "Leslie"}
])


if __name__ == '__main__':
    pass

_2.數據查詢：find_one、find

#!/usr/bin/env python3
# coding=utf-8
# Version:python3.6.1
# Project:pythonMongoDBDemo
# File:2、數據查詢.py
# Data:2020/10/18 17:13
# Author:LGSP_Harold
from mongo_db import client

try:
    teachers = client.school.teacher.find({})
    for one in teachers:
        print(one["_id"], one["name"])
    print("________________")

    teacher = client.school.teacher.find_one({"name": "Jim", "role": "校長"})
    print(teacher["_id"], teacher["name"], teacher["role"])
except Exception as e:
    print(e)


if __name__ == '__main__':
    pass

_3.數據修改：update_one、update_many

#!/usr/bin/env python3
# coding=utf-8
# Version:python3.6.1
# Project:pythonMongoDBDemo
# File:3、數據修改.py
# Data:2020/11/9 15:09
# Author:LGSP_Harold
from mongo_db import client

try:
    client.school.teacher.update_many({}, {"$set": {"role": ["班主任"]}})
    client.school.teacher.update_one({"name": "Harold"}, {"$set": {"sex": "男", "age": 95}})
    client.school.teacher.update_one({"name": "Harold", "sex": {"$ne": "男"}}, {"$push": {"role": "校長"}})
except Exception as e:
    print(e)


if __name__ == '__main__':
    pass

_4.數據刪除：delete_one、delete_many

#!/usr/bin/env python3
# coding=utf-8
# Version:python3.6.1
# Project:pythonMongoDBDemo
# File:4、數據刪除.py
# Data:2020/11/9 17:53
# Author:LGSP_Harold
from mongo_db import client

try:
    client.school.teacher.delete_one({"name": "Jim"})
    client.school.teacher.delete_many({})
except Exception as e:
    print(e)

if __name__ == '__main__':
    pass

_5.其他操作：skip、limit、count、distinct、sort

#!/usr/bin/env python3
# coding=utf-8
# Version:python3.6.1
# Project:pythonMongoDBDemo
# File:5、其他操作.py
# Data:2020/11/9 18:02
# Author:LGSP_Harold
from mongo_db import client

try:
    students = client.school.student.find({}).skip(0).limit(10)
    for one in students:
        print(one)

    result = client.school.student.distinct("age")
    for one in result:
        print(one)

    students = client.school.student.find({}).sort([("name", -1)])
    for one in students:
        print(one)

    count = client.school.student.find({}).count()
    print(count)
except Exception as e:
    print(e)


if __name__ == '__main__':
    pass

 11、把文件存放在NoSql數據庫：

_1.MongoDB默認存儲16M，可在配置文件修改（不建議修改，16M是最經濟的存儲方案）

_2.GridFS存儲引擎，主要用於存儲超過16M的文件，對大文件有着更好的性能

Grid FS使用兩個集合來存儲文件，一個是chunks集合，用來存放文件；另一個集合是files，用於存儲文件的元數據（名字、大小、上傳時間、自行規定的屬性信息等）

Grid FS會把文件分割成若干chunks（256KB），然後在files記錄它們

_2.1.默認情況下MongoClient不提供操作GridFS，需要創建GridFS對象

#!/usr/bin/env python3
# coding=utf-8
# Version:python3.6.1
# Project:pythonMongoDBDemo
# File:6、GridFS.py
# Data:2020/11/9 22:14
# Author:LGSP_Harold
from mongo_db import client
from gridfs import GridFS

db = client.school
gfs = GridFS(db, "book")


if __name__ == '__main__':
    pass

_2.2.保存文件

#!/usr/bin/env python3
# coding=utf-8
# Version:python3.6.1
# Project:pythonMongoDBDemo
# File:6、GridFS.py
# Data:2020/11/9 22:14
# Author:LGSP_Harold
from mongo_db import client
from gridfs import GridFS

db = client.school
gfs = GridFS(db, "book")

file = open("D:/ProgramFiles/Development/PythonWorkspace/pythonMongoDBDemo/python無師自通.pdf", "rb")
args = {"type": "PDF", "keyword": "linux"}
gfs.put(file, filename="python無師自通.pdf", **args)
file.close()


if __name__ == '__main__':
    pass

_2.3.查詢GridFS中存儲的文件：find和find_one函數可以查詢Grid FS中存儲的文件；exists判斷是否存儲某個文件

#!/usr/bin/env python3
# coding=utf-8
# Version:python3.6.1
# Project:pythonMongoDBDemo
# File:7、GridFS查詢存儲的文件_find_one.py
# Data:2020/11/9 22:51
# Author:LGSP_Harold
from mongo_db import client
from gridfs import GridFS
import math

db = client.school

gfs = GridFS(db, collection="book")
book = gfs.find_one({"filename": "python無師自通.pdf"})
print(book.filename)
print(book.type)
print(book.keyword)
print("%dM" % (math.ceil(book.length/1024/1024)))

if __name__ == '__main__':
    pass

 

#!/usr/bin/env python3
# coding=utf-8
# Version:python3.6.1
# Project:pythonMongoDBDemo
# File:8、GridFS查詢存儲的文件_find.py
# Data:2020/11/9 23:53
# Author:LGSP_Harold
from datetime import timedelta

from mongo_db import client
from gridfs import GridFS

db = client.school
gfs = GridFS(db, "book")
books = gfs.find({"type": "PDF"})
for one in books:
    uploadDate = (one.uploadDate + timedelta(hours=8)).strftime("%Y-%m-%d %H:%M:%S")
    print(one._id, one.filename, uploadDate)

if __name__ == '__main__':
    pass

 

#!/usr/bin/env python3
# coding=utf-8
# Version:python3.6.1
# Project:pythonMongoDBDemo
# File:9、GridFS判斷是否存儲了文件.py
# Data:2020/11/10 0:52
# Author:LGSP_Harold
from mongo_db import client
from gridfs import GridFS
from bson.objectid import ObjectId

db = client.school
gfs = GridFS(db, "book")

rs = gfs.exists(ObjectId("5fa9534ec58627c971fbfc6a"))
print(rs)

rs = gfs.exists(**{"filename": "111.html"})
print(rs)


if __name__ == '__main__':
    pass

 _2.4.讀取文件

#!/usr/bin/env python3
# coding=utf-8
# Version:python3.6.1
# Project:pythonMongoDBDemo
# File:10、GridFS讀取文件.py
# Data:2020/11/10 1:04
# Author:LGSP_Harold
from mongo_db import client
from gridfs import GridFS
from bson.objectid import ObjectId

db = client.school
gfs = GridFS(db, "book")

document = gfs.get(ObjectId("5fa9534ec58627c971fbfc6a"))
file = open("D:/ProgramFiles/Development/PythonWorkspace/pythonMongoDBDemo/python無師自通1.pdf", "wb")
file.write(document.read())
file.close()



if __name__ == '__main__':
    pass

_2.5.刪除文件

#!/usr/bin/env python3
# coding=utf-8
# Version:python3.6.1
# Project:pythonMongoDBDemo
# File:11、GridFS刪除文件.py
# Data:2020/11/10 1:13
# Author:LGSP_Harold
from mongo_db import client
from gridfs import GridFS
from bson.objectid import ObjectId

db = client.school
gfs = GridFS(db, "book")
gfs.delete(ObjectId("5fa9534ec58627c971fbfc6a"))


if __name__ == '__main__':
    pass

 

 

P.S.1：pymongo文檔：https://api.mongodb.com/python/current/tutorial.html