mongodb 內存組成
對於使用WiredTiger，mongodb的內存，主要包括WirTiger引擎內部cache 與filesystem cache，對於WirTiger引擎內部cache 的限制可以通過The storage.wiredTiger.engineConfig.cacheSizeGB limits 進行限制。操作系統會把剩餘可用的內存給 filesystem cache使用，如內存中保留壓縮的mogodb文件，此外操作系統還會 buffer file system blocks and file system cache (注意區分：filesystem 與file system).
mongodb 內存使用
mongodb通過把文件mapp到內存來訪問數據，在os 層可以使用top 命令查看相應的process ,換句訪問的文件越多，process 也越多，使用的內存也越多，也可以DB內也可以使用命令來查看虛擬內存的大小，與常駐的內存。eg:
PRIMARY> db.serverStatus().mem
{
"bits" : 64, #OS位數
"resident" : 79151, #常駐內存/物理內存
"virtual" : 97810, #虛擬內存
"supported" : true, #是否支持額外的內存顯示
"mapped" : 0,
"mappedWithJournal" : 0
}
[vm:24000:mgtest:3.4.6]$top -H
top - 09:25:26 up 1652 days, 22:16, 1 user, load average: 2.28, 2.15, 2.09
Tasks: 1832 total, 1 running, 1831 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.2%us, 0.2%sy, 0.0%ni, 99.6%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 264291560k total, 174980604k used, 89310956k free, 1986892k buffers
Swap: 18874364k total, 0k used, 18874364k free, 40814252k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
17821 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.36 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17822 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.15 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17823 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 1025:49 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17829 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 107:48.61 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17830 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 12:51.93 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17831 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 52:03.17 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17832 mongodb 20 0 95.5g 77g 13m S 0.4 30.6 2255:06 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17833 mongodb 20 0 95.5g 77g 13m S 0.7 30.6 2252:49 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17834 mongodb 20 0 95.5g 77g 13m S 0.4 30.6 2253:09 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17835 mongodb 20 0 95.5g 77g 13m S 0.4 30.6 2252:23 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17836 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 93:28.86 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17837 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 22876:13 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17838 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 247:44.60 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17839 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.10 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17840 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:33.99 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17841 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:02.35 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17842 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 392:42.50 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17843 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 52:19.16 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17844 mongodb 20 0 95.5g 77g 13m S 0.4 30.6 454:07.62 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17845 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 209:58.88 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17846 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.02 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17847 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17848 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17849 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:23.83 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17850 mongodb 20 0 95.5g 77g 13m S 0.7 30.6 1325:44 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17851 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 40:42.66 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17852 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 52:46.56 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17853 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17854 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17855 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17856 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17857 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17858 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17859 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17860 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17861 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17862 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17863 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17864 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
17865 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf
[vm:24000:mgtest:3.4.6]
內存建議調整
1.使用cg 設置整個實例的內存使用值
2.storage.wiredTiger.engineConfig.cacheSizeGB 設置爲cg 的50%
storage.wiredTiger.engineConfig.cacheSizeGB參數說明：定義 WiredTiger引擎內部cache 的最大值(所有的數據)，但是創建索引時所消耗的內存是獨立於WiredTiger 的，從3.4開始，WiredTiger可以設置0.25G~1000GB。
內存淘汰策略
參考：https://mongoing.com/archives/8781
內存使用查看
主要命令：
mem: db.serverStatus.mem
tcmalloc: db.serverStatus().tcmalloc.tcmalloc.formattedString
mgtest:PRIMARY> db.stats()
{
"db" : "meam",
"collections" : 4466,
"views" : 0,
"objects" : 93999866,
"avgObjSize" : 546.7290265179739,
"dataSize" : 51392455231, # 51G
"storageSize" : 4775952384, #4G
"numExtents" : 0,
"indexes" : 12664,
"indexSize" : 6932480000, #7G
"ok" : 1
}
mgtest:PRIMARY>
mgtest:PRIMARY> db.serverStatus().tcmalloc
{
"generic" : {
"current_allocated_bytes" : NumberLong("67615090640"),
"heap_size" : NumberLong("100761542656")
},
"tcmalloc" : {
"pageheap_free_bytes" : 577167360, #
"pageheap_unmapped_bytes" : NumberLong("18533167104"),
"max_total_thread_cache_bytes" : NumberLong(1073741824),
"current_total_thread_cache_bytes" : 15028816,
"total_free_bytes" : NumberLong("14036117552"),
"central_cache_free_bytes" : NumberLong("14020716864"),
"transfer_cache_free_bytes" : 371872,
"thread_cache_free_bytes" : 15028816,
"aggressive_memory_decommit" : 0,
"formattedString" : "------------------------------------------------\n
MALLOC: 67615090640 (64482.8 MiB) Bytes in use by application\n
MALLOC: + 577167360 ( 550.4 MiB) Bytes in page heap freelist\n
MALLOC: + 14020716864 (13371.2 MiB) Bytes in central cache freelist\n
MALLOC: + 371872 ( 0.4 MiB) Bytes in transfer cache freelist\n
MALLOC: + 15028816 ( 14.3 MiB) Bytes in thread cache freelists\n
MALLOC: + 659284160 ( 628.7 MiB) Bytes in malloc metadata\n
MALLOC: ------------\n
MALLOC: = 82887659712 (79047.8 MiB) Actual memory used (physical + swap)\n
MALLOC: + 18533167104 (17674.6 MiB) Bytes released to OS (aka unmapped)\n
MALLOC: ------------\n
MALLOC: = 101420826816 (96722.4 MiB) Virtual address space used\n
MALLOC:\nMALLOC: 8139591 Spans in use\n
MALLOC: 213 Thread heaps in use\n
MALLOC: 4096 Tcmalloc page size\n
------------------------------------------------\n
Call ReleaseFreeMemory() to release freelist memory to the OS (via madvise()).\n
Bytes released to the OS take up virtual address space but no physical memory.\n"
}
}
mgtest:PRIMARY>
mgtest:PRIMARY> db.serverStatus().mem
{
"bits" : 64, #OS位數
"resident" : 79151, #常駐內存/物理內存
"virtual" : 97810, #虛擬內存
"supported" : true, #是否支持額外的內存顯示
"mapped" : 0,
"mappedWithJournal" : 0
}
#查看索引的大小
mgtest:PRIMARY>
db.getCollectionNames().map(name => ({totalIndexSize: db.getCollection(name).stats().totalIndexSize, name: name})).sort((a, b) => a.totalIndexSize - b.totalIndexSize).forEach(printjson)
{ "totalIndexSize" : 10002432, "name" : "T_article_news_flZXB91g18fcqjl75QH6" }
{ "totalIndexSize" : 10059776, "name" : "T_article_news_cSFcNrBaiV7Cud52EaXb" }
{ "totalIndexSize" : 10117120, "name" : "T_article_news_PHymgm317Gz2A2ws0m36" }
{ "totalIndexSize" : 10121216, "name" : "T_article_news_342hVXE31f56wV0H736i" }
{ "totalIndexSize" : 10240000, "name" : "T_article_news_31MEVF98elvUrEZm1hnY" }
{ "totalIndexSize" : 10256384, "name" : "T_article_news_1sjqPaiEZ7DpFHa0H1iv" }
{ "totalIndexSize" : 10285056, "name" : "T_article_news_9ruEXeExFeRXWZvWng3Q" }
{ "totalIndexSize" : 10436608, "name" : "T_article_news_Bu4XmJBN02CEePW7EQiY" }
{ "totalIndexSize" : 10469376, "name" : "T_article_news_Jswrd1tyQbvknwyb7I03" }
{ "totalIndexSize" : 10485760, "name" : "T_article_news_FGMDEyyRg4a4a4Ot2dqW" }
{ "totalIndexSize" : 10575872, "name" : "T_article_news_mxaPsH76hw64HdKwy1i3" }
{ "totalIndexSize" : 10596352, "name" : "T_article_news_3HY77JTj4adyIbHY14Vo" }
{ "totalIndexSize" : 10657792, "name" : "T_article_news_Pkrf2913812E4X32Mh8P" }
{ "totalIndexSize" : 10670080, "name" : "T_article_news_2FhZe8jZObCJ85Nm1P0G" }
{ "totalIndexSize" : 10747904, "name" : "T_article_news_x9XW27Bfz953t7e97O6v" }
{ "totalIndexSize" : 10752000, "name" : "T_article_news_tt7S7C7UX8bb6meRz9oZ" }
{ "totalIndexSize" : 10858496, "name" : "T_article_news_RLbel0jQf1zC3gsd537O" }
{ "totalIndexSize" : 10878976, "name" : "T_article_news_58038F9kMY4BL9U8ACH7" }
{ "totalIndexSize" : 10899456, "name" : "T_article_news_3TPye0a63xb98ac64z75" }
{ "totalIndexSize" : 11051008, "name" : "T_article_news_8170mvJE27MK4wYW03Q2" }
{ "totalIndexSize" : 11177984, "name" : "T_article_news_i1vp48iiMajLq81bj40E" }
{ "totalIndexSize" : 11264000, "name" : "T_article_news_oFKE42IAk7pZ8KqC26BL" }
參考文檔
https://segmentfault.com/a/1190000021320867
https://mongoing.com/archives/8781 #MongoDB 如何使用內存？爲什麼內存滿了？
--內存使用說明：
https://www.mongodb.com/blog/post/mongo-db-memory-usage
--內存使用過多問題
https://dba.stackexchange.com/questions/148395/mongodb-using-too-much-memory
https://stackoverflow.com/questions/131303/how-to-measure-actual-memory-usage-of-an-application-or-process
http://gk.link/a/10bqK