redis

Overview of Redis
The word Redis means Remote DIctionary Server
Initial release in 2009
It is an advanced key-value store or a data structure store
Runs entirely in memory
All data is kept in memory
Quick data access since it is maintained in memory
Data can be backed up to disk periodically
Single threaded server
Extensible via Lua scripts
Able to replicate data between servers
Clustering also available

About Redis
"Redis is an open source, BSD licensed, advanced key-value cache and store. It is oftenreferred to as a data structure server since keys can contain strings, hashs, lists, sets, sorted sets, bitmaps and hyperloglogs."

Redis：
KV cache and store
in-memory
持久化
主從(藉助於sentinel實現一定意義上的HA)
Clustering(分佈式)

    數據結構服務器：
        String, List, Hash, Set, Sorted Set, Bitmap, HyperLoglog

Redis
Redis is an in-memory but persistent on disk database
1 Million small key --> String value pairs use ~ 100MB of memory
Single threaded - but CPU should not be the bottleneck
Average Linux system can deliver even 500k requests per second
Limit is likely the available memory in your system
max. 232 keys

Persistence
Snapshotting
Data is asynchronously thransferred from memory to disk
AOF(Append Only File)
Each medifying operation is written to a file
Can recreate data store by replaying operations
Without interrupting service, will rebuild AOF as the shortest sequence of commands neded to rebuild the current dataset in memory

Replication
Redis supports master-slave replication
Master-slave replication can be chained
Be careful:
Slaves are writeable!
Potential for data inconsistency
Fully compatible with Pub/Sub features

Differences to Memcached
Memcached is a "distributed memory object caching system"
Redis persists data to disk eventually
Memcached is an LRU cache
Redis has different data types and more features
Memcached is multithreaded
Similar speed

Redis的優勢
豐富的(資料形態)操作
Hashs, Lists, Sets, Sorted Sets, HyperLoglog等
內建replication及cluster
就地更新(in-place update)操作
支持持久化(磁盤)
避免雪崩效應
Memcached的優勢
多線程
善用多核CPU
更少的阻塞操作
更少的內存開銷
更少的內存分配壓力
可能有更少的內存碎片

Prominent Adopters
Twitter
Pinterest
Tumblr
GitHub
Stack Overflow
digg
Blizard
flickr
WeiBo
... ...

Redis 3.0
2015年4月1日正式推出
Redis Cluster
新的"embedded string"
LRU演算法的改進
預設隨機取5個樣本，插入並排序至一個pool，移除最佳者，如此反覆，直到內存用量小於maxmemory的設定
樣本5比先前的3多
從局部最優趨向全局最優

Redis特性
BDBMS
Oracle, DB2, PostgreSQL, MySQL, SQL Server, ...
NoSQL
Cassandra, HBase, Memcached, MongoDB, Redis, ...
NewSQL
Aerospike, FoundationDB, RethinkDB, ...

存儲系統有三類：
RDBMS：關係型數據庫
NoSQL：非關係型數據庫
KV NoSQL：redis
Colum Family NoSQL：HBase
Documention NoSQL：MongoDB
Graph NoSQL：Neo4j
NewSQL：分佈式關係型數據庫

Redis特性
Key-value NoSQL
Memcached, Redis, ...
Column family NoSQL
Cassandra, HBase, ...
Documen NoSQL
MongoDB, ...
Graph NoSQL
Neo4j, ...

Commands
Redis-server
redis-cli
Command line interface
Redis-benchmark
Benchmarking utility
redis-check-dump & redis-check-aof
Corrupted RDB/AOF files utilities

Redis的組件：
redis.io

Overview
Family of fundamental data structures
Strings and string containers
Accessed / indexed by key
Directly exposed -- No abstraction layers
Rich set of atomic operations over the structures
Detailed reference using big-O notation for complexities
Basic publish / subscribe infrastructure

官方站點：www.redis.io

Redis守護進程：
監聽端口：6379/tcp

Keys
Arbitrary ASCII strings
Define some format convention and adhere to it
Key length matters!
Multiple name spaces are available
Separate DBs indexed by an integer value
SELECT command
Multiples DBs vs. Single DB + key prefixes
Keys can expire automatically

Data structures
Strings
Caching, counters, realtime metrice...
Hashes
"Object" storage...
Lists
Logs, queues, message passing...
Sets
Membership, tracking...
Ordered sets
Leaderboards, activity feeds...

Strings
help @string
SET
GET
EXISTS
Integers
DECR
INCR

String：
SET key value [EX #] [NX|XX]
GET
INCR
DECR
EXIST

Advanced Data Structures

Lists
help @list
RPUSH
LPUSH
LPOP
RPOP

Lists：
LPUSH
RPUSH
LPOP
RPOP
LINDEX
LSET

Sets
help @set
SADD
SMEMBERS
SINTER

Sets：
SADD
SINTER
SUNION
SPOP
SISMEMBER

Sorted Sets
help @sorted_set
ZADD
ZSCORE
ZRANGE
ZRANGEBYSCORE

Sorted Sets：
ZADD
ZRANGE
ZCARD
ZRANK

Hashes
help @hash
HSET
HGET
HGETALL

Hashes：
HSET
HSETNX
HKEYS
HVALS
HDEL

Bitmaps, HyperLogLog

認證實現方法：
(1) redis.conf
requirepass PASSWORD
(2) redis-cli
AUTH PASSWORD

清空數據庫：
FLUSHDB：清空當前庫
FLUSHALL：清空所有庫

事務：
通過MULTI, EXEC, WATCH等命令實現事務功能：將一個或多個命令歸併爲一個操作提請服務器按順序執行的機制，不支持回滾操作；

    MULTI：啓動一個事務；
    EXEC：執行事務；
        一次性將事務中的所有操作執行完成後返回給客戶端；
    WATCH：樂觀鎖：在EXEC命令執行之前，用於監視指定數量鍵，如果監視中的某任意鍵數據被修改，則服務器拒絕執行事務；

Connection相關命令：
AUTH：認證相關
ECHO
PING
QUIT
SELECT

Server相關的命令：
CLIENT SETNAME connection-name：設置當前連接名稱
CLIENT GETNAME：獲取當前client連接名；
CLIENT KILL ip:port：踢掉客戶端；
CONFIG RESETSTAT：重啓狀態信息
CONFIG SET parameter value：運行中修改某參數的值；
CONFIG REVRITE：將修改在內存中的參數的值寫入配置文件；

DESIZE：顯示數據庫中鍵的數量；

BGSAVE
SAVE
LASTSAVE：獲取最近一次將數據保存到磁盤的時間戳

    MONITOR：事實監控所接收到的請求；

SHUTDOWN：將數據同步到磁盤安全關閉；

SLAVEOF host port：配置主從

SHOWLOG：顯示慢查詢日誌；

    TIME：返回當前時間；

SYNC：複製功能內建命令；

發佈與訂閱(publish/subscribe)
頻道：消息隊列

    SUBSCRIBE：訂閱一個或多個隊列；
PUBLISH：向頻道發佈消息；

UNSUBSCRIBE：退訂此前訂閱的頻道；

PSUBSCRIBE：模式訂閱

Redis的持久化：
RDB和AOF
RDB：snapshot，二進制格式，按事先定製的策略，週期性地將數據保存至磁盤，默認數據文件爲dump.rdb；
客戶端也可以顯示使用SAVE或BGSAVE命令啓動快照保存機制；
SAVE：同步，在主線程中保存快照，此時會阻塞所有客戶端請求；
BGSAVE：異步，
AOF：Append Only File
記錄每一次寫操作至指定的文件尾部實現持久化，當redis重啓時，可通過重新執行文件中的命令在內存中重建數據庫
BGREWRITEAOF：AOF文件重寫
不會讀取正在使用AOF文件，而通過將內存中的數據以命令的方式保存到臨時文件中，完成之後替換原來的AOF文件；

實驗環境：
系統版本：Centos6.5
內核版本：2.6.32-504.el6.x86_64
網卡1：VMnet0 172.16.100.6
網卡2：VMnet8 NAT DHCP

[root@CentOS6 ~]# wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-6.repo
[root@CentOS6 ~]# yum info redis
Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
Available Packages
Name        : redis
Arch        : x86_64
Version     : 3.2.12
Release     : 2.el6
Size        : 522 k
Repo        : epel
Summary     : A persistent key-value database
URL         : http://redis.io
License     : BSD
Description : Redis is an advanced key-value store. It is often referred to as a data
            : structure server since keys can contain strings, hashes, lists, sets and
            : sorted sets.
            :
            : You can run atomic operations on these types, like appending to a string;
            : incrementing the value in a hash; pushing to a list; computing set
            : intersection, union and difference; or getting the member with highest
            : ranking in a sorted set.
            :
            : In order to achieve its outstanding performance, Redis works with an
            : in-memory dataset. Depending on your use case, you can persist it either
            : by dumping the dataset to disk every once in a while, or by appending
            : each command to a log.
            :
            : Redis also supports trivial-to-setup master-slave replication, with very
            : fast non-blocking first synchronization, auto-reconnection on net split
            : and so forth.
            :
            : Other features include Transactions, Pub/Sub, Lua scripting, Keys with a
            : limited time-to-live, and configuration settings to make Redis behave like
            : a cache.
            :
            : You can use Redis from most programming languages also.
[root@CentOS6 ~]# yum -y install redis
[root@CentOS6 ~]# cp /etc/redis.conf{,.orgi}
[root@CentOS6 ~]# vim /etc/redis.conf
bind 127.0.0.1 172.16.100.6
[root@CentOS6 ~]# service redis start
[root@CentOS6 ~]# ss -tnl
State       Recv-Q Send-Q                                 Local Address:Port                                   Peer Address:Port
LISTEN      0      128                                               :::22                                               :::*
LISTEN      0      128                                                *:22                                                *:*
LISTEN      0      128                                        127.0.0.1:6379                                              *:*
[root@CentOS6 ~]# redis-cli -h 172.16.100.6
172.16.100.6:6379> help

172.16.100.6:6379> help @STRING

172.16.100.6:6379> HELP APPEND

172.16.100.6:6379> CLIENT LIST    #查看連接客戶端
id=2 addr=172.16.100.6:59420 fd=6 name= age=228 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=client

172.16.100.6:6379> SELECT 1    #打開1號數據庫，默認16個數據庫

172.16.100.6:6379[1]> SELECT 0    #默認爲0號數據庫

172.16.100.6:6379> HELP SET

  SET key value [EX seconds] [PX milliseconds] [NX|XX]
  summary: Set the string value of a key
  since: 1.0.0
  group: string

172.16.100.6:6379> SET disto fedora
OK
172.16.100.6:6379> GET disto
"fedora"
172.16.100.6:6379> SET disto centos
OK
172.16.100.6:6379> append disto slackware
(integer) 15
172.16.100.6:6379> GET disto
"centosslackware"
172.16.100.6:6379> STRLEN disto
(integer) 15
172.16.100.6:6379> SET count 0
OK
172.16.100.6:6379> INCR count
(integer) 1
172.16.100.6:6379> INCR count
(integer) 2
172.16.100.6:6379> DECR count
(integer) 1
172.16.100.6:6379> DECR count
(integer) 0
172.16.100.6:6379> SET disto gentoo NX
(nil)    #未能執行操作
172.16.100.6:6379> GET disto
"centosslackware"    #鍵不存在時才設置新值
172.16.100.6:6379> SET foo bar XX    #鍵存在才設置新值
(nil)
172.16.100.6:6379> set foo bar
OK
172.16.100.6:6379> incr foo    #只能對整數值操作
(error) ERR value is not an integer or out of range

172.16.100.6:6379> HELP @LIST

172.16.100.6:6379> LPUSH l1 mon
(integer) 1
172.16.100.6:6379> LINDEX l1 0
"mon"
172.16.100.6:6379> LPUSH l1 sun
(integer) 2
172.16.100.6:6379> LINDEX l1 0
"sun"
172.16.100.6:6379> RPUSH l1 tue
(integer) 3
172.16.100.6:6379> LINDEX l1 2
"tue"
172.16.100.6:6379> LSET l1 1 fri
OK
172.16.100.6:6379> LINDEX l1 1
"fri"
172.16.100.6:6379> RPOP l1
"tue"
172.16.100.6:6379> RPOP l1
"fri"
172.16.100.6:6379> RPOP l1
"sun"
172.16.100.6:6379> RPOP l1
(nil)

172.16.100.6:6379> HELP @SET

172.16.100.6:6379> SADD w1 mon tue wed thu fre sat sun
(integer) 7
172.16.100.6:6379> SADD w2 tue thu day
(integer) 3
172.16.100.6:6379> SINTER w1 w2
1) "thu"
2) "tue"
172.16.100.6:6379> SUNION w1 w2
1) "wed"
2) "tue"
3) "thu"
4) "day"
5) "mon"
6) "sun"
7) "fre"
8) "sat"
172.16.100.6:6379> SPOP w1
"mon"
172.16.100.6:6379> SPOP w1
"wed"
172.16.100.6:6379> SISMEMBER w1 wed
(integer) 0
172.16.100.6:6379> SISMEMBER w1 tue
(integer) 1    #1代表元素存在

172.16.100.6:6379> HELP @SORTED_SET

172.16.100.6:6379> ZADD weekday1 1 mon 2 tue 3 wed
(integer) 3
172.16.100.6:6379> ZCARD weekday1
(integer) 3
172.16.100.6:6379> ZRANK weekday1 tue
(integer) 1
172.16.100.6:6379> ZRANK weekday1 wed
(integer) 2
172.16.100.6:6379> ZRANK weekday1 mon
(integer) 0
172.16.100.6:6379> ZSCORE weekday1 tue
"2"
172.16.100.6:6379> ZRANGE weekday1 0 2
1) "mon"
2) "tue"
3) "wed"
172.16.100.6:6379> ZRANGE weekday1 0 1
1) "mon"
2) "tue"

172.16.100.6:6379> HELP @HASH

172.16.100.6:6379> HSET h1 a mon
(integer) 1
172.16.100.6:6379> HGET h1 a
"mon"
172.16.100.6:6379> HSET h1 b tue
(integer) 1
172.16.100.6:6379> HGET h1 a
"mon"
172.16.100.6:6379> HGET h1 b
"tue"
172.16.100.6:6379> HKEYS h1
1) "a"
2) "b"
172.16.100.6:6379> HVALS h1
1) "mon"
2) "tue"
172.16.100.6:6379> HLEN h1
(integer) 2

[root@CentOS6 ~]# vim /etc/redis.conf
requirepass smoke520    #啓用密碼
[root@CentOS6 ~]# service redis restart

[root@CentOS6 ~]# redis-cli -h 172.16.100.6
172.16.100.6:6379> SELECT 0
(error) NOAUTH Authentication required.
172.16.100.6:6379> AUTH smoke520    #認證
OK
172.16.100.6:6379> SELECT 0
OK

[root@CentOS6 ~]# redis-cli -h 172.16.100.6
172.16.100.6:6379> FLUSHDB
OK
172.16.100.6:6379> MULTI    #執行事務
OK
172.16.100.6:6379> SET ip 192.168.1.1
QUEUED
172.16.100.6:6379> SET port 8080
QUEUED
172.16.100.6:6379> GET port
QUEUED
172.16.100.6:6379> EXEC
1) OK
2) "192.168.1.1"
3) OK
4) "8080"
172.16.100.6:6379> WATCH ip
OK

打開一個新的終端連接redis；

[root@CentOS6 ~]# redis-cli
127.0.0.1:6379> GET ip
"192.168.1.1"
127.0.0.1:6379> SET ip 172.16.100.99
OK
127.0.0.1:6379>

在原來的終端連接的redis；

172.16.100.6:6379> EXEC
(nil)    #事務執行失敗；
127.0.0.1:6379> MULTI
OK
127.0.0.1:6379> GET ip
QUEUED
127.0.0.1:6379> SET port 5379
QUEUED
127.0.0.1:6379> SETTTT
(error) ERR unknown command 'SETTTT'
127.0.0.1:6379> EXEC
(error) EXECABORT Transaction discarded because of previous errors.
127.0.0.1:6379> HELP PING

128.127.0.0.1:6379> PING
PONG
127.0.0.1:6379> help ECHO

  ECHO message
  summary: Echo the given string
  since: 1.0.0
  group: connection
127.0.0.1:6379> ECHO "hello redis"
"hello redis"
127.0.0.1:6379> HELP @connection
127.0.0.1:6379> help @server
127.0.0.1:6379> CLIENT GETNAME
(nil)
127.0.0.1:6379> CLIENT SETNAME localconn
OK
127.0.0.1:6379> CLIENT GETNAME
"localconn"
127.0.0.1:6379> HELP INFO
127.0.0.1:6379> INFO    #獲取服務器配置信息
# Server
redis_version:3.2.12
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:b2d74fe5fff7657d
redis_mode:standalone
os:Linux 2.6.32-504.el6.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
gcc_version:4.4.7
process_id:33383
run_id:7b7fe366481f4c2bcc3cf3f0b4395ed8b65a7abd
tcp_port:6379
uptime_in_seconds:11849
uptime_in_days:0
hz:10
lru_clock:3706298
executable:/usr/bin/redis-server
config_file:/etc/redis.conf

# Clients
connected_clients:3
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0

# Memory
used_memory:854480
used_memory_human:834.45K
used_memory_rss:5447680
used_memory_rss_human:5.20M
used_memory_peak:854824
used_memory_peak_human:834.79K
total_system_memory:1952243712
total_system_memory_human:1.82G
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
mem_fragmentation_ratio:6.38
mem_allocator:jemalloc-3.6.0

# Persistence
loading:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1563976945
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:0
rdb_current_bgsave_time_sec:-1
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok

# Stats
total_connections_received:3
total_commands_processed:22
instantaneous_ops_per_sec:0
total_net_input_bytes:644
total_net_output_bytes:30094
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
evicted_keys:0
keyspace_hits:3
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:2020
migrate_cached_sockets:0

# Replication
role:master
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

# CPU
used_cpu_sys:15.04
used_cpu_user:7.74
used_cpu_sys_children:0.57
used_cpu_user_children:0.00

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=2,expires=0,avg_ttl=0
127.0.0.1:6379> DBSIZE
(integer) 2
127.0.0.1:6379> monitor
OK    #使用quit退出；
127.0.0.1:6379> HELP SUBSCRIBE
127.0.0.1:6379> help @pubsub
127.0.0.1:6379> SUBSCRIBE news    #訂閱消息
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "news"
3) (integer) 1

打開一個新的終端連接redis；

127.0.0.1:6379> PUBLISH news hello
(integer) 1

在原來的終端查看；

127.0.0.1:6379> SUBSCRIBE news
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "news"
3) (integer) 1
1) "message"
2) "news"
3) "hello"

繼續發送；

127.0.0.1:6379> PUBLISH news redis
(integer) 1

在原來的終端查看；

127.0.0.1:6379> SUBSCRIBE news
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "news"
3) (integer) 1
1) "message"
2) "news"
3) "hello"
1) "message"
2) "news"
3) "redis"

[root@CentOS6 ~]# redis-cli
127.0.0.1:6379> PSUBSCRIBE "news.i[to]" #訂閱多個頻道
Reading messages... (press Ctrl-C to quit)
1) "psubscribe"
2) "news.i[to]"
3) (integer) 1
1) "pmessage"
2) "news.i[to]"
3) "news.io"
4) "hello"

打開一個新的終端連接redis；
127.0.0.1:6379> PUBLISH news.io hello
(integer) 1

在原來的終端查看；
127.0.0.1:6379> PSUBSCRIBE "news.i[to]"
Reading messages... (press Ctrl-C to quit)
1) "psubscribe"
2) "news.i[to]"
3) (integer) 1
1) "pmessage"
2) "news.i[to]"
3) "news.io"
4) "hello"

繼續發送；
127.0.0.1:6379> PUBLISH news.it redis
(integer) 1

在原來的終端查看；
127.0.0.1:6379> PSUBSCRIBE "news.i[to]"
Reading messages... (press Ctrl-C to quit)
1) "psubscribe"
2) "news.i[to]"
3) (integer) 1
1) "pmessage"
2) "news.i[to]"
3) "news.io"
4) "hello"
1) "pmessage"
2) "news.i[to]"
3) "news.it"
4) "redis"