redis — 如何將redis內存使用量壓縮一半（四）

1.背景說明

redis存儲有固定內存，如果以某種方式達到其內存極限，我們的系統將開始出現故障，Redis內存使用量可能會成爲瓶頸。

使用最新版本的Spring Boot，有兩個主要依賴項- Spring Boot Web和Spring Data Reactive Redis，Spring Data Reactive Redis將用於連接和使用Redis的內部應用程序。從本質上講，Redis依賴項默認使用Lettuce Redis客戶端，並且受最新版本的Spring Boot支持。

2.MessagePack

降低內存使用量使用MessagePack，MessagePack工作原理可以參照官網說明MsgPack spec.md，其特徵以官方msgpack官網用一句話總結：It’s like JSON.but fast and small。

引入依賴，在pom.xml文件中添加以下依賴項：

<!--redis內存壓縮-->
<dependency>
    <groupId>org.msgpack</groupId>
    <artifactId>msgpack-core</artifactId>
    <version>0.8.20</version>
</dependency>
<dependency>
    <groupId>org.msgpack</groupId>
    <artifactId>jackson-dataformat-msgpack</artifactId>
    <version>0.8.20</version>
</dependency>

創建了一個名爲控制器MsgPackController：

class MsgPackRedisSerializer<T> implements RedisSerializer<T> {
    public static final Charset DEFAULT_CHARSET;
    private final JavaType javaType;
    private ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory())
            .registerModules(new Jdk8Module(), new JavaTimeModule())
            .configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, true)
            .configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false)
            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
            .setSerializationInclusion(JsonInclude.Include.NON_NULL);

    public MsgPackRedisSerializer(Class<T> type) {
        this.javaType = JavaTypeHandler.getJavaType(type);
    }

    public T deserialize(@Nullable byte[] bytes) throws SerializationException {
        if (bytes == null || bytes.length == 0) {
            return null;
        } else {
            try {
                return this.objectMapper.readValue(bytes, 0, bytes.length, this.javaType);
            } catch (Exception ex) {
                throw new SerializationException("Could not read MsgPack JSON: " + ex.getMessage(), ex);
            }
        }
    }

    public byte[] serialize(@Nullable Object value) throws SerializationException {
        if (value == null) {
            return new byte[0];
        } else {
            try {
                return this.objectMapper.writeValueAsBytes(value);
            } catch (Exception ex) {
                throw new SerializationException("Could not write MsgPack JSON: " + ex.getMessage(), ex);
            }
        }
    }

    static {
        DEFAULT_CHARSET = StandardCharsets.UTF_8;
    }

}

實例MessagePackFactory被傳遞到中ObjectMapper。這將充當Redis和我們的Spring Boot應用程序之間數據的二進制格式和字符串格式之間的橋樑。

3.壓縮

比較dataset.bytes當前內存與先前記錄的內存，使用率將減少一半，我們可以進一步減少它。

• Snappy壓縮算法

它不旨在最大程度地壓縮，也不旨在與任何其他壓縮庫兼容。相反，它的目標是非常高的速度和合理的壓縮。

使用Snappy就像在中添加依賴項一樣簡單pom.xml，並且幾行代碼更改。只需Snappy.compress在序列化和Snappy.decompress反序列化時添加即可：

<dependency>
    <groupId>org.xerial.snappy</groupId>
    <artifactId>snappy-java</artifactId>
    <version>1.1.7.3</version>
</dependency>

配置redis序列化配置類：

public class SnappyMsgPackRedisSerializer<T> implements RedisSerializer<T> {
    public static final Charset DEFAULT_CHARSET;
    private final JavaType javaType;
    private ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory())
            .registerModules(new Jdk8Module(), new JavaTimeModule())
            .configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, true)
            .configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false)
            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
            .setSerializationInclusion(JsonInclude.Include.NON_NULL);

    public SnappyMsgPackRedisSerializer(Class<T> type) {
        this.javaType = JavaTypeHandler.getJavaType(type);
    }

    @Override
    public T deserialize(@Nullable byte[] bytes) throws SerializationException {
        if (bytes == null || bytes.length == 0) {
            return null;
        } else {
            try {
                final byte[] uncompressBytes = Snappy.uncompress(bytes); //解壓
                return this.objectMapper.readValue(uncompressBytes, 0, uncompressBytes.length, this.javaType);
            } catch (Exception ex) {
                throw new SerializationException("Could not read MsgPack JSON: " + ex.getMessage(), ex);
            }
        }
    }

    @Override
    public byte[] serialize(@Nullable Object value) throws SerializationException {
        if (value == null) {
            return new byte[0];
        } else {
            try {
                final byte[] bytes = this.objectMapper.writeValueAsBytes(value);
                return Snappy.compress(bytes); //壓縮
            } catch (Exception ex) {
                throw new SerializationException("Could not write MsgPack JSON: " + ex.getMessage(), ex);
            }
        }
    }

    static {
        DEFAULT_CHARSET = StandardCharsets.UTF_8;
    }
}