字符串數據脫敏框架
- 可選擇唯品會的工具：https://github.com/vipshop/vjtools/blob/master/vjkit/docs/data_masking.md
- 也可以自定義，用正則或者其他方式（如：commons-lang3#StringUtils）
與logback集成
-
分析：
由於logback的api和語法多種多樣（http://logback.qos.ch/manual/architecture.html, 如下），導致在logback級別處理數據脫敏不太方便，爲了保持日誌框架原本自由的使用方式，所以在調用前處理日誌脫敏，且注意加上必要的日誌打印判斷提高性能。 -
logback基本語法：
String message = "This is a message.";
logger.info(message);
logger.info("This is a message");
String param = "some variable";
logger.info("This is a message. {}", param);
String param1 = "some variable1";
String param2 = "some variable2";
logger.info("This is a message. {} {}", param1, param2);
String params = {"some variable1", "some variable2"};
logger.info("This is a message. {} {}", params);
- 日誌脫敏使用方式：
@Test
public void testDataMask() {
String mobile = "13812345678";
if (logger.isInfoEnabled()) {
logger.info("用戶的手機號爲：{}", DataMask.mask(mobile, SensitiveType.Phone));
}
}
輸出結果：
2020-06-28 16:01:33.868 INFO --- [ main] c.m.c.util.log.DataMaskTests : 用戶的手機號爲：138*****678
logback框架擴展
測試用logback.xml：
<?xml version="1.0" encoding="UTF-8" ?>
<configuration>
<conversionRule conversionWord="clr" converterClass="org.springframework.boot.logging.logback.ColorConverter"/>
<conversionRule conversionWord="wex"
converterClass="org.springframework.boot.logging.logback.WhitespaceThrowableProxyConverter"/>
<conversionRule conversionWord="wEx"
converterClass="org.springframework.boot.logging.logback.ExtendedWhitespaceThrowableProxyConverter"/>
<conversionRule conversionWord="m" converterClass="com.caiya.common.util.log.JSONStringMessageConverter"/>
<property name="LOG_FILE_NAME" value="app"/>
<property name="FILE_PATTERN"
value="%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %m - %replace(%ex){'\n','---'} %n"/>
<property name="CONSOLE_LOG_PATTERN"
value="%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}"/>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>${CONSOLE_LOG_PATTERN}</pattern>
</encoder>
</appender>
<appender name="FILE"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<!--See also http://logback.qos.ch/manual/appenders.html#RollingFileAppender -->
<File>./logs/${LOG_FILE_NAME}.log</File>
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="com.caiya.common.util.log.TestPatternLayout">
<pattern>${FILE_PATTERN}</pattern>
</layout>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<fileNamePattern>./logs/archive/${LOG_FILE_NAME}-%d{yyyy-MM-dd}-%i.log.gz</fileNamePattern>
<maxHistory>180</maxHistory>
<totalSizeCap>30GB</totalSizeCap>
</rollingPolicy>
</appender>
<root level="debug">
<appender-ref ref="FILE"/>
<appender-ref ref="CONSOLE"/>
</root>
</configuration>
擴展一：使用自定義的MessageConverter
如下，使用fastjson序列化所有的變量：
package com.caiya.common.util.log;
import ch.qos.logback.classic.pattern.MessageConverter;
import ch.qos.logback.classic.spi.ILoggingEvent;
import com.alibaba.fastjson.JSON;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.helpers.MessageFormatter;
import java.util.stream.Stream;
/**
* 用fastjson轉換日誌參數爲json字符串
*
* @author wangnan
* @since 1.1.1, 2020/6/24
**/
public class JSONStringMessageConverter extends MessageConverter {
private static final Logger logger = LoggerFactory.getLogger(JSONStringMessageConverter.class);
@Override
public String convert(ILoggingEvent event) {
if (event.getArgumentArray() != null) {
try {
return MessageFormatter.arrayFormat(event.getMessage(), Stream.of(event.getArgumentArray()).map(JSON::toJSONString).toArray()).getMessage();
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
}
return event.getFormattedMessage();
}
}
擴展二：使用自定義的PatternLayout
如下，可自由實現對象數據的轉換：
package com.caiya.common.util.log;
import ch.qos.logback.classic.PatternLayout;
import ch.qos.logback.classic.pattern.MessageConverter;
import ch.qos.logback.classic.spi.ILoggingEvent;
/**
* TestPatternLayout.
*
* @author wangnan
* @since 1.1.1, 2020/6/24
**/
public class TestPatternLayout extends PatternLayout {
@Override
public String doLayout(ILoggingEvent event) {
# TODO 這裏擴展你想輸出的內容形式
return super.doLayout(event);# 默認返回
}
}