spring cloud config 工作原理
1. spring cloud config 測試案例
說明：爲方便起見，這裏使用本地文件(native)作數據後端,Spring cloud config 支持的數據後端參考：EnvironmentRepositoryConfiguration
1.0 spring cloud config server
啓動類：ConfigServerApplication
// 開啓Spring Config Server 功能，主要爲引入Server自動化配置開啓服務器功能：ConfigServerAutoConfiguration
// 對外開放http功能(ConfigServerMvcConfiguration)：EnvironmentController、ResourceController
@EnableConfigServer
@SpringBootApplication
public class ConfigServerApplication {
public static void main(String[] args) {
SpringApplication.run(ConfigServerApplication.class, args);
}
}
配置服務器配置：bootstrap.yml
server:
port: 8080
spring:
application:
name: local-config-server
profiles:
# NativeRepositoryConfiguration 加載本地環境(git/subversion/jdbc等)
active: native
cloud:
config:
server:
native:
# 注意： 新版本本地文件目錄結尾要加 /
search-locations: classpath:/configdata/
# 版本號
version: V1.0.0
# label 缺省默認配置目錄
# default-label: common
樣例測試文件：classpath:/configdata/
application-common.properties:
common.config=application common config
server.port=8088
application-private.properties:
private.config=application private config
server.port=8089
項目結構圖：
1.1 spring cloud config client
啓動類：
@SpringBootApplication
public class ConfigClientApplication {
public static void main(String[] args) {
SpringApplication.run(ConfigClientApplication.class, args);
}
}
控制器：
@RestController
public class ConfigController {
@Autowired
private Environment environment;
@Value("${common.config:common}")
private String commonConfig;
@Value("${private.config:private}")
private String privateConfig;
@GetMapping("/common")
public String commonConfig(){
return commonConfig;
}
@GetMapping("/private")
public String privateConfig(){
return privateConfig;
}
@GetMapping("/getConfig/{env}")
public String getConfig(@PathVariable("env") String env){
return environment.getProperty(env);
}
}
配置文件：bootstrap.yml
spring:
application:
name: local-config-client
cloud:
config:
# 配置名稱：自定義 > ${spring.application.name} > application(優先級順序)
name: ${spring.application.name}
# ConfigClientProperties構造時默認spring.profiles.active
# 注意：後面覆蓋前面的配置
profile: common,private
# 標籤：git 默認master, file: 指定項目配置目錄，一般默認項目名，存在服務器配置目錄根目錄下
label: ${spring.application.name}-config
## 配置中心服務器地址
uri: http://localhost:8080
1.2 測試效果
spring cloud config server直接訪問： http://localhost:8080/local-config-client/common,private/local-config-client-config
{
"name": "local-config-client",
"profiles": [
"common,private"
],
"label": "local-config-client-config",
"version": "V1.0.0",
"state": null,
"propertySources": [
{
"name": "classpath:/configdata/local-config-client-config/application-private.properties",
"source": {
"private.config": "application private config",
"server.port": "8089"
}
},
{
"name": "classpath:/configdata/application-common.properties",
"source": {
"common.config": "application common config",
"server.port": "8088"
}
}
]
}
spring cloud config client 訪問：
私有配置訪問：
GET http://localhost:8089/private
HTTP/1.1 200
Content-Type: text/plain;charset=UTF-8
Content-Length: 26
Date: Mon, 27 Jul 2020 00:59:19 GMT
application private config
公共配置訪問：
GET http://localhost:8089/common
HTTP/1.1 200
Content-Type: text/plain;charset=UTF-8
Content-Length: 25
Date: Mon, 27 Jul 2020 01:00:19 GMT
application common config
Response code: 200; Time: 24ms; Content length: 25 bytes
重疊配置訪問：取決於profiles順序，後面優先級 > 前面，底層使用LinkedHashSet保證有序
GET http://localhost:8089/getConfig/server.port
HTTP/1.1 200
Content-Disposition: inline;filename=f.txt
Content-Type: text/plain;charset=UTF-8
Content-Length: 4
Date: Mon, 27 Jul 2020 01:01:18 GMT
8089
2. spring cloud config client 工作原理
原理：PropertySourceLocator：spring cloud擴展屬性源，用於自定義加載環境屬性源，常用配置中心基本都是基於PropertySourceLocator實現，例如：NACOS, 詳見博客：nacos 分佈式配置中心工作原理源碼分析
ConfigServicePropertySourceLocator: spring cloud client屬性源加載器，本質通過Http請求從配置服務器獲取
public class ConfigServicePropertySourceLocator implements PropertySourceLocator {
private static Log logger = LogFactory.getLog(ConfigServicePropertySourceLocator.class);
private RestTemplate restTemplate;
private ConfigClientProperties defaultProperties;
// ConfigServicePropertySourceLocator 使用 ConfigClientProperties 客戶端屬性初始化
public ConfigServicePropertySourceLocator(ConfigClientProperties defaultProperties) {
this.defaultProperties = defaultProperties;
}
@Override
@Retryable(interceptor = "configServerRetryInterceptor")
public org.springframework.core.env.PropertySource<?> locate(org.springframework.core.env.Environment environment) {
// 配置覆蓋：spring cloud config client私有配置由於spring boot全局配置（name：應用名、profile：生效環境、label: 標籤）
ConfigClientProperties properties = this.defaultProperties.override(environment);
// 創建組合屬性源
CompositePropertySource composite = new OriginTrackedCompositePropertySource("configService");
// 創建Http客戶端RestTemplate， 支持用戶名/密碼（Http Basic認證）、Token訪問，賬戶信息配置在ConfigClientProperties
RestTemplate restTemplate = this.restTemplate == null ? getSecureRestTemplate(properties) : this.restTemplate;
Exception error = null;
String errorBody = null;
try {
String[] labels = new String[] { "" };
if (StringUtils.hasText(properties.getLabel())) {
labels = StringUtils
.commaDelimitedListToStringArray(properties.getLabel());
}
// 獲取狀態信息，主要用於存儲當前配置的狀態，用於後續版本匹配
String state = ConfigClientStateHolder.getState();
// Try all the labels until one works
for (String label : labels) {
// 從spring cloud config server 獲取最新配置信息
Environment result = getRemoteEnvironment(restTemplate, properties, label.trim(), state);
if (result != null) {
log(result); // 服務端響應配置信息打印
// result.getPropertySources() can be null if using xml
if (result.getPropertySources() != null) {
for (PropertySource source : result.getPropertySources()) {
@SuppressWarnings("unchecked")
// 進行Origin值對象處理，滿足條件則保存現值和原始值
Map<String, Object> map = translateOrigins(source.getName(),(Map<String, Object>) source.getSource());
composite.addPropertySource(new OriginTrackedMapPropertySource(source.getName(),map));
}
}
// 若服務端有訪問state（Vault 配置存儲）或version（SCM 系統數據存儲，例如：git、svn等）字段，則新增PropertySource，用於維護這2種狀態
if (StringUtils.hasText(result.getState()) || StringUtils.hasText(result.getVersion())) {
HashMap<String, Object> map = new HashMap<>();
putValue(map, "config.client.state", result.getState());
putValue(map, "config.client.version", result.getVersion());
composite.addFirstPropertySource(new MapPropertySource("configClient", map));
}
return composite;
}
}
errorBody = String.format("None of labels %s found", Arrays.toString(labels));
}
catch (HttpServerErrorException e) {
error = e;
if (MediaType.APPLICATION_JSON
.includes(e.getResponseHeaders().getContentType())) {
errorBody = e.getResponseBodyAsString();
}
}
catch (Exception e) {
error = e;
}
if (properties.isFailFast()) {
throw new IllegalStateException(
"Could not locate PropertySource and the fail fast property is set, failing"
+ (errorBody == null ? "" : ": " + errorBody),
error);
}
logger.warn("Could not locate PropertySource: "
+ (error != null ? error.getMessage() : errorBody));
return null;
}
private void log(Environment result) {
if (logger.isInfoEnabled()) {
logger.info(String.format(
"Located environment: name=%s, profiles=%s, label=%s, version=%s, state=%s",
result.getName(),
result.getProfiles() == null ? ""
: Arrays.asList(result.getProfiles()),
result.getLabel(), result.getVersion(), result.getState()));
}
if (logger.isDebugEnabled()) {
List<PropertySource> propertySourceList = result.getPropertySources();
if (propertySourceList != null) {
int propertyCount = 0;
for (PropertySource propertySource : propertySourceList) {
propertyCount += propertySource.getSource().size();
}
logger.debug(String.format(
"Environment %s has %d property sources with %d properties.",
result.getName(), result.getPropertySources().size(),
propertyCount));
}
}
}
private Map<String, Object> translateOrigins(String name, Map<String, Object> source) {
Map<String, Object> withOrigins = new HashMap<>();
for (Map.Entry<String, Object> entry : source.entrySet()) {
boolean hasOrigin = false;
if (entry.getValue() instanceof Map) {
@SuppressWarnings("unchecked")
Map<String, Object> value = (Map<String, Object>) entry.getValue();
if (value.size() == 2 && value.containsKey("origin") && value.containsKey("value")) {
Origin origin = new ConfigServiceOrigin(name, value.get("origin"));
OriginTrackedValue trackedValue = OriginTrackedValue.of(value.get("value"), origin);
withOrigins.put(entry.getKey(), trackedValue);
hasOrigin = true;
}
}
if (!hasOrigin) {
withOrigins.put(entry.getKey(), entry.getValue());
}
}
return withOrigins;
}
private void putValue(HashMap<String, Object> map, String key, String value) {
if (StringUtils.hasText(value)) {
map.put(key, value);
}
}
// 重點： 從 spring cloud config server 獲取配置
private Environment getRemoteEnvironment(RestTemplate restTemplate, ConfigClientProperties properties, String label, String state) {
String path = "/{name}/{profile}";
String name = properties.getName();
String profile = properties.getProfile();
String token = properties.getToken();
int noOfUrls = properties.getUri().length;
if (noOfUrls > 1) {
logger.info("Multiple Config Server Urls found listed.");
}
Object[] args = new String[] { name, profile };
if (StringUtils.hasText(label)) {
if (label.contains("/")) {
label = label.replace("/", "(_)");
}
args = new String[] { name, profile, label };
path = path + "/{label}";
}
ResponseEntity<Environment> response = null;
for (int i = 0; i < noOfUrls; i++) {
Credentials credentials = properties.getCredentials(i);
String uri = credentials.getUri();
String username = credentials.getUsername();
String password = credentials.getPassword();
logger.info("Fetching config from server at : " + uri);
try {
HttpHeaders headers = new HttpHeaders();
//設置請求頭對應spring cloud config server： EnvironmentController.labelledIncludeOrigin
headers.setAccept(Collections.singletonList(MediaType.parseMediaType(V2_JSON)));
// 若spring cloud config server 需安全認證，用戶信息保存在請求頭中
addAuthorizationToken(properties, headers, username, password);
if (StringUtils.hasText(token)) {
// 請求頭添加Token
headers.add(TOKEN_HEADER, token);
}
if (StringUtils.hasText(state) && properties.isSendState()) {
// 發送當前配置狀態信息
headers.add(STATE_HEADER, state);
}
final HttpEntity<Void> entity = new HttpEntity<>((Void) null, headers);
// 發送Http請求獲取配置：http://localhost:8080/local-config-client/common,private/local-config-client-config
response = restTemplate.exchange(uri + path, HttpMethod.GET, entity,Environment.class, args);
}
catch (HttpClientErrorException e) {
if (e.getStatusCode() != HttpStatus.NOT_FOUND) {
throw e;
}
}
catch (ResourceAccessException e) {
logger.info("Connect Timeout Exception on Url - " + uri
+ ". Will be trying the next url if available");
if (i == noOfUrls - 1) {
throw e;
}
else {
continue;
}
}
if (response == null || response.getStatusCode() != HttpStatus.OK) {
return null;
}
Environment result = response.getBody();
return result;
}
return null;
}
public void setRestTemplate(RestTemplate restTemplate) {
this.restTemplate = restTemplate;
}
// 創建Http客戶端（RestTemplate）獲取配置數據
private RestTemplate getSecureRestTemplate(ConfigClientProperties client) {
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
if (client.getRequestReadTimeout() < 0) {
throw new IllegalStateException("Invalid Value for Read Timeout set.");
}
if (client.getRequestConnectTimeout() < 0) {
throw new IllegalStateException("Invalid Value for Connect Timeout set.");
}
requestFactory.setReadTimeout(client.getRequestReadTimeout());
requestFactory.setConnectTimeout(client.getRequestConnectTimeout());
RestTemplate template = new RestTemplate(requestFactory);
Map<String, String> headers = new HashMap<>(client.getHeaders());
if (headers.containsKey(AUTHORIZATION)) {
headers.remove(AUTHORIZATION); // To avoid redundant addition of header
}
// 自定義 RestTemplate 攔截器，這裏主要給Http請求額外增加自定義請求頭信息
if (!headers.isEmpty()) {
template.setInterceptors(Arrays.<ClientHttpRequestInterceptor>asList(new GenericRequestHeaderInterceptor(headers)));
}
return template;
}
// 添加認證信息：支持Http Basic認證和Token認證(Authorization)
private void addAuthorizationToken(ConfigClientProperties configClientProperties,
HttpHeaders httpHeaders, String username, String password) {
String authorization = configClientProperties.getHeaders().get(AUTHORIZATION);
if (password != null && authorization != null) {
throw new IllegalStateException(
"You must set either 'password' or 'authorization'");
}
if (password != null) {
byte[] token = Base64Utils.encode((username + ":" + password).getBytes());
httpHeaders.add("Authorization", "Basic " + new String(token));
}
else if (authorization != null) {
httpHeaders.add("Authorization", authorization);
}
}
/**
* Adds the provided headers to the request.
* RestTemplate 自定義攔截器，主要用於請求添加請求頭信息，工作原理類似J2EE中的過濾器：Filter
*/
public static class GenericRequestHeaderInterceptor implements ClientHttpRequestInterceptor {
private final Map<String, String> headers;
public GenericRequestHeaderInterceptor(Map<String, String> headers) {
this.headers = headers;
}
@Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
for (Entry<String, String> header : this.headers.entrySet()) {
request.getHeaders().add(header.getKey(), header.getValue());
}
return execution.execute(request, body);
}
protected Map<String, String> getHeaders() {
return this.headers;
}
}
static class ConfigServiceOrigin implements Origin {
private final String remotePropertySource;
private final Object origin;
ConfigServiceOrigin(String remotePropertySource, Object origin) {
this.remotePropertySource = remotePropertySource;
Assert.notNull(origin, "origin may not be null");
this.origin = origin;
}
@Override
public String toString() {
return "Config Server " + this.remotePropertySource + ":"
+ this.origin.toString();
}
}
}
2.1 spring cloud config client 補充內容
自動配置：ConfigClientAutoConfiguration
ConfigServerHealthIndicator: 配置服務器檢查指示器
org.springframework.cloud.config.client.ConfigServerHealthIndicator.doHealthCheck
@Override
protected void doHealthCheck(Builder builder) throws Exception {
// 本質調用 ConfigServicePropertySourceLocator 從新拉取一下遠程配置旁段檢查狀態
PropertySource<?> propertySource = getPropertySource();
builder.up();
if (propertySource instanceof CompositePropertySource) {
List<String> sources = new ArrayList<>();
for (PropertySource<?> ps : ((CompositePropertySource) propertySource)
.getPropertySources()) {
sources.add(ps.getName());
}
builder.withDetail("propertySources", sources);
}
else if (propertySource != null) {
builder.withDetail("propertySources", propertySource.toString());
}
else {
builder.unknown().withDetail("error", "no property sources located");
}
}
ConfigClientWatch： 客戶端監視器，定時檢查State是否更新，若更新則通過ContextRefresher更新配置
public class ConfigClientWatch implements Closeable, EnvironmentAware {
private static Log log = LogFactory.getLog(ConfigServicePropertySourceLocator.class);
private final AtomicBoolean running = new AtomicBoolean(false);
private final ContextRefresher refresher;
private Environment environment;
public ConfigClientWatch(ContextRefresher refresher) {
this.refresher = refresher;
}
@Override
public void setEnvironment(Environment environment) {
this.environment = environment;
}
@PostConstruct
public void start() {
this.running.compareAndSet(false, true);
}
// 定時任務刷新，主要要開啓定時任務需要藉助@EnableScheduling開啓@Scheduled掃描，定時spring task才生效
@Scheduled(initialDelayString = "${spring.cloud.config.watch.initialDelay:180000}", fixedDelayString = "${spring.cloud.config.watch.delay:500}")
public void watchConfigServer() {
if (this.running.get()) {
// 獲取最新配置狀態
String newState = this.environment.getProperty("config.client.state");
// 線程上下文持有狀態：舊狀態
String oldState = ConfigClientStateHolder.getState();
//狀態不一致說明配置更新，則ContextRefresher刷新環境：刷新配置 + 刷新RefreshScope Bean,注意，要想動態刷新，組件必須聲明 @RefreshScope，否則只更新Environment不更新Bean
if (stateChanged(oldState, newState)) {
ConfigClientStateHolder.setState(newState);
this.refresher.refresh();
}
}
}
/* for testing */ boolean stateChanged(String oldState, String newState) {
return (!hasText(oldState) && hasText(newState)) || (hasText(oldState) && !oldState.equals(newState));
}
@Override
public void close() {
this.running.compareAndSet(true, false);
}
}
特別注意：ConfigClientWatch 檢測environment.getProperty("config.client.state")與ConfigClientStateHolder存儲值是否相同來判斷配置是否更新，但是在native、git 不借助spring cloud bus 客戶端無法檢測服務端配置更新，根本無法更新本地Environment中的state，因此ConfigClientWatch在次環境基本無效,官方聲明該組件目前只適用於Vault作爲配置數據後端存儲
3. spring cloud config server 工作原理
3.1 獲取Spring cloud config client 環境配置基礎流程
接口：org.springframework.cloud.config.server.environment.EnvironmentController.getEnvironment
public Environment getEnvironment(String name, String profiles, String label, boolean includeOrigin) {
//spring clond config client: name、label 預處理
if (name != null && name.contains("(_)")) {
// "(_)" is uncommon in a git repo name, but "/" cannot be matched
// by Spring MVC
name = name.replace("(_)", "/");
}
if (label != null && label.contains("(_)")) {
// "(_)" is uncommon in a git branch name, but "/" cannot be matched
// by Spring MVC
label = label.replace("(_)", "/");
}
//org.springframework.cloud.config.server.config.ConfigServerMvcConfiguration.environmentController
// repository: EnvironmentEncryptorEnvironmentRepository裝飾 SearchPathCompositeEnvironmentRepository, 額外增加配置解密功能
// SearchPathCompositeEnvironmentRepository是 CompositeEnvironmentRepository 子類，這裏是Native則維護 NativeEnvironmentRepository
Environment environment = this.repository.findOne(name, profiles, label, includeOrigin);
if (!this.acceptEmpty && (environment == null || environment.getPropertySources().isEmpty())) {
throw new EnvironmentNotFoundException("Profile Not found");
}
return environment;
}
3.2 獲取配置 SearchPathCompositeEnvironmentRepository 父類 CompositeEnvironmentRepository
org.springframework.cloud.config.server.environment.CompositeEnvironmentRepository.findOne
@Override
public Environment findOne(String application, String profile, String label, boolean includeOrigin) {
Environment env = new Environment(application, new String[] { profile }, label, null, null);
// 一般只有1個後端作爲配置數據存儲
if (this.environmentRepositories.size() == 1) {
// 調用內部維護EnvironmentRepository獲取真正的配置：NativeEnvironmentRepository
Environment envRepo = this.environmentRepositories.get(0).findOne(application,profile, label, includeOrigin);
env.addAll(envRepo.getPropertySources());
env.setVersion(envRepo.getVersion());
env.setState(envRepo.getState());
}
else {
for (EnvironmentRepository repo : environmentRepositories) {
env.addAll(repo.findOne(application, profile, label, includeOrigin).getPropertySources());
}
}
return env;
}
3.3 底層獲取配置邏輯： NativeEnvironmentRepository
接口: org.springframework.cloud.config.server.environment.NativeEnvironmentRepository.findOne
工作原理：採用 SpringApplicationBuilder 採用配置路徑根路徑和標籤路徑(根路徑/標籤名)，以WebApplicationType.NONE形式啓動，相當於重新構建並啓動SpringApplication, 但是新啓動的IOC容器不對外提供服務，相當於是沙箱內運行的容器，這裏主要藉助SpringApplication更好進行環境參數處理，類似操作邏輯參考：ContextRefresher.addConfigFilesToEnvironment
public Environment findOne(String config, String profile, String label, boolean includeOrigin) {
SpringApplicationBuilder builder = new SpringApplicationBuilder(PropertyPlaceholderAutoConfiguration.class);
ConfigurableEnvironment environment = getEnvironment(profile);
builder.environment(environment);
builder.web(WebApplicationType.NONE).bannerMode(Mode.OFF);
if (!logger.isDebugEnabled()) {
// Make the mini-application startup less verbose
builder.logStartupInfo(false);
}
// SpringApplication 啓動參數配置
//--spring.config.name 應用名(sringcloud config 客戶端應用名)
//--spring.cloud.bootstrap.enabled=false: 取消 bootstrap.yaml 配置引入
//--spring.config.location： 指定SpringApplication啓動配置目錄: 配置根目錄、配置目錄/標籤目錄 列表
String[] args = getArgs(config, profile, label);
//配置 ConfigFileApplicationListener 用於 SpringApplication 加載配置文件
builder.application().setListeners(Arrays.asList(new ConfigFileApplicationListener()));
// SpringApplication 啓動
try (ConfigurableApplicationContext context = builder.run(args)) {
// 清除spring cloud config client 相關屬性源配置：spring.main.web-application-type、spring.main.web-application-type
environment.getPropertySources().remove("profiles");
// 清理spring自身屬性源配置，只保留spring cloud config client自身相關配置
// PassthruEnvironmentRepository.findOne：去除spring boot相關配置：PassthruEnvironmentRepository.standardSources
// 將 org.springframework.core.env.PropertySource 轉爲 org.springframework.cloud.config.environment.PropertySource 封裝到
// org.springframework.cloud.config.environment.Environment 返回
return clean(new PassthruEnvironmentRepository(environment).findOne(config, profile, label, includeOrigin));
}
catch (Exception e) {
String msg = String.format(
"Could not construct context for config=%s profile=%s label=%s includeOrigin=%b",
config, profile, label, includeOrigin);
String completeMessage = NestedExceptionUtils.buildMessage(msg,
NestedExceptionUtils.getMostSpecificCause(e));
throw new FailedToConstructEnvironmentException(completeMessage, e);
}
}
至此spring cloud config client/server 工作原理分析完成