Spring Cloud Consul外部配置動態刷新源碼解析

Consul

Consul是一款中間件，可提供KV存儲功能，同時提供了創建、修改、查詢KV存儲的HTTP API，因此可作爲配置中心。

Spring Cloud Consul

Spring Cloud Consul是基於Spring Cloud的公共接口，提供了集成Consul配置能力的微服務框架。

源碼分析

ConfigWatch

這個類中提供了一個定時任務

    @Override
	public void start() {
		if (this.running.compareAndSet(false, true)) {
			this.watchFuture = this.taskScheduler.scheduleWithFixedDelay(
					this::watchConfigKeyValues, this.properties.getWatch().getDelay());
		}
	}

該方法定時執行（getDelay()：每當上一次執行完成後，經過固定延遲時間開始下一次執行）watchConfigKeyValues方法。

    @Timed("consul.watch-config-keys")
	public void watchConfigKeyValues() {
		if (this.running.get()) {
			for (String context : this.consulIndexes.keySet()) {

				// turn the context into a Consul folder path (unless our config format
				// are FILES)
				if (this.properties.getFormat() != FILES && !context.endsWith("/")) {
					context = context + "/";
				}

				try {
					Long currentIndex = this.consulIndexes.get(context);
					if (currentIndex == null) {
						currentIndex = -1L;
					}

					log.trace("watching consul for context '" + context + "' with index "
							+ currentIndex);

					// use the consul ACL token if found
					String aclToken = this.properties.getAclToken();
					if (StringUtils.isEmpty(aclToken)) {
						aclToken = null;
					}
					// 調用consul的HTTP API，獲取配置
					Response<List<GetValue>> response = this.consul.getKVValues(context,
							aclToken,
							new QueryParams(this.properties.getWatch().getWaitTime(),
									currentIndex));

					// if response.value == null, response was a 404, otherwise it was a
					// 200
					// reducing churn if there wasn't anything
					if (response.getValue() != null && !response.getValue().isEmpty()) {
						Long newIndex = response.getConsulIndex();
						// currentIndex是緩存了KV配置的上一次更新的版本號，newIndex是當前版本號，若兩者不等，則說明配置已更新，所以需要刷新
						if (newIndex != null && !newIndex.equals(currentIndex)) {
							// don't publish the same index again, don't publish the first
							// time (-1) so index can be primed
							if (!this.consulIndexes.containsValue(newIndex)
									&& !currentIndex.equals(-1L)) {
								log.trace("Context " + context + " has new index "
										+ newIndex);
								RefreshEventData data = new RefreshEventData(context,
										currentIndex, newIndex);
								// 通過Spring的事件機制觸發配置動態刷新
								this.publisher.publishEvent(
										new RefreshEvent(this, data, data.toString()));
							}
							else if (log.isTraceEnabled()) {
								log.trace("Event for index already published for context "
										+ context);
							}
							this.consulIndexes.put(context, newIndex);
						}
						else if (log.isTraceEnabled()) {
							log.trace("Same index for context " + context);
						}
					}
					else if (log.isTraceEnabled()) {
						log.trace("No value for context " + context);
					}

				}
				catch (Exception e) {
					// only fail fast on the initial query, otherwise just log the error
					if (this.firstTime && this.properties.isFailFast()) {
						log.error(
								"Fail fast is set and there was an error reading configuration from consul.");
						ReflectionUtils.rethrowRuntimeException(e);
					}
					else if (log.isTraceEnabled()) {
						log.trace("Error querying consul Key/Values for context '"
								+ context + "'", e);
					}
					else if (log.isWarnEnabled()) {
						// simplified one line log message in the event of an agent
						// failure
						log.warn("Error querying consul Key/Values for context '"
								+ context + "'. Message: " + e.getMessage());
					}
				}
			}
		}
		this.firstTime = false;
	}

重要的代碼行我已經加了中文註釋。主要就是調用Consul的HTTP API獲取配置，同時通過版本號判斷配置是否更新，如果更新則通過事件機制，發佈RefreshEvent事件，觸發本地配置刷新。

RefreshEventListener

這個類實現了SmartApplicationListener接口，SmartApplicationListener接口繼承了ApplicationListener接口，因此會消費RefreshEvent。

    @Override
	public void onApplicationEvent(ApplicationEvent event) {
		if (event instanceof ApplicationReadyEvent) {
			handle((ApplicationReadyEvent) event);
		}
		else if (event instanceof RefreshEvent) {
			handle((RefreshEvent) event);
		}
	}

handle方法

    public void handle(RefreshEvent event) {
		if (this.ready.get()) { // don't handle events before app is ready
			log.debug("Event received " + event.getEventDesc());
			Set<String> keys = this.refresh.refresh();
			log.info("Refresh keys changed: " + keys);
		}
	}

這裏調用了持有的refresh實例變量的refresh變量進行刷新。refresh的類型是ContextRefresher，它是Spring Cloud Commons定義的一個類。

ContextRefresher

    public synchronized Set<String> refresh() {
		Set<String> keys = refreshEnvironment();
		this.scope.refreshAll();
		return keys;
	}

refreshEnvironment方法

    public synchronized Set<String> refreshEnvironment() {
        // 獲取刷新前所有配置
		Map<String, Object> before = extract(
				this.context.getEnvironment().getPropertySources());
		// 實際執行刷新配置
		addConfigFilesToEnvironment();
		// 將新配置和老配置進行合併（若有更新則覆蓋老配置，若無則保留老配置）
		Set<String> keys = changes(before,
				extract(this.context.getEnvironment().getPropertySources())).keySet();
		this.context.publishEvent(new EnvironmentChangeEvent(this.context, keys));
		return keys;
	}

addConfigFilesToEnvironment方法

    ConfigurableApplicationContext addConfigFilesToEnvironment() {
		ConfigurableApplicationContext capture = null;
		try {
		    // 複製當前運行環境
			StandardEnvironment environment = copyEnvironment(
					this.context.getEnvironment());
			// 構建一個新SpringApplicationBuilder
			SpringApplicationBuilder builder = new SpringApplicationBuilder(Empty.class)
					.bannerMode(Mode.OFF).web(WebApplicationType.NONE)
					.environment(environment);
			// Just the listeners that affect the environment (e.g. excluding logging
			// listener because it has side effects)
			builder.application()
					.setListeners(Arrays.asList(new BootstrapApplicationListener(),
							new ConfigFileApplicationListener()));
			// 生成一個新的SpringApplication，通過Spring的生命週期去刷新配置，保存在本方法的environment臨時變量中
			capture = builder.run();
			if (environment.getPropertySources().contains(REFRESH_ARGS_PROPERTY_SOURCE)) {
				environment.getPropertySources().remove(REFRESH_ARGS_PROPERTY_SOURCE);
			}
			MutablePropertySources target = this.context.getEnvironment()
					.getPropertySources();
			String targetName = null;
			// 從environment中的PropertySource添加刷新後的配置
			for (PropertySource<?> source : environment.getPropertySources()) {
				String name = source.getName();
				if (target.contains(name)) {
					targetName = name;
				}
				// 如果不屬於默認PropertySource才添加
				if (!this.standardSources.contains(name)) {
					if (target.contains(name)) {
						target.replace(name, source);
					}
					else {
						if (targetName != null) {
							target.addAfter(targetName, source);
						}
						else {
							// targetName was null so we are at the start of the list
							target.addFirst(source);
							targetName = name;
						}
					}
				}
			}
		}
		finally {
			ConfigurableApplicationContext closeable = capture;
			while (closeable != null) {
				try {
					closeable.close();
				}
				catch (Exception e) {
					// Ignore;
				}
				if (closeable.getParent() instanceof ConfigurableApplicationContext) {
					closeable = (ConfigurableApplicationContext) closeable.getParent();
				}
				else {
					break;
				}
			}
		}
		return capture;
	}

重要的代碼加了註釋，下面看capture = builder.run();這行代碼，最終進入了SpringApplication的run方法

    /**
	 * Run the Spring application, creating and refreshing a new
	 * {@link ApplicationContext}.
	 * @param args the application arguments (usually passed from a Java main method)
	 * @return a running {@link ApplicationContext}
	 */
	public ConfigurableApplicationContext run(String... args) {
		StopWatch stopWatch = new StopWatch();
		stopWatch.start();
		ConfigurableApplicationContext context = null;
		Collection<SpringBootExceptionReporter> exceptionReporters = new ArrayList<>();
		configureHeadlessProperty();
		SpringApplicationRunListeners listeners = getRunListeners(args);
		listeners.starting();
		try {
			ApplicationArguments applicationArguments = new DefaultApplicationArguments(args);
			ConfigurableEnvironment environment = prepareEnvironment(listeners, applicationArguments);
			configureIgnoreBeanInfo(environment);
			Banner printedBanner = printBanner(environment);
			context = createApplicationContext();
			exceptionReporters = getSpringFactoriesInstances(SpringBootExceptionReporter.class,
					new Class[] { ConfigurableApplicationContext.class }, context);
			prepareContext(context, environment, listeners, applicationArguments, printedBanner);
			refreshContext(context);
			afterRefresh(context, applicationArguments);
			stopWatch.stop();
			if (this.logStartupInfo) {
				new StartupInfoLogger(this.mainApplicationClass).logStarted(getApplicationLog(), stopWatch);
			}
			listeners.started(context);
			callRunners(context, applicationArguments);
		}
		catch (Throwable ex) {
			handleRunFailure(context, ex, exceptionReporters, listeners);
			throw new IllegalStateException(ex);
		}

		try {
			listeners.running(context);
		}
		catch (Throwable ex) {
			handleRunFailure(context, ex, exceptionReporters, null);
			throw new IllegalStateException(ex);
		}
		return context;
	}

由此看出，確實是新建了一個SpringApplication，這個類的作用就是對Spring的ApplicationContext進行一系列啓動前的配置。與動態刷新相關的是prepareContext(context, environment, listeners, applicationArguments, printedBanner);這一行

prepareContext方法

    private void prepareContext(ConfigurableApplicationContext context, ConfigurableEnvironment environment,
			SpringApplicationRunListeners listeners, ApplicationArguments applicationArguments, Banner printedBanner) {
		context.setEnvironment(environment);
		postProcessApplicationContext(context);
		// 觸發刷新配置相關的ApplicationContextInitializer
		applyInitializers(context);
		listeners.contextPrepared(context);
		if (this.logStartupInfo) {
			logStartupInfo(context.getParent() == null);
			logStartupProfileInfo(context);
		}
		// Add boot specific singleton beans
		ConfigurableListableBeanFactory beanFactory = context.getBeanFactory();
		beanFactory.registerSingleton("springApplicationArguments", applicationArguments);
		if (printedBanner != null) {
			beanFactory.registerSingleton("springBootBanner", printedBanner);
		}
		if (beanFactory instanceof DefaultListableBeanFactory) {
			((DefaultListableBeanFactory) beanFactory)
					.setAllowBeanDefinitionOverriding(this.allowBeanDefinitionOverriding);
		}
		if (this.lazyInitialization) {
			context.addBeanFactoryPostProcessor(new LazyInitializationBeanFactoryPostProcessor());
		}
		// Load the sources
		Set<Object> sources = getAllSources();
		Assert.notEmpty(sources, "Sources must not be empty");
		load(context, sources.toArray(new Object[0]));
		listeners.contextLoaded(context);
	}

applyInitializers方法

    /**
	 * Apply any {@link ApplicationContextInitializer}s to the context before it is
	 * refreshed.
	 * @param context the configured ApplicationContext (not refreshed yet)
	 * @see ConfigurableApplicationContext#refresh()
	 */
	@SuppressWarnings({ "rawtypes", "unchecked" })
	protected void applyInitializers(ConfigurableApplicationContext context) {
		for (ApplicationContextInitializer initializer : getInitializers()) {
			Class<?> requiredType = GenericTypeResolver.resolveTypeArgument(initializer.getClass(),
					ApplicationContextInitializer.class);
			Assert.isInstanceOf(requiredType, context, "Unable to call initializer.");
			initializer.initialize(context);
		}
	}

循環應用所有註冊的ApplicationContextInitializer到ApplicationContext。其中有一個PropertySourceBootstrapConfiguration的ApplicationContextInitializer，它負責Spring Cloud的外部PropertySource加載。

PropertySourceBootstrapConfiguration

initialize方法

    @Override
	public void initialize(ConfigurableApplicationContext applicationContext) {
		List<PropertySource<?>> composite = new ArrayList<>();
		AnnotationAwareOrderComparator.sort(this.propertySourceLocators);
		boolean empty = true;
		ConfigurableEnvironment environment = applicationContext.getEnvironment();
		// 遍歷propertySourceLocators，獲取PropertySourceLocator並應用
		for (PropertySourceLocator locator : this.propertySourceLocators) {
			Collection<PropertySource<?>> source = locator.locateCollection(environment);
			if (source == null || source.size() == 0) {
				continue;
			}
			List<PropertySource<?>> sourceList = new ArrayList<>();
			for (PropertySource<?> p : source) {
				sourceList.add(new BootstrapPropertySource<>(p));
			}
			logger.info("Located property source: " + sourceList);
			composite.addAll(sourceList);
			empty = false;
		}
		if (!empty) {
			MutablePropertySources propertySources = environment.getPropertySources();
			String logConfig = environment.resolvePlaceholders("${logging.config:}");
			LogFile logFile = LogFile.get(environment);
			for (PropertySource<?> p : environment.getPropertySources()) {
				if (p.getName().startsWith(BOOTSTRAP_PROPERTY_SOURCE_NAME)) {
					propertySources.remove(p.getName());
				}
			}
			insertPropertySources(propertySources, composite);
			reinitializeLoggingSystem(environment, logConfig, logFile);
			setLogLevels(applicationContext, environment);
			handleIncludedProfiles(environment);
		}
	}

該類有一個PropertySourceLocator集合，其中有一個ConsulPropertySourceLocator。

ConsulPropertySourceLocator

它有一個locate方法

@Override
	@Retryable(interceptor = "consulRetryInterceptor")
	public PropertySource<?> locate(Environment environment) {
		if (environment instanceof ConfigurableEnvironment) {
			ConfigurableEnvironment env = (ConfigurableEnvironment) environment;

			String appName = this.properties.getName();

			if (appName == null) {
				appName = env.getProperty("spring.application.name");
			}

			List<String> profiles = Arrays.asList(env.getActiveProfiles());

			String prefix = this.properties.getPrefix();

			List<String> suffixes = new ArrayList<>();
			if (this.properties.getFormat() != FILES) {
				suffixes.add("/");
			}
			else {
				suffixes.add(".yml");
				suffixes.add(".yaml");
				suffixes.add(".properties");
			}
			// 獲取默認context（consul的概念）
			String defaultContext = getContext(prefix,
					this.properties.getDefaultContext());

			for (String suffix : suffixes) {
				this.contexts.add(defaultContext + suffix);
			}
			for (String suffix : suffixes) {
				addProfiles(this.contexts, defaultContext, profiles, suffix);
			}

			String baseContext = getContext(prefix, appName);

			for (String suffix : suffixes) {
				this.contexts.add(baseContext + suffix);
			}
			for (String suffix : suffixes) {
				addProfiles(this.contexts, baseContext, profiles, suffix);
			}

			Collections.reverse(this.contexts);

			CompositePropertySource composite = new CompositePropertySource("consul");

			for (String propertySourceContext : this.contexts) {
				try {
					ConsulPropertySource propertySource = null;
					// 若Consul配置類型爲FILES
					if (this.properties.getFormat() == FILES) {
						Response<GetValue> response = this.consul.getKVValue(
								propertySourceContext, this.properties.getAclToken());
						addIndex(propertySourceContext, response.getConsulIndex());
						if (response.getValue() != null) {
							ConsulFilesPropertySource filesPropertySource = new ConsulFilesPropertySource(
									propertySourceContext, this.consul, this.properties);
							filesPropertySource.init(response.getValue());
							propertySource = filesPropertySource;
						}
					}
					// 若Consul配置類型非FILES
					else {
						propertySource = create(propertySourceContext, this.contextIndex);
					}
					if (propertySource != null) {
						composite.addPropertySource(propertySource);
					}
				}
				catch (Exception e) {
					if (this.properties.isFailFast()) {
						log.error(
								"Fail fast is set and there was an error reading configuration from consul.");
						ReflectionUtils.rethrowRuntimeException(e);
					}
					else {
						log.warn("Unable to load consul config from "
								+ propertySourceContext, e);
					}
				}
			}

			return composite;
		}
		return null;
	}

這裏纔是真正再次去consul獲取最新的配置，並進行替換的地方。常用的是非FILES類型，所以看一下注釋的create方法

create方法

    private ConsulPropertySource create(String context, Map<String, Long> contextIndex) {
		ConsulPropertySource propertySource = new ConsulPropertySource(context,
				this.consul, this.properties);
		propertySource.init();
		addIndex(context, propertySource.getInitialIndex());
		return propertySource;
	}

ConsulPropertySource是Spring Cloud Consul定義的PropertySource。

init方法

public void init() {
		if (!this.context.endsWith("/")) {
			this.context = this.context + "/";
		}
		// 調用consul的HTTP API，獲取配置
		Response<List<GetValue>> response = this.source.getKVValues(this.context,
				this.configProperties.getAclToken(), QueryParams.DEFAULT);

		this.initialIndex = response.getConsulIndex();

		final List<GetValue> values = response.getValue();
		ConsulConfigProperties.Format format = this.configProperties.getFormat();
		// 根據配置類型（key-value，PROPERTIES，YAML）進行解析
		switch (format) {
		case KEY_VALUE:
			parsePropertiesInKeyValueFormat(values);
			break;
		case PROPERTIES:
		case YAML:
			parsePropertiesWithNonKeyValueFormat(values, format);
		}
	}

到此，就將consul上配置的最新值解析到了應用本地。分析就到這裏了，後面的就是將這個ConsulPropertySource實例應用到本地的ApplicationContext中。

總結

Spring Cloud Consul的KV配置動態刷新，主要是依賴了Consul提供的HTTP API，在應用端定時去讀取Consul的配置，並根據版本號（Index）判斷配置是否更新，如果更新再通過Spring的事件機制發佈一個事件。Spring消費該事件時，會通過新建一個SpringApplication的方式，去刷新配置。