點評CAT是靈活性非常高的RPM項目，但是實際使用的時候，我們希望可以彈性地添加Server並讓客戶端可以動態的發現。
因此想到了使用註冊發現服務，比如Consul。
git上開源的版本，使用的是三個配置文件：client.xml，server.xml和datasources.xml，並要求放在/data/appdatas/cat目錄下，client.xml和server.xml都配置了可用的Server服務，現在目標就是將其自動註冊和發現。
修改清單如下：
- modify: /pom.xml，項目依賴配置
- add: cat-consul-client項目，client consul發現支持
- add: cat-consul-server項目，server consul註冊支持
- modify: cat-client
- /pom.xml，項目依賴配置
- com.dianping.cat.build.ComponentsConfigurator，依賴配置
- com.dianping.cat.message.io.ChannelManager，強侵入修改
- modify: cat-home
- /pom.xml，項目依賴配置
- /src/main/webapp/WEB-INF/web.xml，項目配置
- /src/main/META-INF/app.properties，consul信息配置
我的操作步驟如下：
一、添加cat-consul-client項目
爲了儘可能少地代碼侵入，我創建了一個子項目，將client端有關的代碼改動都放到這個項目下，根目錄下的pom.xml修改如下：
1、在modules下添加（放在第一個位置，作爲第一個編譯的項目）：
<module>cat-consul-client</module>
2、依賴添加dependencyManagement->dependency下：
<dependency>
<groupId>你自己的groupid</groupId>
<artifactId>cat-consul-client</artifactId>
<version>${project.version}</version>
</dependency>
3、我用了com.ecwid.consul包（和spring-cloud-starter-consul-discovery相同），它的json版本依賴比原CAT使用的版本要高，因此改其版本：
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.2</version>
</dependency>
cat-consul-client的pom.xml文件：
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>parent</artifactId>
<groupId>com.dianping.cat</groupId>
<version>2.0.0</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>你的groupid</groupId>
<artifactId>cat-consul-client</artifactId>
<name>cat-consul-client</name>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>org.unidal.framework</groupId>
<artifactId>web-framework</artifactId>
</dependency>
<dependency>
<groupId>com.ecwid.consul</groupId>
<artifactId>consul-api</artifactId>
<version>1.4.0</version>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.unidal.maven.plugins</groupId>
<artifactId>plexus-maven-plugin</artifactId>
<executions>
<execution>
<id>generate plexus component descriptor</id>
<phase>process-classes</phase>
<goals>
<goal>plexus</goal>
</goals>
<configuration>
<className>你的包名.build.ComponentsConfigurator</className>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<properties>
<project.build.sourceEncoding>utf-8</project.build.sourceEncoding>
</properties>
</project>
CAT應用了plexus來組織模塊關係，ComponentsConfigurator後面介紹。
創建一個類來存儲consul相關的配置信息，這裏我把server的job和alert配置暫時也放在了這裏，並將部分配置挪到了/META-INF/app.properties文件中（本身CAT的client也需要這個文件來註冊app name），具體代碼如下：
package 你的包名;
import com.dianping.cat.Cat;
import com.dianping.cat.configuration.client.entity.ClientConfig;
import com.dianping.cat.configuration.client.entity.Server;
import com.ecwid.consul.v1.ConsulClient;
import com.ecwid.consul.v1.QueryParams;
import com.ecwid.consul.v1.Response;
import com.ecwid.consul.v1.agent.model.NewService;
import com.ecwid.consul.v1.health.model.HealthService;
import org.unidal.helper.Files;
import org.unidal.lookup.extension.Initializable;
import org.unidal.lookup.extension.InitializationException;
import org.unidal.lookup.logging.LogEnabled;
import org.unidal.lookup.logging.Logger;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.List;
import java.util.Properties;
import java.util.UUID;
/**
* 保存了部分Consul必須的參數，轉存配置到文件
* registerServer(): 註冊到Consul
*
*/
public class CatConsul implements LogEnabled, Initializable {
private static final String PROPERTIES_XML = "/META-INF/app.properties";
private String consulUrl = "localhost";
private int consulPort = 8500;
private String serviceName = "cat-consul-service";
private String serviceId;
private String serviceAddress;
private String jobMachine = "true";
private String alertMachine = "true";
private String appName;
private Logger m_logger;
private boolean isInitialized = false;
public String getConsulUrl() {
return consulUrl;
}
public boolean isInitialized() {
return isInitialized;
}
public String getAppName(){
return appName;
}
public void enableLogging(Logger logger) {
m_logger = logger;
}
public String getServiceName() {
return serviceName;
}
public void setServiceName(String serviceName) {
this.serviceName = serviceName;
}
public String getServiceId() {
if(serviceId == null) {
File uuidFile = new File(Cat.getCatHome(), "config/uuid");
if(uuidFile.isFile()) {
try {
serviceId = Files.forIO().readFrom(uuidFile, "utf-8");
} catch (IOException e) {
e.printStackTrace();
}
}
else {
serviceId = this.serviceName + "-" + UUID.randomUUID();
try {
Files.forIO().writeTo(uuidFile, serviceId);
} catch (IOException e) {
e.printStackTrace();
}
}
}
return serviceId;
}
public void setServiceId(String serviceId) {
this.serviceId = serviceId;
}
public void setServiceAddress(String serviceAddress) {
this.serviceAddress = serviceAddress;
}
public CatConsul() {
}
public CatConsul(String consulUrl, int consulPort) {
this.consulUrl = consulUrl;
this.consulPort = consulPort;
}
public ClientConfig getClientConfig() {
ClientConfig config = new ClientConfig().setMode("client");
List<HealthService> services = getServers();
for (HealthService healthService : services) {
HealthService.Service service = healthService.getService();
Server server = new Server(service.getAddress());
server.setPort(service.getPort());
config.addServer(server);
}
// no remote server, add local
if(services.size() == 0) {
Server server = new Server(getServiceAddress());
// TODO default port
server.setPort(2280);
config.addServer(server);
}
return config;
}
public File getClientFile() {
return getFile(getClientConfig().toString(), "client");
}
public File getServerFile() {
return getFile(getServerConfig(), "server");
}
public File getFile(String content, String fileName) {
File configFile = null;
try {
configFile = new File(Cat.getCatHome(), fileName + ".xml");
Files.forIO().writeTo(configFile, content);
} catch (IOException e) {
m_logger.warn("generate file error " + configFile);
}
return configFile;
}
public List<HealthService> getServers() {
ConsulClient client = new ConsulClient(consulUrl, consulPort);
Response<List<HealthService>> healthyServices = client.getHealthServices("cat-consul-service", true, QueryParams.DEFAULT);
return healthyServices.getValue();
}
public String getServiceAddress() {
if(this.serviceAddress == null) {
try {
this.serviceAddress = InetAddress.getLocalHost().getHostAddress().toString();
} catch (UnknownHostException e) {
m_logger.warn("can't load default host ...");
}
}
return this.serviceAddress;
}
public void registerServer() {
NewService newService = new NewService();
newService.setId(getServiceId());
newService.setName(getServiceName());
newService.setAddress(getServiceAddress());
// TODO default port
newService.setPort(2280);
// set check
NewService.Check serviceCheck = new NewService.Check();
serviceCheck.setHttp("http://" + getServiceAddress() + ":8080/cat/actuator/health");
serviceCheck.setTimeout("5s");
serviceCheck.setInterval("10s");
serviceCheck.setDeregisterCriticalServiceAfter("30m");
serviceCheck.setTlsSkipVerify(false);
newService.setCheck(serviceCheck);
ConsulClient client = new ConsulClient(this.consulUrl, this.consulPort);
client.agentServiceRegister(newService);
m_logger.info(String.format("register this service(name:%s, address: %s, port: %s, health check: %s) to consul, consul url is %s, port is %s", serviceName, serviceAddress, 2280, serviceCheck.getHttp(), consulUrl, consulPort));
}
public String getRemoteServers() {
List<HealthService> services = getServers();
// no more server, write local
if(services.size() == 0) {
return getServiceAddress() + ":8080";
}
StringBuilder sb = new StringBuilder();
for (int i = 0; i < services.size(); i ++) {
HealthService.Service service = services.get(i).getService();
// TODO CAT http port 8080
sb.append(service.getAddress()).append(":").append("8080");
if(i < services.size() - 1) {
sb.append(",");
}
}
return sb.toString();
}
/**
* need call after register, to load itself.
*
* @return
*/
public String getServerConfig() {
return String.format(serverFormat, jobMachine, alertMachine, getRemoteServers());
}
private String serverFormat = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n" +
"<config local-mode=\"false\" hdfs-machine=\"false\" job-machine=\"%s\" alert-machine=\"%s\">\n" +
"\t<storage local-base-dir=\"/data/appdatas/cat/bucket/\" max-hdfs-storage-time=\"15\" local-report-storage-time=\"7\" local-logivew-storage-time=\"7\">\n" +
"\t<hdfs id=\"logview\" max-size=\"128M\" server-uri=\"hdfs://10.1.77.86/user/cat\" base-dir=\"logview\"/>\n" +
"\t\t<hdfs id=\"dump\" max-size=\"128M\" server-uri=\"hdfs://10.1.77.86/user/cat\" base-dir=\"dump\"/>\n" +
"\t\t<hdfs id=\"remote\" max-size=\"128M\" server-uri=\"hdfs://10.1.77.86/user/cat\" base-dir=\"remote\"/>\n" +
"\t</storage>\n" +
"\t<console default-domain=\"Cat\" show-cat-domain=\"true\">\n" +
"\t\t<remote-servers>%s</remote-servers>\n" +
"\t</console>\n" +
"</config>";
public void initialize() throws InitializationException {
InputStream in = null;
try {
in = Thread.currentThread().getContextClassLoader().getResourceAsStream(PROPERTIES_XML);
if (in == null) {
in = Cat.class.getResourceAsStream(PROPERTIES_XML);
}
if (in != null) {
Properties prop = new Properties();
prop.load(in);
String _appName = prop.getProperty("app.name");
if (_appName != null) {
m_logger.info(String.format("Find domain name %s from app.properties.", _appName));
this.appName = _appName;
} else {
m_logger.info(String.format("Can't find app.name from app.properties."));
}
String _consulUrl = prop.getProperty("consul.url");
if (_consulUrl != null) {
m_logger.info(String.format("Find consul url %s from app.properties.", _consulUrl));
this.consulUrl = _consulUrl;
}
int _consulport = Integer.decode(prop.getProperty("consul.port", "0"));
if (_consulport != 0) {
m_logger.info(String.format("Find consul port %s from app.properties.", _consulport));
this.consulPort = _consulport;
}
String _alertMachine = prop.getProperty("alert.machine");
if (_alertMachine != null) {
m_logger.info(String.format("Find alert machine %s from app.properties.", _alertMachine));
this.alertMachine = _alertMachine;
}
String _jobMachine = prop.getProperty("job.machine");
if (_jobMachine != null) {
m_logger.info(String.format("Find job machine %s from app.properties.", _jobMachine));
this.jobMachine = _jobMachine;
}
String _localAddress = prop.getProperty("service.address");
if (_localAddress != null) {
m_logger.info(String.format("Find local service address %s from app.properties.", _localAddress));
this.serviceAddress = _localAddress;
}
this.isInitialized = true;
} else {
m_logger.info(String.format("Can't find app.properties in %s", PROPERTIES_XML));
}
} catch (Exception e) {
m_logger.error("cat consul initialize error", e);
throw new InitializationException(e.getMessage(), e);
} finally {
if (in != null) {
try {
in.close();
} catch (Exception e) {
}
}
}
}
}
再創建一個類替代原先的DefaultClientConfigManager類，添加這個類主要是爲了改寫原來的初始化和後續連接檢查時獲取的更新服務端信息：
package 你的包名;
import com.dianping.cat.configuration.*;
import com.dianping.cat.configuration.client.entity.ClientConfig;
import com.dianping.cat.configuration.client.entity.Domain;
import com.dianping.cat.configuration.client.entity.Server;
import com.dianping.cat.configuration.client.transform.DefaultSaxParser;
import com.ecwid.consul.v1.health.model.HealthService;
import org.unidal.helper.Files;
import org.unidal.lookup.annotation.Inject;
import org.unidal.lookup.extension.Initializable;
import org.unidal.lookup.extension.InitializationException;
import org.unidal.lookup.logging.LogEnabled;
import org.unidal.lookup.logging.Logger;
import java.io.File;
import java.util.Collections;
import java.util.List;
import java.util.Map;
public class ConsulClientConfigManager implements LogEnabled, ClientConfigManager, Initializable {
private Logger m_logger;
private ClientConfig m_config;
@Inject
private CatConsul catConsul;
public void enableLogging(Logger logger) {
m_logger = logger;
}
public Domain getDomain() {
Domain domain = null;
if (m_config != null) {
Map<String, Domain> domains = m_config.getDomains();
domain = domains.isEmpty() ? null : domains.values().iterator().next();
}
if (domain != null) {
return domain;
} else {
return new Domain("UNKNOWN").setEnabled(false);
}
}
public int getMaxMessageLength() {
if (m_config == null) {
return 5000;
} else {
return getDomain().getMaxMessageSize();
}
}
//this is for get now server's ip
//TODO change code with ChannelManager#loadServerConfig
public String getServerConfigUrl() {
List<HealthService> healthServiceList = catConsul.getServers();
StringBuilder sb = new StringBuilder();
for(HealthService healthService: healthServiceList){
HealthService.Service service = healthService.getService();
// TODO CAT http port 8080
sb.append(service.getAddress()).append(":").append("8080");
sb.append(";");
}
return "{\"kvs\":{\"routers\":\"" + sb.toString() + "\",\"sample\":\"1.0\"}}";
}
public List<Server> getServers() {
if (m_config == null) {
return Collections.emptyList();
} else {
return m_config.getServers();
}
}
public int getTaggedTransactionCacheSize() {
return 1024;
}
public boolean isCatEnabled() {
if (m_config == null) {
return false;
} else {
return m_config.isEnabled();
}
}
public boolean isDumpLocked() {
if (m_config == null) {
return false;
} else {
return m_config.isDumpLocked();
}
}
public void initialize() throws InitializationException {
initialize(null);
}
public void initialize(File configFile) throws InitializationException {
try {
if(!catConsul.isInitialized()) {
catConsul.initialize();
}
ClientConfig globalConfig = null;
ClientConfig clientConfig = new ClientConfig();
clientConfig.addDomain(new Domain(catConsul.getAppName()));
configFile = catConsul.getClientFile();
String xml = Files.forIO().readFrom(configFile, "utf-8");
globalConfig = DefaultSaxParser.parse(xml);
m_logger.info(String.format("Global config file(%s) found.", configFile));
// merge the two configures together to make it effected
if (globalConfig != null && clientConfig != null) {
globalConfig.accept(new ClientConfigMerger(clientConfig));
}
if (clientConfig != null) {
clientConfig.accept(new ClientConfigValidator());
}
m_config = clientConfig;
m_logger.info("consol client config initialized.");
} catch (Exception e) {
throw new InitializationException(e.getMessage(), e);
}
}
}
修改cat-client項目中com.dianping.cat.build.ComponentsConfigurator類，將ClientConfigManager的實現替換爲上述實現：
all.add(C(ClientConfigManager.class, ConsulClientConfigManager.class).req(CatConsul.class));
接着修改cat-client項目中com.dianping.cat.message.io.ChannelManager，這個類的修改是唯一侵入的，沒辦法，它是關鍵而且源碼中是new出來的，不過這裏只修改了其中一個的方法：
private String loadServerConfig() {
try {
// String url = m_configManager.getServerConfigUrl();
// InputStream inputstream = Urls.forIO().readTimeout(2000).connectTimeout(1000).openStream(url);
// String content = Files.forIO().readFrom(inputstream, "utf-8");
String content = m_configManager.getServerConfigUrl();
KVConfig routerConfig = (KVConfig) m_jsonBuilder.parse(content.trim(), KVConfig.class);
String current = routerConfig.getValue("routers");
m_sample = Double.valueOf(routerConfig.getValue("sample").trim());
return current.trim();
} catch (Exception e) {
// ignore
}
return null;
}
最後修改cat-client的pom.xml文件，加上項目依賴：
<dependency>
<groupId>com.ucmed.cat</groupId>
<artifactId>cat-consul-client</artifactId>
</dependency>
cat-consul-client的ComponentsConfigurator類：
package 你的包名.build;
import 你的包名.CatConsul;
import org.unidal.lookup.configuration.AbstractResourceConfigurator;
import org.unidal.lookup.configuration.Component;
import java.util.ArrayList;
import java.util.List;
public class ComponentsConfigurator extends AbstractResourceConfigurator {
public static void main(String[] args) {
generatePlexusComponentsXmlFile(new ComponentsConfigurator());
}
public List<Component> defineComponents() {
List<Component> all = new ArrayList<Component>();
all.add(C(CatConsul.class));
return all;
}
}
二、cat-consul-home項目相關
依然先修改根目錄下的pom.xml：
1、modules，建議放在倒數第二位：
<module>cat-consul-home</module>
2、dependencyManagement->dependencies:
<dependency>
<groupId>你的groupid</groupId>
<artifactId>cat-consul-home</artifactId>
<version>${project.version}</version>
</dependency>
cat-consul-home的pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>parent</artifactId>
<groupId>com.dianping.cat</groupId>
<version>2.0.0</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>你的groupid</groupId>
<artifactId>cat-consul-home</artifactId>
<name>cat-consul-home</name>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>你的groupid</groupId>
<artifactId>cat-consul-client</artifactId>
</dependency>
<dependency>
<groupId>com.dianping.cat</groupId>
<artifactId>cat-core</artifactId>
</dependency>
<dependency>
<groupId>org.unidal.framework</groupId>
<artifactId>web-framework</artifactId>
</dependency>
<dependency>
<groupId>com.ecwid.consul</groupId>
<artifactId>consul-api</artifactId>
<version>1.4.0</version>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.quartz-scheduler</groupId>
<artifactId>quartz</artifactId>
<version>2.2.1</version>
</dependency>
</dependencies>
<properties>
<project.build.sourceEncoding>utf-8</project.build.sourceEncoding>
</properties>
</project>
添加CatServlet類，server端初始化用：
package 你的包名;
import java.io.File;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.EnumSet;
import javax.servlet.DispatcherType;
import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import 你的包名.CatConsul;
import org.unidal.initialization.DefaultModuleContext;
import org.unidal.initialization.ModuleContext;
import org.unidal.initialization.ModuleInitializer;
import org.unidal.web.AbstractContainerServlet;
public class CatServlet extends AbstractContainerServlet {
private static final long serialVersionUID = 1L;
private Exception m_exception;
@Override
protected void initComponents(ServletConfig servletConfig) throws ServletException {
try {
ModuleContext ctx = new DefaultModuleContext(getContainer());
ModuleInitializer initializer = ctx.lookup(ModuleInitializer.class);
CatConsul catConsul = ctx.lookup(CatConsul.class);
catConsul.initialize();
catConsul.registerServer();
File clientXmlFile = catConsul.getClientFile();
File serverXmlFile = catConsul.getServerFile();
servletConfig.getServletContext().setAttribute("cat-consul-config-module", catConsul);
ctx.setAttribute("cat-client-config-file", clientXmlFile);
ctx.setAttribute("cat-server-config-file", serverXmlFile);
initializer.execute(ctx);
} catch (Exception e) {
m_exception = e;
System.err.println(e);
throw new ServletException(e);
}
}
@Override
protected void service(HttpServletRequest req, HttpServletResponse res) throws ServletException, IOException {
res.setCharacterEncoding("utf-8");
res.setContentType("text/plain");
PrintWriter writer = res.getWriter();
if (m_exception != null) {
writer.write("Server has NOT been initialized successfully!\r\n\r\n");
m_exception.printStackTrace(writer);
} else {
writer.write("Not implemented yet!");
}
}
}
添加CatConsulHealthFilter，這裏consul是通過http方式註冊的，所以添加供consul健康檢查的filter:
package 你的包名;
import javax.servlet.*;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.PrintWriter;
/**
* 給consul返回健康狀態
*
*/
public class CatConsulHealthFilter implements Filter {
public void init(FilterConfig filterConfig) throws ServletException {
}
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
HttpServletResponse httpResponse = (HttpServletResponse) servletResponse;
PrintWriter writer = httpResponse.getWriter();
writer.write("{\"status\":\"UP\"}");
writer.flush();
}
public void destroy() {
}
}
修改cat-home項目的pom.xml，添加項目依賴:
<dependency>
<groupId>你的groupid</groupId>
<artifactId>cat-consul-home</artifactId>
</dependency>
修改cat-home項目webapp/WEB-INF/web.xml，替換原有的CatServlet（原有的需要移除），並添加健康檢查filter配置：
<filter>
<filter-name>health-filter</filter-name>
<filter-class>com.ucmed.cat.consul.home.CatConsulHealthFilter</filter-class>
</filter>
<servlet>
<servlet-name>cat-servlet</servlet-name>
<servlet-class>com.ucmed.cat.consul.home.CatServlet</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>
<filter-mapping>
<filter-name>health-filter</filter-name>
<url-pattern>/actuator/health</url-pattern>
<dispatcher>REQUEST</dispatcher>
</filter-mapping>
修改resources/META-INF/app.properties:
app.name=cat
# default this two is localhost:8500
consul.url=consul的地址
consul.port=consul的端口
# default this two is false
alert.machine=false
job.machine=false
# default with InetAddress.getLocalHost().getHostAddress().toString()
#service.address=本機服務的ip，用於特殊環境
最後mvn clean install -DskipTests 編譯安裝，其他參考原項目文檔，enjoy~