1. 導讀
本文主要講述RabbitMQ客戶端的封裝和使用，思路不一定準確，但目前生產中是這樣使用的，有不對的地方，歡迎批評指正。
本文需要對springboot、spring的spel表達式、rabbitmq工作流程、死信隊列機制有一定的瞭解，熟悉springboot starter的大概思路。
客戶端思路
定義客戶端
定義所有實體類的父類InterfaceFactoryBean
實現ImportBeanDefinitionRegistrar,掃描自定義註解標註的路徑
公用service接口，第三方持久層需實現
2. 客戶端
最近公司微服務架構中有需要同步數據庫表數據的需求，考慮到封裝性和擴展性，減少第三方項目的開發工作量，思路如下(本文不涉及緩存)：
2.1. 服務端創建Fanout模式的交換機，提供數據發送接口
2.2. 封裝客戶端，客戶端作爲jar包存在，供第三方項目(其他項目，以下都以第三方代稱)依賴使用
2.3. 第三方項目依賴的客戶端啓動創建隊列綁定上面的交換機，並創建死信交換機、隊列
2.4. 第三方項目依賴的客戶端接到數據後，通過反射執行第三方實現的客戶端中定義的持久層接口，實現數據入庫。
2.5定義死信隊列接收消費失敗的消息，死信隊列通過RestTemplate入庫，客戶端實現
2.6. 失敗的消息會重試3次，重試後再次失敗，消息進入死信隊列。每次新接收的消息會檢查庫中是否存在同類型的死信，如果有，直接入庫，然後嘗試消費當前第一條，成功就繼續，失敗則終止。（同步的是人員、機構等數據，對順序性要求高）
3. 開始擼代碼 – 客戶端
3.1 創建一個spring boot 項目引入依賴如下：
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
<dependency>
<groupId>com.chinacoal.microservice</groupId>
<artifactId>common-utils</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.1.23</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
@Autowired
private QueueConfig queueConfig;
@RabbitListener(exclusive = false,bindings=@QueueBinding(
value=@Queue(autoDelete = "false",durable = "true",value = "#{queueConfig.getName()}",arguments = {
@Argument(name = "x-dead-letter-exchange", value = "#{queueConfig.getDLX_EXCHANGE_NAME()}"),
@Argument(name = "x-dead-letter-routing-key", value ="#{queueConfig.getROUTE_KEY()}"),
@Argument(name = "x_message_ttl", value = "30000")
}),exchange=@Exchange(value = "orgFanoutExchange", durable = "true",type = ExchangeTypes.FANOUT,autoDelete = "false")))
public void receiveMessage(String receiveMessage, Message message, Channel channel) throws Exception {
// 手動簽收
log.info("接收到消息:[{}]", receiveMessage);
//執行業務邏輯
try {
if(doWork(receiveMessage,message)) {
//throw new FileUploadException();
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
}else {
channel.basicNack(message.getMessageProperties().getDeliveryTag(), false,false);
}
} catch (Exception e) {
log.error("消息簽收失敗:", e);
Integer count = Integer.valueOf(String.valueOf(message.getMessageProperties().getHeaders().get("count")));
if(count >= 3) {
channel.basicNack(message.getMessageProperties().getDeliveryTag(), false,false);
}else {
message.getMessageProperties().getHeaders().put("count", count + 1);
throw e;
}
}
}
主要屬性解釋：
@RabbitListener //聲明客戶端
@Queue //聲明一個隊列
exclusive //表示該消息隊列是否只在當前connection生效
durable //是否開啓持久化
value = "#{queueConfig.getName()}" //隊列的名稱，該表達式爲spel表達式，queueConfig 需要提供getName方法
@Argument(name = "x-dead-letter-exchange", value = "#{queueConfig.getDLX_EXCHANGE_NAME()}"),@Argument(name = "x-dead-letter-routing-key", value ="#{queueConfig.getROUTE_KEY()}"),
//綁定死信隊列和死信隊列的routeKey,@Argument中的name爲固定值
死信隊列聲明代碼
@Autowired
private QueueConfig queueConfig;
@RabbitListener(exclusive = false,bindings=@QueueBinding(
value=@Queue(autoDelete = "false",durable = "true",value = "#{queueConfig.getDLX_QUEUE_NAME()}"),
exchange=@Exchange(value = "#{queueConfig.getDLX_EXCHANGE_NAME()}", durable = "true",type = ExchangeTypes.DIRECT,autoDelete = "false")))
public void receiveMessage(String receiveMessage, Message message, Channel channel) throws Exception {
log.info("接收到死信消息:[{}]", receiveMessage);
//執行業務邏輯
try {
if(doWork(receiveMessage)) {
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
}else {
channel.basicNack(message.getMessageProperties().getDeliveryTag(), false,false);
}
} catch (Exception e) {
log.error("死信消息簽收失敗:", e);
channel.basicNack(message.getMessageProperties().getDeliveryTag(), false,true);
throw e;
}
}
3.3 自定義註解，實現實體類和持久層掃描，和所有實體類的父類型
掃描實體類的註解
@Documented
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@Import(BeanDefinitionRegistrar.class)
public @interface Model {
/**
* @return
*/
String[] value() default {};
/**
* 掃描包
*
* @return
*/
String[] basePackages() default {};
/**
* 掃描的基類
*
* @return
*/
Class<?>[] basePackageClasses() default {};
/**
* 包含過濾器
*
* @return
*/
Filter[] includeFilters() default {};
/**
* 排斥過濾器
*
* @return
*/
Filter[] excludeFilters() default {};
掃描持久層的註解
@Documented
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@Import(BeanDefinitionRegistrar.class)
public @interface ServiceScan {
/**
* @return
*/
String[] value() default {};
/**
* 掃描包
*
* @return
*/
String[] basePackages() default {};
/**
* 掃描的基類
*
* @return
*/
Class<?>[] basePackageClasses() default {};
/**
* 包含過濾器
*
* @return
*/
Filter[] includeFilters() default {};
/**
* 排斥過濾器
*
* @return
*/
Filter[] excludeFilters() default {};
}
}
定義InterfaceFactoryBean，以此爲所有實體的父類，爲後續json轉實體類型提供便利，解決代碼耦合
@Data
public class InterfaceFactoryBean<T> implements FactoryBean<T>{
private Class<T> interfaceClass;
@Override
public T getObject() throws Exception {
// 檢查 h 不爲空，否則拋異常
Objects.requireNonNull(interfaceClass);
return (T) Enhancer.create(interfaceClass,new DymicInvocationHandler());
}
@Override
public Class<?> getObjectType() {
return interfaceClass;
}
@Override
public boolean isSingleton() {
return true;
}
}
3.4 實現ImportBeanDefinitionRegistrar 實現掃描註解標註的包路徑下的類，加入靜態集合
@Slf4j
public class BeanDefinitionRegistrar implements ImportBeanDefinitionRegistrar {
private static final String RESOURCE_PATTERN = "**/*.class";
public static final Map<String, Class<?>> MODEL_MAPPING = new ConcurrentHashMap<String, Class<?>>();
public static final Map<String, Class<?>> SERVICE_MAPPING = new ConcurrentHashMap<String, Class<?>>();
/**
* @param importingClassMetadata
* @param registry
*/
@Override
public void registerBeanDefinitions(AnnotationMetadata importingClassMetadata, BeanDefinitionRegistry registry) {
register(importingClassMetadata,registry,MODEL_MAPPING,ModelScan.class.getName(),0);
register(importingClassMetadata,registry,SERVICE_MAPPING,ServiceScan.class.getName(),1);
}
private void register(AnnotationMetadata importingClassMetadata,BeanDefinitionRegistry registry,Map<String, Class<?>> mapping,String annotationName,Integer flag) {
AnnotationAttributes annAttr = AnnotationAttributes.fromMap(importingClassMetadata.getAnnotationAttributes(annotationName));
String[] basePackages = annAttr.getStringArray("value");
if (ObjectUtils.isEmpty(basePackages)) {
basePackages = annAttr.getStringArray("basePackages");
}
if (ObjectUtils.isEmpty(basePackages)) {
basePackages = getPackagesFromClasses(annAttr.getClassArray("basePackageClasses"));
}
if (ObjectUtils.isEmpty(basePackages)) {
basePackages = new String[] {ClassUtils.getPackageName(importingClassMetadata.getClassName())};
}
List<TypeFilter> includeFilters = extractTypeFilters(annAttr.getAnnotationArray("includeFilters"));
//增加一個包含的過濾器,掃描到的類只要不是抽象的,接口,枚舉,註解,及匿名類那麼就算是符合的
includeFilters.add(new CustomTypeFilter());
List<TypeFilter> excludeFilters = extractTypeFilters(annAttr.getAnnotationArray("excludeFilters"));
excludeFilters.add(new CustomTypeFilter());
List<Class<?>> candidates = scanPackages(basePackages, includeFilters, excludeFilters);
if (candidates.isEmpty()) {
log.info("掃描指定基礎包[{}]時未發現複合條件的基礎類", basePackages.toString());
return;
}
registerBeanDefinitions(candidates, registry,mapping,flag);
}
/**
* @param basePackages
* @param includeFilters
* @param excludeFilters
* @return
*/
private List<Class<?>> scanPackages(String[] basePackages, List<TypeFilter> includeFilters, List<TypeFilter> excludeFilters) {
List<Class<?>> candidates = new ArrayList<Class<?>>();
for (String pkg : basePackages) {
try {
candidates.addAll(findCandidateClasses(pkg, includeFilters, excludeFilters));
} catch (IOException e) {
log.error("掃描指定基礎包[{}]時出現異常", pkg);
continue;
}
}
return candidates;
}
/**
* @param basePackage
* @return
* @throws IOException
*/
private List<Class<?>> findCandidateClasses(String basePackage, List<TypeFilter> includeFilters, List<TypeFilter> excludeFilters) throws IOException {
if (log.isDebugEnabled()) {
log.debug("開始掃描指定包{}下的所有類" + basePackage);
}
List<Class<?>> candidates = new ArrayList<Class<?>>();
String packageSearchPath = ResourcePatternResolver.CLASSPATH_ALL_URL_PREFIX + replaceDotByDelimiter(basePackage) + '/' + RESOURCE_PATTERN;
ResourceLoader resourceLoader = new DefaultResourceLoader();
MetadataReaderFactory readerFactory = new SimpleMetadataReaderFactory(resourceLoader);
Resource[] resources = ResourcePatternUtils.getResourcePatternResolver(resourceLoader).getResources(packageSearchPath);
for (Resource resource : resources) {
MetadataReader reader = readerFactory.getMetadataReader(resource);
if (isCandidateResource(reader, readerFactory, includeFilters, excludeFilters)) {
Class<?> candidateClass = transform(reader.getClassMetadata().getClassName());
if (candidateClass != null) {
candidates.add(candidateClass);
log.debug("掃描到符合要求基礎類:{}" + candidateClass.getName());
}
}
}
return candidates;
}
/**
* 註冊 Bean,
* Bean的名稱格式:
* @param internalClasses
* @param registry
*/
private void registerBeanDefinitions(List<Class<?>> internalClasses, BeanDefinitionRegistry registry,Map<String, Class<?>> mapping,Integer flag) {
for (Class<?> clazz : internalClasses) {
if (mapping.values().contains(clazz)) {
log.debug("重複掃描{}類,忽略重複註冊", clazz.getName());
continue;
}
if(flag == 0 ) {
BeanDefinitionBuilder builder = BeanDefinitionBuilder.genericBeanDefinition(clazz);
GenericBeanDefinition definition = (GenericBeanDefinition) builder.getRawBeanDefinition();
definition.getPropertyValues().add("interfaceClass", clazz);
definition.setBeanClass(InterfaceFactoryBean.class);
definition.setAutowireMode(GenericBeanDefinition.AUTOWIRE_BY_TYPE);
registry.registerBeanDefinition(ClassUtils.getShortNameAsProperty(clazz), definition);
}
mapping.put(ClassUtils.getShortNameAsProperty(clazz), clazz);
}
}
代碼解釋：
基於spring的bean註冊器，掃描自定義註解聲明的包路徑下的所有類，實體類加入MODEL_MAPPING集合，serviceImp加入SERVICE_MAPPING集合，然後通過BeanDefinitionBuilder將所有上面定義的InterfaceFactoryBean的子類注入到spring容器
3.5 執行業務邏輯
@Autowired
private Map<String,InterfaceFactoryBean> maps;//獲取所有InterfaceFactoryBean的實現
private boolean doWork(String receiveMessage, Message m) throws Exception {
Map<String, Object> map = JSON.parseObject(receiveMessage);
String className = (String) map.get("className");
Integer operator = Integer.valueOf(String.valueOf(map.get("operator")));
//校驗數據庫是否存在死信數據，如果有繼續入庫，如果沒有往下執行業務邏輯
String checkSql = "select count(1) from data_exchange_record where className = ? group by className";
MessageWrapper parseObject = JSON.parseObject(receiveMessage, MessageWrapper.class);
if(jDBCUtils.statistics(checkSql,parseObject.getClassName()) > 0) {
//解決重試重複入庫
if("false".equals(m.getMessageProperties().getHeaders().get("status"))) {
jDBCUtils.execInsert(JDBCUtils.INSERT_SQL, parseObject);
m.getMessageProperties().getHeaders().put("status","true");
log.info("數據庫存在該類型，直接入庫:[{}]", receiveMessage);
}
String sql = "select * from data_exchange_record where className= ? order by recordTime asc";
List<MessageWrapper> list = jDBCUtils.getList(sql,parseObject.getClassName());
if(!CollectionUtils.isEmpty(list)) {
//嘗試更新
//成功 更新所有的數據庫中的數據，直到失敗（不重試）
for (int i = 0; i < list.size(); i++) {
MessageWrapper message = list.get(i);
Integer operatorTmp = message.getOperator();
Boolean handleMessage = handleMessage(operatorTmp,JSON.toJSONString(message),message.getClassName());
boolean flag = false;
if(handleMessage) {
flag = jDBCUtils.execDelete(JDBCUtils.DELETE_SQL,message.getId());
if(!flag) {
break;
}
}else {
break;
}
}
}
return true;
}
return handleMessage(operator,receiveMessage,className);
}
private Boolean handleMessage(Integer operator,String receiveMessage,String className) throws Exception {
if(operator == 4) { // sql
String sql = (String) JSON.parseObject(receiveMessage).get("json");
return jDBCUtils.execSql(sql);
}
Class<?> forName = Class.forName(className);
if(!BeanDefinitionRegistrar.MODEL_MAPPING.containsValue(forName)){
return false;
}
String shorName = ClassUtils.getShortNameAsProperty(forName);
if(BeanDefinitionRegistrar.SERVICE_MAPPING.keySet().size() == 0) {
return false;
}
String tempKeyName = "";
for (String key : BeanDefinitionRegistrar.SERVICE_MAPPING.keySet()) {
if(key.contains(shorName)) {
tempKeyName = key;
break;
}
}
if(StringUtils.isBlank(tempKeyName)) {
return false;
}
Class<?> clazz = BeanDefinitionRegistrar.SERVICE_MAPPING.get(tempKeyName);
String jsonString = (String) JSON.parseObject(receiveMessage).get("json");
InterfaceFactoryBean<?> factoryBean = maps.get(CommonUtil.toLowCaseFirstOne(shorName));
Method method = null;
//由於操作的是同一個對象，除刪除意外的所有方法的實體類屬性都應該有默認值
if(operator == 1) { //insert
method = getMethod(clazz,"insert");
method = invokeMethod( clazz, method, jsonString,shorName,factoryBean);
return (Boolean) method.invoke(clazz.newInstance(),factoryBean);
}else if(operator == 2) { // update
method = getMethod(clazz,"update");
method = invokeMethod( clazz, method, jsonString,shorName,factoryBean);
return (Boolean) method.invoke(clazz.newInstance(),factoryBean);
}else if(operator == 3) { // delete
method = getMethod(clazz,"delete");
method = invokeMethod( clazz, method, jsonString,shorName,factoryBean);
return (Boolean) method.invoke(clazz.newInstance(),factoryBean);
}
return false;
}
private Method invokeMethod(Class<?> clazz, Method method,String jsonString, String shorName, InterfaceFactoryBean<?> factoryBean2) throws Exception {
JSONObject parse = JSON.parseObject(jsonString);
InterfaceFactoryBean<?> factoryBean = maps.get(CommonUtil.toLowCaseFirstOne(shorName));
if(!Objects.isNull(parse)) {
Method[] methods = factoryBean.getClass().getMethods();
for (int j = 0; j < methods.length; j++) {
Method method2 = methods[j];
String name = method2.getName();
String lowCaseFirstOne = CommonUtil.toLowCaseFirstOne(name.substring(3));
Object param =parse.get(lowCaseFirstOne);
if(name.startsWith("set") && !Objects.isNull(param)) {
String object = String.valueOf(param);
Type[] parameters = method2.getGenericParameterTypes();
Class<?> forName2 = Class.forName(parameters[0].getTypeName());
Method declaredMethod = factoryBean.getClass().getDeclaredMethod(method2.getName(),forName2);
if(forName2 == BigDecimal.class) {
declaredMethod.invoke(factoryBean,new BigDecimal(object));
}else if(forName2 == Date.class){
Date date = new Date();
date.setTime(Long.valueOf(object));
declaredMethod.invoke(factoryBean,date);
}else {
declaredMethod.invoke(factoryBean,object);
}
}
}
}
return method;
}
代碼解釋：
通過上面定義的SERVICE_MAPPING集合對比當前消息的className，然後從容器中取出，通過消息中的operator判斷是什麼操作，然後反射相應的方法。
再從maps中獲取具體的實體類型class對象，反射調用set方法賦值，執行。。。
3.6 jdbc的代碼和共用接口(第三方持久層需要實現)
jdbc
public static final String INSERT_SQL = "insert into data_exchange_record (operator,className,json) values (?,?,?)";
public static final String DELETE_SQL = "delete from data_exchange_record where id = ?";
@Autowired
private JdbcTemplate jdbcTemplate;
/**
* 插入
* @param sql
* @param message
* @return
*/
public Boolean execInsert(String sql,MessageWrapper message) {
try {
return jdbcTemplate.update(sql, message.getOperator(),message.getClassName(),message.getJson()) > 0;
}catch(Exception e ) {
log.error("插入數據失敗:",e);
return false;
}
}
/**
* 刪除
* @param sql
* @param t
* @return
*/
public <T> Boolean execDelete(String sql,T t) {
return jdbcTemplate.update(sql, t) > 0;
}
/**
* 執行sql
* @param sql
* @return
*/
public Boolean execSql(String sql) {
jdbcTemplate.execute(sql);
return false;
}
/**
* 數據同步數據層接口
* @author fly
*
* @param <T>
*/
public interface CommonDefinitionService<T> {
/**
* 插入
* @param t
* @return
*/
Boolean insert(T t);
/**
* 更新
* @param t
* @return
*/
Boolean update(T t);
/**
* 刪除
* @param t
* @return
*/
Boolean delete(T t);
}
3.7 starter實現
------------定義自動配置類
@Log4j2
@Configuration
@ConditionalOnClass({RabbitMQProperties.class})
@EnableConfigurationProperties(RabbitMQProperties.class)
@ConditionalOnProperty(prefix = "data.exchange", value = "enabled", havingValue = "true")
@PropertySource(value = {"classpath:rabbit.properties"})
@ComponentScan("com.data.exchange")
public class DataExchangeAutoConfiguration{
@PostConstruct
public void init() {
log.info("*********啓動[{}]自動配置*********",DataExchangeAutoConfiguration.class);
}
}
然後定義resources/META/spring.factories
org.springframework.boot.autoconfigure.EnableAutoConfiguration=com.data.exchange.config.DataExchangeAutoConfiguration
客戶端到這裏就差不多完畢了。寫的匆忙，後續繼續補充，代碼後續上傳到github。
