Sentinel學習(八) —— 規則持久化

規則持久化分成兩種方式：拉模式和推模式。

拉模式

原理簡述

FileRefreshableDataSource 定時從指定文件中讀取規則JSON文件【圖中的本地文件】，如果發現文件發生變化，就更新規則緩存。
FileWritableDataSource 接收控制檯規則推送，並根據配置，修改規則JSON文件【圖中的本地文件】。

功能實現

加依賴

<dependency>
  <groupId>com.alibaba.csp</groupId>
  <artifactId>sentinel-datasource-extension</artifactId>
</dependency>

增加拉模式規則持久化類

/**
 * 拉模式規則持久化
 *
 * @author itmuch.com
 */
public class FileDataSourceInit implements InitFunc {
    @Override
    public void init() throws Exception {
        // TIPS: 如果你對這個路徑不喜歡，可修改爲你喜歡的路徑
        String ruleDir = System.getProperty("user.home") + "/sentinel/rules";
        String flowRulePath = ruleDir + "/flow-rule.json";
        String degradeRulePath = ruleDir + "/degrade-rule.json";
        String systemRulePath = ruleDir + "/system-rule.json";
        String authorityRulePath = ruleDir + "/authority-rule.json";
        String paramFlowRulePath = ruleDir + "/param-flow-rule.json";

        this.mkdirIfNotExits(ruleDir);
        this.createFileIfNotExits(flowRulePath);
        this.createFileIfNotExits(degradeRulePath);
        this.createFileIfNotExits(systemRulePath);
        this.createFileIfNotExits(authorityRulePath);
        this.createFileIfNotExits(paramFlowRulePath);

        // 流控規則
        ReadableDataSource<String, List<FlowRule>> flowRuleRDS = new FileRefreshableDataSource<>(
            flowRulePath,
            flowRuleListParser
        );
        // 將可讀數據源註冊至FlowRuleManager
        // 這樣當規則文件發生變化時，就會更新規則到內存
        FlowRuleManager.register2Property(flowRuleRDS.getProperty());
        WritableDataSource<List<FlowRule>> flowRuleWDS = new FileWritableDataSource<>(
            flowRulePath,
            this::encodeJson
        );
        // 將可寫數據源註冊至transport模塊的WritableDataSourceRegistry中
        // 這樣收到控制檯推送的規則時，Sentinel會先更新到內存，然後將規則寫入到文件中
        WritableDataSourceRegistry.registerFlowDataSource(flowRuleWDS);

        // 降級規則
        ReadableDataSource<String, List<DegradeRule>> degradeRuleRDS = new FileRefreshableDataSource<>(
            degradeRulePath,
            degradeRuleListParser
        );
        DegradeRuleManager.register2Property(degradeRuleRDS.getProperty());
        WritableDataSource<List<DegradeRule>> degradeRuleWDS = new FileWritableDataSource<>(
            degradeRulePath,
            this::encodeJson
        );
        WritableDataSourceRegistry.registerDegradeDataSource(degradeRuleWDS);

        // 系統規則
        ReadableDataSource<String, List<SystemRule>> systemRuleRDS = new FileRefreshableDataSource<>(
            systemRulePath,
            systemRuleListParser
        );
        SystemRuleManager.register2Property(systemRuleRDS.getProperty());
        WritableDataSource<List<SystemRule>> systemRuleWDS = new FileWritableDataSource<>(
            systemRulePath,
            this::encodeJson
        );
        WritableDataSourceRegistry.registerSystemDataSource(systemRuleWDS);

        // 授權規則
        ReadableDataSource<String, List<AuthorityRule>> authorityRuleRDS = new FileRefreshableDataSource<>(
            authorityRulePath,
            authorityRuleListParser
        );
        AuthorityRuleManager.register2Property(authorityRuleRDS.getProperty());
        WritableDataSource<List<AuthorityRule>> authorityRuleWDS = new FileWritableDataSource<>(
            authorityRulePath,
            this::encodeJson
        );
        WritableDataSourceRegistry.registerAuthorityDataSource(authorityRuleWDS);

        // 熱點參數規則
        ReadableDataSource<String, List<ParamFlowRule>> paramFlowRuleRDS = new FileRefreshableDataSource<>(
            paramFlowRulePath,
            paramFlowRuleListParser
        );
        ParamFlowRuleManager.register2Property(paramFlowRuleRDS.getProperty());
        WritableDataSource<List<ParamFlowRule>> paramFlowRuleWDS = new FileWritableDataSource<>(
            paramFlowRulePath,
            this::encodeJson
        );
        ModifyParamFlowRulesCommandHandler.setWritableDataSource(paramFlowRuleWDS);
    }

    private Converter<String, List<FlowRule>> flowRuleListParser = source -> JSON.parseObject(
        source,
        new TypeReference<List<FlowRule>>() {
        }
    );
    private Converter<String, List<DegradeRule>> degradeRuleListParser = source -> JSON.parseObject(
        source,
        new TypeReference<List<DegradeRule>>() {
        }
    );
    private Converter<String, List<SystemRule>> systemRuleListParser = source -> JSON.parseObject(
        source,
        new TypeReference<List<SystemRule>>() {
        }
    );

    private Converter<String, List<AuthorityRule>> authorityRuleListParser = source -> JSON.parseObject(
        source,
        new TypeReference<List<AuthorityRule>>() {
        }
    );

    private Converter<String, List<ParamFlowRule>> paramFlowRuleListParser = source -> JSON.parseObject(
        source,
        new TypeReference<List<ParamFlowRule>>() {
        }
    );

    private void mkdirIfNotExits(String filePath) throws IOException {
        File file = new File(filePath);
        if (!file.exists()) {
            file.mkdirs();
        }
    }

    private void createFileIfNotExits(String filePath) throws IOException {
        File file = new File(filePath);
        if (!file.exists()) {
            file.createNewFile();
        }
    }

    private <T> String encodeJson(T t) {
        return JSON.toJSONString(t);
    }
}

配置

在項目的 resources/META-INF/services 目錄下創建文件，名爲 com.alibaba.csp.sentinel.init.InitFunc ，內容爲：
```
# 改成上面FileDataSourceInit的包名類名全路徑即可。
com.itmuch.contentcenter.FileDataSourceInit
```
優缺點分析
- 優點
  - 簡單易懂
  - 沒有多餘依賴（比如配置中心、緩存等）
- 缺點
  - 由於規則是用 FileRefreshableDataSource 定時更新的，所以規則更新會有延遲。如果FileRefreshableDataSource定時時間過大，可能長時間延遲；如果FileRefreshableDataSource過小，又會影響性能；
  - 規則存儲在本地文件，如果有一天需要遷移微服務，那麼需要把規則文件一起遷移，否則規則會丟失。

推模式

原理簡述

控制檯推送規則：
- 將規則推送到Nacos或其他遠程配置中心
- Sentinel客戶端鏈接Nacos，獲取規則配置；並監聽Nacos配置變化，如發生變化，就更新本地緩存（從而讓本地緩存總是和Nacos一致）
控制檯監聽Nacos配置變化，如發生變化就更新本地緩存（從而讓控制檯本地緩存總是和Nacos一致）

整合Apollo

控制面板改造

首先我們來改造sentinel的控制面板。在源碼中官方已經給出來了單元測試。我們來進行改造。

我們在java包下面的com.alibaba.csp.sentinel.dashboard.rule創建一個apollo包，開始創建我們自己的類。

ApolloConfig

由於我們要將限流配置保存到Apollo中，所以我們需要配置地址和調用OpenAPI的token。

@Configuration
@EnableApolloConfig
public class ApolloConfig {
    /**
     * apollo地址
     */
    @Value("${apollo.sentinel.portal.url}")
    private String apolloPortalUrl;

    /**
     * apollo token
     */
    @Value("${apollo.sentinel.token}")
    private String apolloApplicationToken;

    /**
     * @Author xyhua
     * @Description apollo openApi
     * @Date 09:39 2020-06-02
     * @param
     * @return
     **/
    @Bean
    public ApolloOpenApiClient apolloOpenApiClient() {
        ApolloOpenApiClient client = ApolloOpenApiClient.newBuilder()
            .withPortalUrl(apolloPortalUrl)
            .withToken(apolloApplicationToken)
            .build();
        return client;

    }
}

ApolloCommonService

主要用來對Apollo裏的值和Sentinel中的實體類相互轉換。在保存規則和讀取規則都會調這裏面的方法。

@Service
public class ApolloCommonService {
    /**
     * 沒有找到配置項，apollo 返回的錯誤碼
     */
    private static final int NO_FOUND_ERROR_CODE = 404;
    @Autowired
    private ApolloOpenApiClient apolloOpenApiClient;

    @Value("${apollo.sentinel.env}")
    private String env;

    @Value("${apollo.sentinel.appid}")
    private String appId;

    @Value("${apollo.sentinel.cluster.name}")
    private String clusterName;

    @Value("${apollo.sentinel.namespace.name}")
    private String namespaceName;

    @Value("${apollo.sentinel.modify.user}")
    private String modifyUser;

    @Value("${apollo.sentinel.modify.comment}")
    private String modifyComment;

    @Value("${apollo.sentinel.release.comment}")
    private String releaseComment;

    @Value("${apollo.sentinel.release.user}")
    private String releaseUser;

    /**
     * @Author xyhua
     * @Description 從apollo中獲取規則
     * @Date 19:52 2020-06-02
     * @param
     * @return
     **/
    public <T> List<T> getRules(String appName, String flowDataIdSuffix, Class<T> ruleClass) {
        // flowDataId
        String flowDataId = appName + flowDataIdSuffix;
        OpenNamespaceDTO openNamespaceDTO = apolloOpenApiClient.getNamespace(appId, env, clusterName, namespaceName);
        String rules = openNamespaceDTO
                .getItems()
                .stream()
                .filter(p -> p.getKey().equals(flowDataId))
                .map(OpenItemDTO::getValue)
                .findFirst()
                .orElse("");

        if (StringUtil.isEmpty(rules)) {
            return new ArrayList<>();
        }

        List<T> flow = JSON.parseArray(rules, ruleClass);
        if (Objects.isNull(flow)) {
            return new ArrayList<>();
        }
        return flow;
    }

    /**
     * @Author xyhua
     * @Description 設置規則類型
     * @Date 01:31 2020-06-03
     * @param
     * @return
     **/
    public void publishRules(String appName, String flowDataIdSuffix, String rules) {
        // flowDataId
        String flowDataId = appName + flowDataIdSuffix;
        AssertUtil.notEmpty(appName, "app name cannot be empty");
        if (rules == null) {
            return;
        }
        OpenItemDTO openItemDTO = new OpenItemDTO();
        openItemDTO.setKey(flowDataId);
        openItemDTO.setValue(rules);
        openItemDTO.setComment(modifyComment);
        openItemDTO.setDataChangeCreatedBy(modifyUser);
        try {
            apolloOpenApiClient.createOrUpdateItem(appId, env, clusterName, namespaceName, openItemDTO);
        } catch (Exception e) {
            if (e.getCause() instanceof ApolloOpenApiException) {
                ApolloOpenApiException apolloOpenApiException = (ApolloOpenApiException) e.getCause();
                if (Integer.valueOf(NO_FOUND_ERROR_CODE).equals(apolloOpenApiException.getStatus())) {
                    apolloOpenApiClient.createItem(appId, env, clusterName, namespaceName, openItemDTO);
                    System.out.println("初始化應用配置 -> {}" + flowDataId);
                }
            } else {
                e.printStackTrace();
            }
        }
        // Release configuration
        NamespaceReleaseDTO namespaceReleaseDTO = new NamespaceReleaseDTO();
        namespaceReleaseDTO.setEmergencyPublish(true);
        namespaceReleaseDTO.setReleaseComment(releaseComment);
        namespaceReleaseDTO.setReleasedBy(releaseUser);
        namespaceReleaseDTO.setReleaseTitle(releaseComment);
        apolloOpenApiClient.publishNamespace(appId, env, clusterName, namespaceName, namespaceReleaseDTO);
    }

    /**
     * @Author xyhua
     * @Description 刪除規則
     * @Date 01:33 2020-06-03
     * @param
     * @return
     **/
    public void deleteRules(String rulekey, String operator) {
        try {
            apolloOpenApiClient.removeItem(appId, env, clusterName, namespaceName, rulekey, operator);
        } catch (Exception e) {
            if (e.getCause() instanceof ApolloOpenApiException) {
                ApolloOpenApiException apolloOpenApiException = (ApolloOpenApiException) e.getCause();
                if (Integer.valueOf(NO_FOUND_ERROR_CODE).equals(apolloOpenApiException.getStatus())) {
                    apolloOpenApiClient.removeItem(appId, env, clusterName, namespaceName, rulekey, operator);
                }
            } else {
                e.printStackTrace();
            }
        }
        // Release configuration
        NamespaceReleaseDTO namespaceReleaseDTO = new NamespaceReleaseDTO();
        namespaceReleaseDTO.setEmergencyPublish(true);
        namespaceReleaseDTO.setReleaseComment(releaseComment);
        namespaceReleaseDTO.setReleasedBy(releaseUser);
        namespaceReleaseDTO.setReleaseTitle(releaseComment);
        apolloOpenApiClient.publishNamespace(appId, env, clusterName, namespaceName, namespaceReleaseDTO);
    }
}

AbstractApolloCommonService

基礎抽象類，主要定義類Apollo中配置的不同規則前綴和上面提到的Apollo規則的處理類。

@Service
public abstract class AbstractApolloCommonService {

    @Autowired
    protected ApolloCommonService apolloCommonService;

    /**
     * 流控規則前綴標示
     */
    @Value("${apollo.sentinel.flow.key.suffix:-flow}")
    String flowDataIdSuffix;

    /**
     * 熔斷降級規則前綴標示
     */
    @Value("${apollo.sentinel.degrade.key.suffix:-degrade}")
    String degradeDataIdSuffix;

    /**
     * 熱點規則前綴標示
     */
    @Value("${apollo.sentinel.paramFlow.key.suffix:-paramFlow}")
    String paramFlowDataIdSuffix;

    /**
     * 系統規則前綴標示
     */
    @Value("${apollo.sentinel.system.key.suffix:-system}")
    String systemDataIdSuffix;

    /**
     * 授權規則前綴標示
     */
    @Value("${apollo.sentinel.authority.key.suffix:-authority}")
    String authorityDataIdSuffix;
}

DynamicRuleProvider和DynamicRulePublisher

public interface DynamicRuleProvider<T> {
    T getRules(String appName) throws Exception;
}

public interface DynamicRulePublisher<T> {
    void publish(String app, T rules) throws Exception;
}

這個是Sentinel提供的對規則的處理接口。分別包含getRules和publish方法用來對規則進行獲取和保存，針對不同的限流規則我們實現不同的實體類就可以了。我們用限流規則來舉例。

FlowRuleApolloProvider和FlowRuleApolloPublisher

@Component("flowRuleApolloProvider")
public class FlowRuleApolloProvider extends AbstractApolloCommonService implements
        DynamicRuleProvider<List<FlowRuleEntity>> {

    @Override
    public List<FlowRuleEntity> getRules(String appName) {
        return apolloCommonService.getRules(appName, flowDataIdSuffix, FlowRuleEntity.class);
    }
}

@Component("flowRuleApolloPublisher")
public class FlowRuleApolloPublisher extends AbstractApolloCommonService implements
        DynamicRulePublisher<List<FlowRuleEntity>> {

    @Override
    public void publish(String app, List<FlowRuleEntity> rules) {
        apolloCommonService.publishRules(app, flowDataIdSuffix, JSON.toJSONString(rules));
    }
}

FlowControllerV2

這是控制檯的限流處理類，需要引用我們剛剛創建的類。

@RestController
@RequestMapping(value = "/v2/flow")
public class FlowControllerV2 {
	...
    @Autowired
    @Qualifier("flowRuleApolloProvider")
    private DynamicRuleProvider<List<FlowRuleEntity>> ruleProvider;
    @Autowired
    @Qualifier("flowRuleApolloPublisher")
    private DynamicRulePublisher<List<FlowRuleEntity>> rulePublisher;
    ...
}

sidebar.html

修改頁面，讓他調用V2的controller

          <li ui-sref-active="active" ng-if="!entry.isGateway">
            <a ui-sref="dashboard.flow({app: entry.app})">
              <i class="glyphicon glyphicon-filter"></i>&nbsp;&nbsp;流控規則</a>
          </li>

重啓下sentinel控制檯，設置一下流控規則，我們就看到規則已經保存到Apollo中了。

客戶端改造

引入依賴

        <dependency>
            <groupId>com.alibaba.csp</groupId>
            <artifactId>sentinel-datasource-apollo</artifactId>
        </dependency>

增加Apollo配置

apollo:
  meta: http://39.106.161.250:8080
  bootstrap:
    enabled: true
    namespaces: FISH.Sentinel-Common

增加限流規則配置，告訴客戶端如何讀取Apollo中的配置規則。

spring:
  cloud:
    sentinel:
      filter:
        enabled: true
      transport:
        dashboard: localhost:8080
      datasource:
        ds:
          apollo:
            namespaceName: FISH.Sentinel-Common
            flowRulesKey: order-flow
            ruleType: flow

這樣客戶端就能動態讀取Apollo配置的規則了。

Sentinel學習(八) —— 規則持久化

拉模式

原理簡述

功能實現

推模式

原理簡述

整合Apollo

使用c#強大的表達式樹實現對象的深克隆之解決循環引用的問題

痞子衡嵌入式：恩智浦i.MX RT1xxx系列MCU啓動那些事（12.A）- uSDHC eMMC啓動時間(RT1170)

GPT-4o 引領人機交互新風向，向量數據庫賽道沸騰了

本地SSL證書過期輸入命令在IIS自動生成

.NET週刊【5月第2期 2024-05-12】

Sentinel學習(八) —— 規則持久化

Nacos學習(一) —— Nacos配置管理實戰

Sentinel學習(五) —— 控制檯使用和源碼

Kafka學習(二)——生產者

Kafka學習(一)——初識與入門

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結