clickhouse系統日誌

在操作clickhouse的時候，會有一些日誌被記錄下來，日誌佔用的空間也不少。我們可以設置一下

查詢日誌 query_log

調用查詢語句時，會記錄日誌，記錄sql語句，使用的數據庫和表，佔用的內存等。
https://clickhouse.com/docs/en/operations/system-tables/query_log

設置定期刪除

在config.xml中有配置查詢日誌的位置

<!-- Query log. Used only for queries with setting log_queries = 1. -->
    <query_log>
        <!-- What table to insert data. If table is not exist, it will be created.
             When query log structure is changed after system update,
              then old table will be renamed and new table will be created automatically.
        -->
        <database>system</database>
        <table>query_log</table>
        <!--
            PARTITION BY expr: https://clickhouse.com/docs/en/table_engines/mergetree-family/custom_partitioning_key/
            Example:
                event_date
                toMonday(event_date)
                toYYYYMM(event_date)
                toStartOfHour(event_time)
        -->
        <partition_by>toYYYYMM(event_date)</partition_by>
        <!--
            Table TTL specification: https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/mergetree/#mergetree-table-ttl
            Example:
                event_date + INTERVAL 1 WEEK
                event_date + INTERVAL 7 DAY DELETE
                event_date + INTERVAL 2 WEEK TO DISK 'bbb'

        <ttl>event_date + INTERVAL 30 DAY DELETE</ttl>
        -->

        <!--
            ORDER BY expr: https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/mergetree#order_by
            Example:
                event_date, event_time
                event_date, type, query_id
                event_date, event_time, initial_query_id

        <order_by>event_date, event_time, initial_query_id</order_by>
        -->

        <!-- Instead of partition_by, you can provide full engine expression (starting with ENGINE = ) with parameters,
             Example: <engine>ENGINE = MergeTree PARTITION BY toYYYYMM(event_date) ORDER BY (event_date, event_time) SETTINGS index_granularity = 1024</engine>
          -->

        <!-- Interval of flushing data. -->
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
        <!-- Maximal size in lines for the logs. When non-flushed logs amount reaches max_size, logs dumped to the disk. -->
        <max_size_rows>1048576</max_size_rows>
        <!-- Pre-allocated size in lines for the logs. -->
        <reserved_size_rows>8192</reserved_size_rows>
        <!-- Lines amount threshold, reaching it launches flushing logs to the disk in background. -->
        <buffer_size_rows_flush_threshold>524288</buffer_size_rows_flush_threshold>
        <!-- Indication whether logs should be dumped to the disk in case of a crash -->
        <flush_on_crash>false</flush_on_crash>

        <!-- example of using a different storage policy for a system table -->
        <!-- storage_policy>local_ssd</storage_policy -->
    </query_log>

關閉查詢日誌

在users.xml中，配置策略，爲你的用戶指定策略<log_queries>0</log_queries>。0表示關閉查詢日誌，1表示打開查詢日誌。

    <!-- Profiles of settings. -->
    <profiles>
        <!-- Default settings. -->
        <default>
                        <log_queries>0</log_queries>
        </default>
    </profiles>

https://clickhouse.com/docs/en/operations/settings/settings#log-queries

part_log 分區日誌

我們知道clickhouse可以對數據存儲進行聚合分區，比如設定一天，可以把一天的數據合併到一個數據塊。這樣可以提高效率，也方便管理（主要就是定時刪除舊的數據）。
https://clickhouse.com/docs/en/operations/system-tables/part_log
https://clickhouse.com/docs/en/operations/server-configuration-parameters/settings#part-log

設置

同樣在config.xml中有設置

    <part_log>
        <database>system</database>
        <table>part_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
        <max_size_rows>1048576</max_size_rows>
        <reserved_size_rows>8192</reserved_size_rows>
        <buffer_size_rows_flush_threshold>524288</buffer_size_rows_flush_threshold>
        <flush_on_crash>false</flush_on_crash>
    </part_log>

trace_log 跟蹤日誌

抽樣分析的日誌，分析當前clickhouse的運行狀態

asynchronous_metric_log

記錄操作asynchronous_metric表的日誌。asynchronous_metric表記錄了對內存等使用的信息

query_views_log

對view的查詢日誌
<log_query_views>0</log_query_views>關閉查詢視圖日誌

設置ttl

除了查詢日誌，很多日誌都無法關閉，可以設置ttl，定義超過多久的數據刪除

<ttl>event_date + INTERVAL 30 DAY DELETE</ttl>

如果定義log中有<engine></engine>字段，就不可以用<ttl></ttl>，需要在<engine></engine>中定義ttl，不然啓動clickhouse會報錯

[ 1591898 ] {} <Error> Application: Code: 36. DB::Exception: If 'engine' is specified for system table, TTL parameters should be specified directly inside 'engine' and 'ttl' setting doesn't make sense. (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below):