cdh5-MariaDB 配置（暫未排版）

在多數分佈MariaDB的設施默認設置使用保守的緩衝區的大小和內存使用。


使用保守的緩衝區大小和內存使用率
















Cloudera的數據庫管理服務器，監控活動，報告管理，Cloudera 導航，Hive 的元數據




二進制日誌提供的好處例如：在MariaDB 恢復後的複製或者點對點回復




這配置文件是Cloudera 推薦的設置












1. 如果 MariaDB 服務正在啓動，先停止它。
$ sudo service mariadb stop
2. 將舊的 InnoDB 日誌被分到 /var/lib/mysql/ 以外的路徑
日誌文件地址： /var/lib/mysql/ib_logfile0
/var/lib/mysql/ib_logfile1
3. 確定 marigDB 配置文件 my.cnf 的地址：
4. 修改 my.cnf 使他滿足下列要求:
o 爲了防止死鎖，設置隔離級別爲read committed。
o 在多數分佈式系統中MariaDB 默認使用保守的緩衝區大小和內存使用率。 Cloudera Management Service roles need high write throughput because they might insert many records in the database. Cloudera recommends that you set theinnodb_flush_method property to O_DIRECT.
o Set the max_connections property according to the size of your cluster:
 Small clusters (fewer than 50 hosts) - You can store more than one database (for example, both the Activity Monitor and Service Monitor) on the same host. If you do this, you should:
 Put each database on its own storage volume.
 Allow 100 maximum connections for each database and then add 50 extra connections. For example, for two databases, set the maximum connections to 250. If you store five databases on one host (the databases for Cloudera Manager Server, Activity Monitor, Reports Manager, Cloudera Navigator, and Hive metastore), set the maximum connections to 550.
 Large clusters (more than 50 hosts) - Do not store more than one database on the same host. Use a separate host for each database/host pair. The hosts need not be reserved exclusively for databases, but each database should be on a separate host.
o Binary logging is not a requirement for Cloudera Manager installations. Binary logging provides benefits such as MariaDB replication or point-in-time incremental recovery after database restore. Examples of this configuration follow. For more information, see The Binary Log.
Here is an option file with Cloudera recommended settings:


根據簇的大小設置合適的max_connections值




允許每個數據庫最大連接數爲100個，然後添加50個額外的連接






5. If AppArmor is running on the host where MariaDB is installed, you might need to configure AppArmor to allow MariaDB to write to the binary.
6. Ensure the MariaDB server starts at boot.




如果MariaDB所在主機上運行有AppArmor，你可能需要配置AppArmor允許MariaDB寫入二進制


Start the MariaDB server:


啓動 MariaDB 服務


chown -R mysql.mysql 




$ sudo /usr/bin/mysql_secure_installation
[...]
Enter current password for root (enter for none):
OK, successfully used password, moving on...
[...]
Set root password? [Y/n] y
New password:
Re-enter new password:
Remove anonymous users? [Y/n] Y
[...]
Disallow root login remotely? [Y/n] N
[...]
Remove test database and access to it [Y/n] Y
[...]
Reload privilege tables now? [Y/n] Y
All done!


在 Cloudera Manager Server 主機上面安裝 JDBC 驅動，也需要在 活動監控， 報告管理， Hive元數據服務，燒餅服務， cloudera 導航審查de 每一個主機角色上面安裝。




Creating Databases for Activity Monitor, Reports Manager, Hive Metastore Server, Sentry Server, Cloudera Navigator Audit Server, and Cloudera Navigator Metadata Server


爲Cloudera Manager 各角色創建數據庫


爲需要數據庫的組件創建用戶組賬戶


create database clouderaManager default character set utf8;


drop database clouderaManager;


grant all on clouderaManager.* TO 'cmroot'@'%' IDENTIFIED BY 'admin';


create database cm_amon default character set utf8;
create database cm_rman default character set utf8;
create database cm_metastore default character set utf8;
create database cm_sentry default character set utf8;
create database cm_nav default character set utf8;
create database cm_navms default character set utf8;


-- =========================================================


grant all on cm_amon.* TO 'cmroot'@'%' IDENTIFIED BY 'admin';
grant all on cm_rman.* TO 'cmroot'@'%' IDENTIFIED BY 'admin';
grant all on cm_metastore.* TO 'cmroot'@'%' IDENTIFIED BY 'admin';
grant all on cm_sentry.* TO 'cmroot'@'%' IDENTIFIED BY 'admin';
grant all on cm_nav.* TO 'cmroot'@'%' IDENTIFIED BY 'admin';
grant all on cm_navms.* TO 'cmroot'@'%' IDENTIFIED BY 'admin';




grant all on cm_amon.*, cm_rman.*, cm_metastore.*, cm_sentry.*, cm_nav.*, cm_navms.* TO 'cmroot'@'%' IDENTIFIED BY 'admin';




Role Database User Password
Activity Monitor amon amon amon_password
Reports Manager rman rman rman_password
Hive Metastore Server metastore hive hive_password
Sentry Server sentry sentry sentry_password
Cloudera Navigator Audit Server nav nav nav_password
Cloudera Navigator Metadata Server navms navms navms_password






=================================






[mysqld]
transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
# symbolic-links = 0


key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1


max_connections = 1000
#expire_logs_days = 10
#max_binlog_size = 100M


#log_bin should be on a disk with enough free space. Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your system
#and chown the specified folder to the mysql user.
log_bin=/data/mysql_dir/mysql_binary_log


binlog_format = mixed


read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M


# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit  = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M


[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid






===========
http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_install_path_c.html




Create Users
The Cloudera Manager Server and managed services require a user account to complete tasks. When installing Cloudera Manager from tarballs, you must create this user account on all hosts manually. Because Cloudera Manager Server and managed services are configured to use the user account cloudera-scm by default, creating a user with this name is the simplest approach. This created user, is used automatically after installation is complete.
To create user cloudera-scm, use a command such as the following:
Cloudera Manager Server 和管理服務需要一個用戶賬戶來完成任務。
當通過 tar 包 安裝Cloudera Manager時，你必須在所有機器上手動創建這個用戶賬戶。
因爲 Cloudear Manager Server 和 管理服務是通過默認賬戶 cloudear-scm 來配置的。
用這個名字創建用戶是最簡單的途徑。在安裝完成之後,這個創建的用戶是自動使用的.


-----------------------------------------------------------------------------------------------------
$ sudo useradd --system --home=/opt/cloudera-manager/cm-5.5.3/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
-----------------------------------------------------------------------------------------------------




Configure a Database for the Cloudera Manager Server
爲 Cloudera Manager Server 配置一個數據庫


Depending on whether you are using an external database, or the embedded PostgreSQL database, do one of the following:
這取決於你使用的是外部數據庫，還是內嵌的 PostgreSQL 數據庫， 請執行以下操作之一：


External database - Prepare the Cloudera Manager Server database as described in Preparing a Cloudera Manager Server External Database.
外部數據庫 - 準備 Cloudera Manager Server 數據庫 




Embedded database - Install an embedded PostgreSQL database as described in Installing and Starting the Cloudera Manager Server Embedded Database.
內嵌式數據庫 - 安裝一個內嵌 PostgreSQL 數據庫.






[mariadb]
name=MariaDB
baseurl=http://yum.mariadb.org/10.1.12/centos6-amd64/
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1