Upgrade from version 1.5.x to 1.6.x
When you upgrade from 1.5.x to 1.6.x it is important to keep the order of upgraded modules which is described in UPGRADE file:
Upgrading from 1.5.x to 1.6.x:
1. Upgrade and restart mfsmaster.
2. Upgrade and restart chunkservers.
3. Upgrade mfs clients (mfstools before remounting MFS with new mfsmount), remount MFS trees.
You do not need to uninstall any older versions. You also do not need to stop the previous version - the new one should stop the previous.
1 (machine with mfsmaster):
#make install
#PREFIX/sbin/mfsmaster
2 (machines with chunkservers):
#make install
#PREFIX/sbin/mfschunkserver
You should restart chunkservers with 5-10 minutes delay.
3 (clients)
Remount the disks:
umount /mnt/mfs
mfsmount /mnt/mfs
If you can stop the whole system for the restart, you can do the process this way:
1 (machine with mfsmaster):
#make install
#PREFIX/sbin/mfsmaster -s
2 (machines with chunkservers):
#make install
#PREFIX/sbin/mfschunkserver
And here you can make all restarts at one moment. But please wait about 3 minutes before starting the master.
3 (master)
#PREFIX/sbin/mfsmaster
4. And now you shoud change all "msfmounts"
Making and installing
The preferred MooseFS deployment method is installation from the source.
Source package supports standard ./configure && make && make install procedure. Significant configure options are:
--disable-mfsmaster - don't build managing server (useful for plain node installation)
--disable-mfschunkserver - don't build chunkserver
--disable-mfsmount - don't build mfsmount and mfstools (they are built by default if fuse development package is detected)
--enable-mfsmount - make sure to build mfsmount and mfstools (error is reported if fuse development package cannot be found)
--prefix=DIRECTORY - install to given prefix (default is /usr/local)
--sysconfdir=DIRECTORY - select configuration files directory (default is ${prefix}/etc)
--localstatedir=DIRECTORY - select top variable data directory (default is ${prefix}/var; MFS metadata are stored in mfs subdirectory, i.e. ${prefix}/var/mfs by default)
--with-default-user=USER - user to run daemons as if not set in configuration files (default is nobody)
--with-default-group=GROUP - group to run daemons as if not set in configuration files (default is nogroup)
For example, to install MooseFS using system FHS-compliant paths on Linux, use: ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var/lib
make install respects standard DESTDIR= variable, allowing to install package in temporary location (e.g. in order to create binary package). Already existing configuration or metadata files won't be overwritten.
Managing server (master)
As the managing server (master) is a crucial element of MooseFS, it should be installed on a machine which guarantees high reliability and access requirements which are adequate for the whole system. It is advisable to use a server with a redundant power supply, ECC memory, and disk array RAID1/RAID5/RAID10. The managing server OS has to be POSIX compliant (systems verified so far: Linux, FreeBSD, Mac OS X and OpenSolaris).
To install the managing process one needs to:
install mfs-master package (make install after running configure without --disable-mfsmaster option in case of installation from source files)
create a user with whose permissions the service is about to work (if such user doesn't already exists)
make sure that the directory for the meta datafiles exists and is writable by this user (make install run as the root does the thing for the user and paths set up by configure, if the user existed before)
configure the service (file mfsmaster.cfg), paying special attention to TCP ports in use
add or create (depending both on the operating system and the distribution) a batch script starting the process mfsmaster
After the installation the managing server is started by running mfsmaster. If the mfsmaster program is executed by the root, it switches to a configured user; otherwise it runs as a user who executed it. In case of a server outage or an improper shutdown the mfsmetarestore utility will restore the file system information.
Metalogger
Metalogger daemon is installed together with mfs-master. The minimal requirements are not bigger than master itself; it can be run on any machine (e.g. any chunkserver), but the best place is the backup machine for MooseFS master (so in case of primary master machine failure it's possible to run master process in place of metalogger - see appropriate HOWTO for details).
To install the managing process one needs to:
- install {{{mfs-master}}} package ({{{make install}}} after running {{{configure}}} without {{{--disable-mfsmaster}}} option in case of installation from source files)
- create a user with whose permissions the service is about to work (if such user doesn't already exists)
- make sure that the directory for the metadata files exists and is writable by this user ({{{make install}}} run as the root does the thing for the user and paths set up by configure, if the user existed before)
- configure the service (file {{{mfsmetalogger.cfg}}}, paying special attention to TCP ports used ({{{MASTER_PORT}}} has to be the same as {{{MATOML_LISTEN_PORT}}} in {{{mfsmaster.cfg}}} on managing server)
- add or create (depending both on the operating system and the distribution) a batch script starting the process {{{mfsmetalogger}}}
After the installation the managing server is started by running {{{mfsmetalogger}}}. If the {{{mfsmetalogger}}} program is executed by the root, it switches to a configured user; otherwise it runs as a user who executed it.
Chunkservers
When the managing server is installed, data servers (chunkservers) may be set up. These machines should have appropriate free space on disks and POSIX compliant OS (verified so far: Linux, FreeBSD, Mac OS X and OpenSolaris). Chunkserver stores data chunks/fragments as files on a common file system (eg. ext3, xfs, ufs). It is important to dedicate file systems used by MooseFS exclusively to it - this is necessary to manage the free space properly. MooseFS does not take into account that a free space accessible to it could be taken by other data. If it's not possible to create a separate disk partition, filesystems in files can be used (have a look at the following instructions).
Linux:
creating:
dd if=/dev/zero of=file bs=100m seek=400 count=0
mkfs -t ext3 file
mounting:
mount -o loop file mount-point
FreeBSD:
creating and mounting:
dd if=/dev/zero of=file bs=100m count=400
mdconfig -a -t vnode -f file -u X
newfs -m0 -O2 /dev/mdX
mount /dev/mdX mount-point
mounting a previously created file system:
mdconfig -a -t vnode -f file -u X
mount /dev/mdX mount-point
Mac OS X:
Start "Disk Utility" from "/Applications/Utilities"
Select from menu "Images->New->Blank Image ..."
Note: on each chunkserver disk some space is reserved for growing chunks and thus inaccessible for creation of the new ones. Only disks with more than 256 MB and chunkservers reporting more than 1 GB of total free space are accessible for new data. Minimal configurations should start from several gigabytes of storage.
To install the data server (chunkserver):
isolate space intended for MooseFS as separate file systems, mounted at a defined point (e.g. /mnt/hd1, /mnt/hd2 itd.)
install mfs-chunkserver package (make install after running configure without --disable-mfschunkserver option in case of instalation from source files)
create a user, with whose permissions the service is about to work (if such user doesn't already exist)
give this user permissions to write to all filesystems dedicated to MooseFS
configure the service (mfschunkserver.cfg file), paying special attention to used TCP ports (MASTER_PORT has to be the same as MATOCS_LISTEN_PORT in mfsmaster.cfg on managing server)
enter list of mount points of file systems dedicated to MooseFS file mfshdd.conf
add or create (depending both on the operating system and distribution) batch script starting process mfschunkserver
Note: It's important which local IP address mfschunkserver uses to connect to mfsmaster. This address is passed by mfsmaster to MFS clients (mfsmount) and other chunkservers to communicate with the chunkserver, so it must be remotely accessible. Thus master address (MASTER_HOST) must be set to such for which chunkserver will use proper local address to connect - usually belonging to the same network as all MFS clients and other chunkservers. Generally loopback addres (localhost, 127.0.0.1) can't be used as MASTER_HOST, as it would make the chunkserver inaccessible for any other host (such configuration would work only on single machine running all of mfsmaster, mfschunkserver and mfsmount).
After installation data server is started simply by running mfschunkserver. If mfschunkserver program is executed by root, it switches to configured user; otherwise it runs as user who executed it.
Clients (mfsmount)
mfsmount requires FUSE to work. FUSE is available on several operating systems: Linux, FreeBSD, OpenSolaris and MacOS X
In case of Linux a kernel module with API version at least 7.8 is required (it can be checked with dmesg command - after loading kernel module there should be line fuse init (API version 7.8)). It is available in fuse package version 2.6.0 or later or Linux kernel 2.6.20 or later. Due to some minor bugs, the newer module is recommended ((fuse 2.7.2 or Linux 2.6.24 (although fuse 2.7.x standalone doesn't contain getattr/write race condition fix)
In case of FreeBSD the module version 0.3.9 or later should be used
On MacOS X MacFUSE 10.5 was tested
Installing MooseFS client:
install mfs-client package (make install after running configure without --disable-mfsmount option in case of installation from source files)
create a directory where MooseFS will be mounted (e.g. /mnt/mfs)
MooseFS is mounted with the following command:
mfsmount [-h master] [-p port] [-l path] [-w mount-point]
where master is the host name of the managing server, port is the same as given in MATOCU_LISTEN_PORT in file mfsmaster.cfg, path is mounted MooseFS subdirectory (default is /, which means mounting the whole file system), mount-point is the previously created directory for MooseFS.