Master Server Installation and Configuration

To begin deploying the MooseFS (MFS) cluster, the Master server must be set up first. It is recommended to configure a network alias for high availability, though not strictly required for basic functionality.

First, install the necessary dependencies and create the system user:

yum install zlib-devel gcc -y
groupadd -r mfs
useradd -r -g mfs mfs -s /sbin/nologin

Next, download and compile the MooseFS source code. Adjust the version number as needed:

tar -xzf moosefs-3.0.100.tar.gz
cd moosefs-3.0.100
./configure --prefix=/opt/mfs --with-default-user=mfs --with-default-group=mfs
make && make install

Set up environment variables for easier administration:

echo "export PATH=\$PATH:/opt/mfs/sbin:/opt/mfs/bin" >> /etc/profile
source /etc/profile

Configure the Master settings in /opt/mfs/etc/mfsmaster.cfg to define the working user and group. Then, define the access rules in /opt/mfs/etc/mfsexports.cfg:

# Allow access from the local network
192.168.1.0/24    /   rw,alldirs,mappassword=securePass
# Allow administrative access for metadata
*                .  rw

Initialize the metadata and start the Master service:

cd /opt/mfs/var/mfs
cp metadata.mfs.empty metadata.mfs
/opt/mfs/sbin/mfsmaster start

You can verify the service is listening on port 9420. To enable the web monitoring interface (port 9425), run:

/opt/mfs/sbin/mfscgiserv start

Configuring the Metalogger (Backup Server)

The Metalogger server stores metadata changelogs to facilitate recovery. Install the software using the same compilation steps as the Master.

Modify the configuration file /opt/mfs/etc/mfsmetalogger.cfg to point to the Master's hostname or IP:

MASTER_HOST = 192.168.1.10

Start the Metalogger service:

/opt/mfs/sbin/mfsmetalogger start

Chunkserver (Storage Node) Setup

Chunkservers store the actual data. After compiling the software, edit the configuration file /opt/mfs/etc/mfschunkserver.cfg to identify the Master:

MASTER_HOST = 192.168.1.10

Define the storage directories in /opt/mfs/etc/mfshdd.cfg. Ensure the defined directories exist and are owned by the MFS user:

# /opt/mfs/etc/mfshdd.cfg
/mnt/hd1
/mnt/hd2

chown -R mfs:mfs /mnt/hd1 /mnt/hd2
/opt/mfs/sbin/mfschunkserver start

Client Installation and Mounting

Clients require FUSE to interact with the MooseFS filesystem. Install FUSE and the MFS client tools:

wget https://github.com/libfuse/libfuse/releases/download/fuse-2.9.9/fuse-2.9.9.tar.gz
tar -xzf fuse-2.9.9.tar.gz
cd fuse-2.9.9
./configure --prefix=/usr && make && make install
modprobe fuse

Compile the MFS client (similar to previous steps) and mount the filesystem:

mkdir -p /mnt/mfs_data
/opt/mfs/bin/mfsmount /mnt/mfs_data -H 192.168.1.10 -o mfspassword=securePass

Data Management and Maintenance

MooseFS allows setting replication goals for data redundancy. To set a file to be stored on 3 different chunkservers:

mfssetgoal -r 3 /mnt/mfs_data/directory

Check the file info to verify chunk distribution:

mfsfileinfo /mnt/mfs_data/directory/file.img

Deleted files are retained in the trash for a specific period. To modify the retention time (in seconds):

mfssettrashtime -r 3600 /mnt/mfs_data/directory

To access and restore deleted files, mount the trash directory:

mfsmount -m /mnt/mfs_trash -H 192.168.1.10
cd /mnt/mfs_trash/trash
mv "00000004|directory|file.txt" undel/

Disaster Recovery and Backup

MFS metadata consists of the main metadata file and changelogs. A simple backup script can periodically sync these files:

#!/bin/bash
while true; do
  rsync -az /opt/mfs/var/mfs/ /backup/mfs/var/
  rsync -az /opt/mfs/etc/mfs/ /backup/mfs/etc/
  sleep 50
done

If the Master server fails unexpectedly (e.g., power loss), the metadata file may be inconsistent. Before restarting the Master, run the automated repair tool:

/opt/mfs/sbin/mfsmetarestore -a
/opt/mfs/sbin/mfsmaster start

For a full recovery from a Metalogger or backup to a new server:

1. Install MFS on the new machine (do not start mfsmaster yet).

2. Copy the metadata and changelog files from the backup or Metalogger to /opt/mfs/var/mfs/.

3. Use the restore tool to merge the changelogs into the main metadata file: ``` /opt/mfs/sbin/mfsmetarestore -m metadata_ml.mfs.back -o metadata.mfs changelog_ml.*.mfs

4. Start the Master and verify connectivity.

Note: After recovery, ensure all Chunkservers and Clients are updated with the new Master IP address in their respective configurations, then remount the filesystem on clients.

Safe Cluster Startup and Shutdown

To prevent data corruption, follow these sequences when managing the cluster state.

Startup Order:

1. Start the Master server (mfsmaster).
2. Start all Chunkservers (mfschunkserver).
3. Start the Metalogger (mfsmetalogger).
4. Mount the filesystem on clients (mfsmount).

Shutdown Order:

1. Unmount the filesystem on all clients (umount).
2. Stop Chunkservers (mfschunkserver stop).
3. Stop the Metalogger (mfsmetalogger stop).
4. Stop the Master server (mfsmaster stop).