This guide walks through setting up MySQL, Hive, and Sqoop on a single Hadoop node (Hadoop01). The steps assume a CentOS 7 environment with Hadoop 2.7.4 already deployed. All operations are performed on Hadoop01 only.

1. MySQL Installation & Configuration

1.1 Remove Existing MariaDB

yum remove mysql-libs -y

1.2 Upload and Extract RPM Bundle

cd /export/software
mkdir mysql_rpm
tar -xvf mysql-5.7.25-1.el7.x86_64.rpm-bundle.tar -C /export/software/mysql_rpm/

1.3 Install MySQL RPMs (order matters due to dependencies)

cd mysql_rpm
rpm -ivh mysql-community-common-5.7.25-1.el7.x86_64.rpm
rpm -ivh mysql-community-libs-5.7.25-1.el7.x86_64.rpm
rpm -ivh mysql-community-libs-compat-5.7.25-1.el7.x86_64.rpm
rpm -ivh mysql-community-client-5.7.25-1.el7.x86_64.rpm
rpm -ivh mysql-community-server-5.7.25-1.el7.x86_64.rpm
rpm -ivh mysql-community-devel-5.7.25-1.el7.x86_64.rpm

After installation, clean temporary files and verify:

cd /export/software
rm -rf mysql_rpm/*
rpm -qa | grep -i mysql

2.1 Start MySQL Service

systemctl start mysqld
systemctl enable mysqld
systemctl status mysqld
ps -le | grep mysqld
netstat -ntlp | grep mysqld

2.2 Set MySQL Root Password

Retrieve the temporary password from the log:

grep 'temporary password' /var/log/mysqld.log

For lab convenience, disable password validation (not recommended for production):

vim /etc/my.cnf
# Add line:
validate_password=off

Restart MySQL and log in with the temporary password:

systemctl restart mysqld
mysql -u root -p

Change password to 123456:

mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('123456');

2.3 Set Default Encoding to UTF-8

Edit /etc/my.cnf and add:

skip_ssl
character_set_server=utf8
init_connect='SET NAMES utf8'

Restart MySQL and verify:

systemctl restart mysqld
mysql -u root -p123456
mysql> SHOW VARIABLES LIKE 'char%';
mysql> SHOW VARIABLES LIKE 'collation%';

2.4 Enable Remote Access

mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123456' WITH GRANT OPTION;
mysql> FLUSH PRIVILEGES;

2. Hive Installation & Configuration

2.1 Extract Hive Archive

cd /export/software
tar -zxvf apache-hive-1.2.1-bin.tar.gz -C /export/servers/

2.2 Configure hive-env.sh

cd /export/servers/apache-hive-1.2.1-bin/conf
cp hive-env.sh.template hive-env.sh
vim hive-env.sh
# Add:
export HADOOP_HOME=/export/servers/hadoop-2.7.4

2.3 Configure hive-site.xml

vim hive-site.xml

Add the following configuration:

<configuration>
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
        <description>MySQL JDBC URL for Hive metastore</description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>123456</value>
    </property>
</configuration>

2.4 Provide MySQL JDBC Driver

cp /export/software/mysql-connector-java-5.1.32.jar /export/servers/apache-hive-1.2.1-bin/lib/

2.5 Add Hive to System PATH (optional)

vim /etc/profile
# Then add appropriate export lines for HIVE_HOME and PATH

3. Sqoop Installation & Configuration

3.1 Extract Sqoop Archive

cd /export/software
tar -zxvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz -C /export/servers/

3.2 Rename Directory

mv /export/servers/sqoop-1.4.7.bin__hadoop-2.6.0 /export/servers/sqoop-1.4.7

3.3 Configure Sqoop

cd /export/servers/sqoop-1.4.7/conf
cp sqoop-env-template.sh sqoop-env.sh
vim sqoop-env.sh
# Add:
export HADOOP_COMMON_HOME=/export/servers/hadoop-2.7.4
export HADOOP_MAPRED_HOME=/export/servers/hadoop-2.7.4
export HIVE_HOME=/export/servers/apache-hive-1.2.1-bin

3.4 Set System Environment Variables

vim /etc/profile
# Add lines for SQOOP_HOME and update PATH

3.5 Verify Sqoop with MySQL Connection

Insure Hadoop is running, then test:

sqoop list-databases --connect jdbc:mysql://localhost:3306 --username root --password 123456

The command should list available MySQL databases, confirming Sqoop works correctly.

Important Notes

• All operations are performed on Hadoop01 only. Hadoop02 and Hadoop03 are not involved.
• Hadoop services must be started before using Sqoop.
• For a lab environment, password policies are relaxed; do not replicate this in production.