Freaks City

This guide walks through setting up MySQL, Hive, and Sqoop on a single Hadoop node (Hadoop01). The steps assume a CentOS 7 environment with Hadoop 2.7.4 already deployed. All operations are performed on Hadoop01 only. 1. MySQL Installation & Configuration 1.1 Remove Existing MariaDB yum remove mysql-libs -y 1.2 Upload and Extract RPM Bund ...

Process Overview To extract keys from JSON data in Hive, follow these steps: Step Action 1 Create a Hive table 2 Load JSON data into the table 3 Extract keys from the JSON Step 1: Create a Hive Table Define a table to store JSON strings. Use the following SQL command: CREATE TABLE IF NOT EXISTS json_data_table ( json_content S ...

Chapter 3: HDFS http://master:50070 1.1 NameNode NameNode is the manager. It stores metadata, which is data about data. Files in HDFS are split into data blocks of 128 MB (originally 64 MB) for storage. Replication strategy: The default replication factor in HDFS is 3. 1.2 Secondary NameNode 1.3 DataNode 1.4 Client 1.5 File Write Process NameN ...

Hive Introduction What is Hive? Hive is an open-source data warehouse solution originally developed by Facebook that operates on Hadoop infrastructure It provides SQL-like query capabilities (HQL) for structured data stored in HDFS Core functionality involves translating SQL queries into MapReduce jobs Primary use case: batch data analytics wi ...

HBase relies on several key configuration files to manage its distributed, column-oriented NoSQL database behavior. These files define critical settings for cluster operation, integration with HDFS, ZooKeeper coordination, and performance tuning. Core Configuration Files The primary configuration files include: hbase-site.xml: Contains site-sp ...

Interaction with the Hive environment following deployment utiliezs specific service interfaces. Verification of available operations begins with the built-in help utility. hive --service help Executing this displays supported components including cli, beeline, hiveserver2, and various utility tools. Configuration paths and auxiliary JAR depen ...

Configuring TaskManager Hostnames Each TaskManager must be configured with its respective hostname in flink-conf.yaml: taskmanager.host: hadoop103 On another node: taskmanager.host: hadoop104 Starting and Stopping a Standalone Cluster From the JobManager node (hadoop102): # Start cluster bin/start-cluster.sh # Stop cluster bin/stop-cluster.s ...

To implement LZO compression within a HBase environment running on Hadoop, it is necessary to compile the native LZO libraries and the corresponding Hadoop-LZO Java bridge from source. Older guides often reference the deprecated hadoop-gpl-compression library, which is incompatible with modern Hadoop versions. The following procedure outlines t ...

This guide walks through the steps to set up a Hadoop 2.10 pseudo-distributed cluster on a single CentOS 7 virtual machine. 1. Create a Hadoop User and Group We will create a dedicated user hdfs and configure it with appropriate permissions. As root user: Create the hdfs user and set a password: adduser hdfs passwd hdfs Add the user to the hdf ...

Binary Extraction and Setup Acquire the Apache Hive 2.3.6 binary archive from the official distribution repository. Extract the contents to a standard application directory and establish a symbolic link for simplified version management. tar -xzf apache-hive-2.3.6-bin.tar.gz -C /usr/local/ cd /usr/local sudo ln -s apache-hive-2.3.6-bin hive E ...