Setting Up Hadoop 2.7.1 on Windows and Managing HDFS Storage
Hadoop Installation on Windows
This guide covers the complete process of deploying Hadoop 2.7.1 on a Windows system, configuring HDFS, and performing file operations.
Prerequisites
Windows operating system
JDK 8 or compatible Java version installed
Hadoop 2.7.1 binary distribution from Apache archives
Windows-specific Hadoop binaries (hadoopon ...
Posted on Fri, 08 May 2026 18:54:35 +0000 by Eclesiastes
Selenium Web Scraping and Flume Data Processing Implementation
Extracting Stock Market Data with SeleniumTo retrieve financial information from dynamic web pages, Selenium is used to automate browser interactions, specifically targeting elements that load via JavaScript. The target involves extracting data from the Shanghai A-shares, Shenzhen A-shares, and aggregated boards. The data is persisted in a stru ...
Posted on Thu, 07 May 2026 11:10:01 +0000 by Ryan Sanders
ZooKeeper Watcher Mechanism: Principles and Practical Guide
ZooKeeper Watcher Mechanism: Principles and Practical Guide
This article explores the ZooKeeper Watcher mechanism, a core component for distributed coordination. It covers fundamental concepts, the publish/subscribe model, workflow, application scenarios, and practical command examples.
Background
We have three cloud servers (2C4G each) running ...
Posted on Thu, 07 May 2026 05:59:21 +0000 by markszy
Configuring and Running Spark SQL with Hive Integration
To build a Spark distribution compatible with Hadoop CDH 5.7.0 and Hive support, navigate to the Spark source directory:
[hadoop@hadoop001 spark-2.1.0]$ pwd
/home/hadoop/source/spark-2.1.0
Compile using Maven with profiles for YARN, Hadoop 2.6, Hive, and Hive Thriftserver:
./build/mvn -Pyarn -Phadoop-2.6 -Phive -Phive-thriftserver \
-Dhadoop ...
Posted on Thu, 07 May 2026 02:26:00 +0000 by kporter.porter