The Essence of Architecture

Architecture is fundamentally about reorganizing a system into an ordered state, a continuous process of system evolution. How does architecture transform disorder into order? The primary methods are decomposition and integration. Decomposition allows developers to focus on specific business domains and skills, enabling agile development. Integration makes the system more flexible, adaptable to chenging requirements, and supports business agility.

Architecture Classification

1. Business Architecture helps developers understand the system at a conceptual level, including business processes, modules, inputs/outputs, and business domains.

2. Application Architecture aids in system implementation at a logical level, defining data interactions, application forms, and communication methods. This makes the system's logic easier to comprehend. For example, SOA (Service-Oriented Architecture) falls under this category.

3. Technical Architecture primarily addresses technology platform selection, such as operating systems, middleware, hardware, multi-datacenter deployment, horizontal scaling, and high availability.

Ultimately, systems and architectures serve people. A highly ordered system with logical design and clear business concepts is paramount. When designing architecture, prioritize business and application architecture.

Evolusion of Large-Scale Website Architecture

Let's trace the evolution starting from an e-commerce website:

1. Single Server: Application services and database reside on the same server.
2. Database Separation: As the single server becomes overloaded, the database is separated from the application. This is driven by resource bottlenecks (CPU, file I/O, network I/O, memory). By dedicating a machine to the database, its I/O and CPU resources are isolated, improving performance. Key resource consumption factors include context switching (due to thread scheduling), file I/O (e.g., logging), network I/O (bandwidth), and memory issues (leaks, overflow).
3. Application Server Clustering: As application servers become overloaded, we move towards a clustered setup. This introduces two challenges:
   • Request Routing: How to direct user requests to the appropriate server? Solutions include DNS or dedicated load balancers.
   • Session Management: How to maintain user sessions across multiple servers?

Vertical and Horizontal Scaling

For large distributed systems, we aim for simple, elegant scaling to handle increasing traffic and data. This means scaling without modifying the software, just by upgrading hardware or adding machines. This is known as elastic design.

Vertical Scaling: Upgrading or adding hardware to a single machine. It's technically simpler and has lower operational costs but has performance limits and high costs for high-end machines. Increasing CPU cores can boost service capacity but may exacerbate lock contention, have limited impact on single-threaded tasks, and fixed thread pools. Increasing memory improves response speed but may not help if the JVM heap is fixed.

Horizontal Scaling: Adding more machines to handle growth. Theoretically, it has no upper limit but is technically more complex and poses greater operational challenges. A combination of both approaches is often used, balancing hardware upgrade costs against software modification costs.

Introducing Load Balancers

Load balancers route requests to backend servers. Common algorithms include:

• Round Robin: Distributes requests sequentially. Simple but ignores server performance differences.
• Random: Selects a server randomly. Effective with high request volumes but may be imbalanced with low traffic.
• Source IP Hash: Maps a client's IP to a server using a hash function. Ansures the same client always hits the same server if the server list is unchanged.
• Weighted Round Robin: Assigns higher weights to more capable servers, distributing requests proportionally to their capacity.
• Least Connections: Routes to the server with the fewest active connections, optimizing resource utilization.

Session Management Challenges

HTTP is stateless, meaning each request is independent. However, many applications require stateful interactions. The session-cookie mechanism addresses this by assigning a unique session ID, stored in a cookie, to track user sessions.

When scaling to multiple application servers, session management becomes complex. Solutions include:

• Session Replication: Synchronizing session data across servers. Simple but bandwidth-intensive and memory-consuming as the cluster grows.
• Centralized Storage (e.g., Redis): Storing sessions in an external, shared store. Reduces per-server memory usage but introduces network latency and dependency on the external store.
• Client-Side Storage (Cookies):strong>: Storing session data in the client's browser. Eliminates external dependencies but poses security risks and has size limitations. Encryption is necessary.

Database Scaling: Read-Write Separation

Separating read and write operations can alleviate database load. This introduces two challenges:

• Data Synchronization: How to replicate data to read replicas? Database systems like MySQL offer Master-Slave replication.
• Application Routing: How does the application direct read/write requests to the appropriate database?

Search Engines as a Read-Only Data Source

Search engines can offload read pressure from databases, offering faster search capabilities. They build indexes from the data they index. Indexing strategies include:

• Full/Incremental: Full builds for initial setup or reconstruction; incremental updates for ongoing changes.
• Real-time/Non-real-time: Real-time indexing provides immediate updates but can impact the source database; non-real-time indexing offers better protection for the source.

Accelerating Data Access: Caching and Distributed Storage

Large websites focus on storage and computation. Distributed storage is a critical component.

• Distributed File Systems: For large files (e.g., images), systems like TFS, GFS, or HDFS are used.
• NoSQL Databases: Positioned between file systems and relational databases, they offer flexibility for diverse data types and access patterns.
• Data Caching: Caches (e.g., Redis) store frequently accessed data to reduce database load. Hot data is pre-loaded or updated on database changes.
• Page Caching: Caching entire pages or content for high-traffic or dynamic pages.

Addressing Relational Database Limitations with Distributed Storage

While relational databases solve most problems, NoSQL systems (e.g., Redis, MongoDB, HBase) address specific limitations. Choosing the right distributed system for the data type and access pattern significantly improves performance, offering high capacity, concurrency, and redundancy.

Database Bottlenecks After Read-Write Separation

Even with read-write separation and NoSQL, the primary database can still face bottlenecks. Data sharding (vertical and horizontal) is the solution.

Vertical Sharding: Dedicated Databases

Splitting different business data into separate databases. This requires handling cross-database transactions, either through distributed transactions or by relaxing transaction requirements.

Horizontal Sharding: Scaling Beyond Vertical Limits

Splitting a single table's data across multiple databases. This differs from read-write separation (which addresses read pressure) and vertical sharding (which splits tables). Each shard contains a subset of the data.

Challenges of Horizontal Sharding

• SQL Routing: Determining the correct database for a request based on a routing key.
• Global ID Generation: Replacing auto-incrementing IDs with globally unique identifiers.
• Cross-Database Queries: Complex queries spanning multiple shards, especially with pagination.

Application Challenges After Database Scaling

As applications grow, they need to be split into smaller, more focused services.

• Business Function Splitting: Dividing the application by business domain (e.g., user, product, transaction services).
• Service-Oriented Architecture (SOA): A layered approach with web frontends, service centers, and database layers. This introduces remote service calls, centralized shared code, and specialized database interaction within services.

SOA offers clearer architecture, improved code quality (via dedicated teams), and better resource management. However, it increases system complexity and dependency management.

Distributed Architecture

A distributed system consists of components on networked computers that communicate and coordinate via messages.

• Components are distributed across networked computers.
• Components communicate and coordinate solely through message passing.

Distributed systems provide scalability, fault tolerance, and availability that single machines cannot. As single-machine processing becomes less cost-effective and hits performance limits, distributed systems become essential.