1. Project Background and Architecture

1.1 Business Context

Mobile news consumption has become dominant as smartphone usage patterns evolve. Users increasingly access content during fragmented time periods, driving demand for personalized news aggregation platforms. This system addresses these needs through a microservices architecture combined with big data analytics for intelligent content delivery.

1.2 System Overview

The platform serves as a comprehensive content analysis and distribution system with three primary interfaces:

• Mobile Client: End-user application for content consumption
• Creator Portal: Interface for content creators and publishers
• Administrative Console: Backend management system for operators

1.3 Terminology Reference

• Platform: Refers to the entire news aggregation system
• Module: Specific functional component within the platform
• End User: Mobile application consumer
• Content Creator: Registered publisher using the creator portal
• Operator: Administrative system user with management privileges
• Core Services: Backend microservices handling business logic
• Data Lake: Centralized storage for analytics and machine learning

2. Functional Requirements

2.1 Mobile Application Features

• Personalized Channels: Customizable content categories with AI-driven recommendations
• Article Feed: Display with metadata including title, thumbnails, and engagement metrics
• Search: Intelligent autocomplete with search history tracking
• User Profile: Personal dashboard for favorites, following lists, and settings
• Content Viewer: Article reading interface with like, comment, share functionality and behavioral tracking
• Identity Verification: Two-tier authentication system for converting users to content creators
• Authentication: Phone number-based registration and login with SMS verification

2.2 Creator Portal Capabilities

• Content Studio: Article lifecycle management (draft, published, rejected, withdrawn states)
• Comment Moderation: Dashboard for managing post comments and engagement settings
• Media Library: Centralized asset management for images and multimedia
• Analytics Dashboard: Performance metrics including views, comments, shares, and收藏
• Audience Insights: Demographic analysis covering gender, age, device type, and content preferences

2.3 Administrative Functions

• User Governance: CRUD operations with account suspension capabilities
• Verification Workflow: Multi-stage approval for identity and资质认证
• Content Curation: Direct article management with pinning and editorial controls
• Compliance Review: Automated and manual screening for sensitive content
• Category Management: Channel and tag taxonomy administration
• Platform Analytics: DAU, traffic trends, geographic distribution, and search analytics
• Content Metrics: Collection volume, publication rates, engagement statistics
• Access Control: Role-based permission management for admin accounts

3. Technology Stack

3.1 Frontend Technologies

• Cross-Platform Framework: Vue.js with Weex for mobile UI
• Data Visualization: ECharts for analytics dashboards
• Real-time Communication: WebSocket for push notifications and live updates

3.2 Backend Infrastructure

• API Gateway: Spring Cloud Gateway for routing, rate limiting, and circuit breaking
• Microservices Framework: Spring Boot 2.x with Spring Cloud ecosystem
• Service Mesh: Internal service discovery and configuration management
• Message Queue: Apache Kafka for event streaming and asynchronous processing
• Database Sharding: MyCat for horizontal partitioning
• Caching Layer: Redis for hot data and session management
• Distributed Coordination: Zookeeper for cluster management

3.3 Data Storage Strategy

• Transactional Data: MySQL for user accounts and core business entities
• Hot Data: MongoDB for frequently accessed content and user activities
• File Storage: FastDFS for static resource management with intelligent caching
• Cold Storage: HBase for archival data and historical records
• Search Engine: Elasticsearch for full-text search across content
• Graph Database: Neo4j for relationship mapping and knowledge graph construction

3.4 Big Data Processing

• Data Ingestion: Sqoop and Kettle for ETL operations
• Batch Processing: Spark with Hive for offline analytics
• Stream Processing: Spark Streaming integrated with Kafka for real-time recommmendations
• Machine Learning: AI pipelines for automated verification and content classification

4. Database Architecture

4.1 Sharding Strategy

The platform implements database sharding across five core schemas to handle scale and separation of concerns:

• User Schema: Stores user profiles, authentication, and social relationships
• Behavior Schema: Captures all user interactions including likes, shares, and comments
• Creator Schema: Manages content creator accounts and performance metrics
• Crawler Schema: Houses web-scraped content before processing
• Admin Schema: Contains administrative configurations and audit logs

4.2 Data Flow Architecture

1. Content Ingestion: Web crawlers populate the crawler database with external articles
2. Creator Submission: Original content enters through the creator database
3. Moderation Pipeline: Both streams converge through a review process before publication
4. User Interaction: All user engagement data flows into the behavior database
5. Analytics Loop: Aggregated data feeds recommendation engines and dashboards

4.3 Denormalization Principles

The system employs logical associations without foreign key constraints to optimize query performance. Redundant fields are strategically duplicated across tables to eliminate expensive JOIN operations, particularly for high-frequency queries. For example, creator names are stored directly in article records to avoid user table lookups.

5. Database Development Standards

5.1 Table Design Rules

• Boolean Fields: Must use is_xxx naming with tinyint(1) type (1=true, 0=false)
• Naming Convention: Lowercase letters, numbers, and underscores only; no leading digits or double underscores with numbers
• Singular Names: Table names should be singular (e.g., user, not users)
• Reserved Words: Avoid MySQL keywords like desc, range, match
• Primary Keys: Must include id field with auto-increment or distributed ID strategy
• Field Comments: Every column requires a descriptive comment
• Decimal Types: Use decimal instead of float or double for precision
• String Length: Use char for fixed-length strings; varchar max 5000; larger text requires separate table

5.2 Indexing Guidelines

• Unique Constraints: Business-unique fields must have unique indexes
• JOIN Limitations: Avoid more than three table joins; ensure indexed join fields
• Partial Indexes: For varchar fields, specify index length based on cardinality analysis
• Left Prefix Matching: Prevent left-side wildcards that invalidate B-Tree indexes
• Composite Indexes: Order columns by selectivity, placing highest cardinality first
• Covering Indexes: Design indexes to include frequently queried fields to avoid table lookups
• Performance Target: Optimize queries to achieve atleast range level, ideally ref or const

6. Project Structure Setup

6.1 Module Organization

The backend follows a hierarchical Maven structure with a parent POM:

<parent>
    <groupId>com.newsplatform</groupId>
    <artifactId>news-platform-parent</artifactId>
    <version>1.0.0</version>
</parent>

Shared Modules:

• platform-common: Centralized configurations for Redis, Kafka, security
• platform-models: Entity definitions, DTOs, enumerations, and MyBatis mappers
• platform-utils: Common utilities for encryption, date handling, JSON processing
• platform-api-definitions: Service interface contracts

Microservices:

• user-service: Authentication and user profile management
• content-service: Article retrieval, search, and channel management
• engagement-service: User behavior tracking and analytics
• creator-service: Content creator portal backend
• admin-service: Administrative operations and moderation
• gateway-service: API gateway and routing

6.2 Development Environment Prerequisites

• JDK 1.8 or higher
• IntelliJ IDEA or Eclipse
• Maven 3.6+
• MySQL 5.7+
• Redis 5.0+
• Docker (for containerized services)

6.3 IDE Configuration

Configure Maven settings to use a consistent local repository. Set project encoding to UTF-8 across all modules. Enable annotation processing for Lombok integration.

6.4 Initial Project Import

Extract the project archive to a path without spaces or special characters. Import as Maven project in IDE and resolve dependencies. Verify module interdependencies before proceeding.

7. Backend Development Guidelines

7.1 Layered Architecture Principles

Design Flow: Top-down approach from presentation to persistence layer Implementation Order: Bottom-up development starting with database schema Single Responsibility: Each class and method should have one clear purpose Interface Segregation: Depend on abstractions, not concrete implementations

7.2 Standard Development Workflow

1. Schema Definition: Design and implement database tables with proper indexing
2. Entity Mapping: Create JPA or MyBatis-Plus entity classes
3. API Contract: Define request/response DTOs and service interfaces
4. Persistence Layer: Implement data access objects with MyBatis-Plus or custom queries
5. Business Logic: Develop service layer with transactional boundaries
6. Controller Implementation: Expose REST endpoints with proper HTTP semantics
7. Testing Strategy: Unit tests for services, integration tests for APIs

7.3 API Versioning Strategy

Version endpoints at both package and URL levels:

package com.newsplatform.content.controller.v1;

@RestController
@RequestMapping("/api/v1/channels")
public class ChannelController {
    // Implementation
}

8. Common Components and Patterns

8.1 Standardized API Response

@Data
@NoArgsConstructor
public class ApiResponse<T> implements Serializable {
    private String requestId;
    private Integer statusCode;
    private String message;
    private T payload;
    private Long timestamp;

    public static <T> ApiResponse<T> success(T data) {
        ApiResponse<T> response = new ApiResponse<>();
        response.setStatusCode(200);
        response.setPayload(data);
        response.setTimestamp(System.currentTimeMillis());
        return response;
    }

    public static <T> ApiResponse<T> error(Integer code, String errorMessage) {
        ApiResponse<T> response = new ApiResponse<>();
        response.setStatusCode(code);
        response.setMessage(errorMessage);
        response.setTimestamp(System.currentTimeMillis());
        return response;
    }
}

8.2 Pagination Support

@Data
@EqualsAndHashCode(callSuper = true)
public class PaginatedResponse<T> extends ApiResponse<List<T>> {
    private Integer currentPage;
    private Integer pageSize;
    private Long totalElements;
    private Integer totalPages;

    public PaginatedResponse(Integer page, Integer size, Long total) {
        this.currentPage = page;
        this.pageSize = size;
        this.totalElements = total;
        this.totalPages = (int) Math.ceil((double) total / size);
    }
}

8.3 Request Validation

@Data
public abstract class PageableRequest {
    @Min(1)
    private Integer pageNumber = 1;
    
    @Min(1)
    @Max(100)
    private Integer pageSize = 10;

    public void normalize() {
        if (pageNumber == null || pageNumber < 1) {
            pageNumber = 1;
        }
        if (pageSize == null || pageSize < 1 || pageSize > 100) {
            pageSize = 10;
        }
    }
}

8.4 Exception Handling

public enum ErrorCode {
    SUCCESS(200, "Operation successful"),
    INVALID_PARAMETER(400, "Invalid request parameter"),
    UNAUTHORIZED(401, "Authentication required"),
    FORBIDDEN(403, "Access denied"),
    NOT_FOUND(404, "Resource not found"),
    CONFLICT(409, "Resource already exists"),
    INTERNAL_ERROR(500, "System error");

    private final int code;
    private final String description;

    ErrorCode(int code, String description) {
        this.code = code;
        this.description = description;
    }

    public ApiResponse<?> toResponse() {
        return ApiResponse.error(code, description);
    }
}

9. Multi-Environment Configuration

9.1 Profile Management

Each microservice includes three property files:

• application-dev.yml: Development environment settings
• application-test.yml: Testing environment configuration
• application-prod.yml: Production deployment parameters

Default profile is set to test in the parent POM:

<profiles>
    <profile>
        <id>test</id>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
    </profile>
</profiles>

Build commands specify the target environment:

mvn clean package -P prod

9.2 Configuration Priorities

Environment-specific configurations override defaults. Sensitive credentials are externalized using Spring Cloud Config or environment variables in production deployments.

10. Channel Management Implementation

10.1 Domain Model

@Data
@TableName("content_channel")
public class Channel implements Serializable {
    @TableId(type = IdType.AUTO)
    private Long channelId;
    
    @TableField("channel_name")
    private String name;
    
    @TableField("description")
    private String description;
    
    @TableField("is_default")
    private Boolean isDefault;
    
    @TableField("is_active")
    private Boolean isActive;
    
    @TableField("display_order")
    private Integer sortOrder;
    
    @TableField("created_at")
    private LocalDateTime createdAt;
}

10.2 API Contract Definition

@Api(tags = "Channel Management")
public interface ChannelOperations {
    
    @ApiOperation("Retrieve paginated channel list with optional name filter")
    ApiResponse<PaginatedResponse<Channel>> searchChannels(ChannelQuery query);
    
    @ApiOperation("Create new channel")
    ApiResponse<Channel> createChannel(@Valid Channel channel);
    
    @ApiOperation("Update existing channel")
    ApiResponse<Void> modifyChannel(@Valid Channel channel);
    
    @ApiOperation("Deactivate channel (soft delete)")
    ApiResponse<Void> deactivateChannel(@PathVariable Long id);
}

10.3 Data Access Layer

@Mapper
public interface ChannelRepository extends BaseMapper<Channel> {
    
    default IPage<Channel> findByCriteria(ChannelQuery query) {
        LambdaQueryWrapper<Channel> wrapper = new LambdaQueryWrapper<>();
        
        if (StringUtils.isNotBlank(query.getKeyword())) {
            wrapper.like(Channel::getName, query.getKeyword());
        }
        
        wrapper.orderByAsc(Channel::getSortOrder);
        
        Page<Channel> page = new Page<>(query.getPageNumber(), query.getPageSize());
        return selectPage(page, wrapper);
    }
}

10.4 Business Logic Layer

@Service
@Transactional
public class ChannelService implements ChannelOperations {
    
    @Autowired
    private ChannelRepository channelRepo;
    
    @Override
    public ApiResponse<PaginatedResponse<Channel>> searchChannels(ChannelQuery query) {
        query.normalize();
        
        IPage<Channel> result = channelRepo.findByCriteria(query);
        
        PaginatedResponse<Channel> paginated = new PaginatedResponse<>(
            query.getPageNumber(), 
            query.getPageSize(), 
            result.getTotal()
        );
        paginated.setPayload(result.getRecords());
        
        return ApiResponse.success(paginated);
    }
    
    @Override
    public ApiResponse<Channel> createChannel(Channel channel) {
        if (channel == null || StringUtils.isBlank(channel.getName())) {
            return ErrorCode.INVALID_PARAMETER.toResponse();
        }
        
        channel.setCreatedAt(LocalDateTime.now());
        channel.setIsActive(true);
        channelRepo.insert(channel);
        
        return ApiResponse.success(channel);
    }
    
    @Override
    public ApiResponse<Void> modifyChannel(Channel channel) {
        if (channel == null || channel.getChannelId() == null) {
            return ErrorCode.INVALID_PARAMETER.toResponse();
        }
        
        Channel existing = channelRepo.selectById(channel.getChannelId());
        if (existing == null) {
            return ErrorCode.NOT_FOUND.toResponse();
        }
        
        channelRepo.updateById(channel);
        return ApiResponse.success(null);
    }
    
    @Override
    public ApiResponse<Void> deactivateChannel(Long id) {
        if (id == null) {
            return ErrorCode.INVALID_PARAMETER.toResponse();
        }
        
        Channel channel = channelRepo.selectById(id);
        if (channel == null) {
            return ErrorCode.NOT_FOUND.toResponse();
        }
        
        if (Boolean.TRUE.equals(channel.getIsActive())) {
            return ErrorCode.CONFLICT.toResponse();
        }
        
        channel.setIsActive(false);
        channelRepo.updateById(channel);
        
        return ApiResponse.success(null);
    }
}

10.5 REST Controller

@RestController
@RequestMapping("/api/v1/channels")
public class ChannelController implements ChannelOperations {
    
    @Autowired
    private ChannelService channelService;
    
    @PostMapping("/search")
    public ApiResponse<PaginatedResponse<Channel>> searchChannels(@RequestBody ChannelQuery query) {
        return channelService.searchChannels(query);
    }
    
    @PostMapping
    public ApiResponse<Channel> createChannel(@Valid @RequestBody Channel channel) {
        return channelService.createChannel(channel);
    }
    
    @PutMapping
    public ApiResponse<Void> modifyChannel(@Valid @RequestBody Channel channel) {
        return channelService.modifyChannel(channel);
    }
    
    @DeleteMapping("/{id}")
    public ApiResponse<Void> deactivateChannel(@PathVariable Long id) {
        return channelService.deactivateChannel(id);
    }
}

11. API Documentation and Testing

11.1 Swagger Integration

Add dependencies to the common module:

<dependency>
    <groupId>io.springfox</groupId>
    <artifactId>springfox-swagger2</artifactId>
    <version>2.9.2</version>
</dependency>
<dependency>
    <groupId>io.springfox</groupId>
    <artifactId>springfox-swagger-ui</artifactId>
    <version>2.9.2</version>
</dependency>

Configuration class:

@Configuration
@EnableSwagger2
public class SwaggerConfig {
    
    @Bean
    public Docket api() {
        return new Docket(DocumentationType.SWAGGER_2)
                .select()
                .apis(RequestHandlerSelectors.basePackage("com.newsplatform"))
                .paths(PathSelectors.any())
                .build()
                .apiInfo(metadata());
    }
    
    private ApiInfo metadata() {
        return new ApiInfoBuilder()
                .title("News Platform API Documentation")
                .description("RESTful APIs for content management and user engagement")
                .version("1.0.0")
                .build();
    }
}

Access documentation at: http://localhost:8080/swagger-ui.html

11.2 Enhanced Documentation with Knife4j

For improved UI and additional features, integrate Knife4j:

<dependency>
    <groupId>com.github.xiaoymin</groupId>
    <artifactId>knife4j-spring-boot-starter</artifactId>
    <version>2.0.4</version>
</dependency>

Enhanced configuration:

@Configuration
@EnableSwagger2
@EnableKnife4j
@Import(BeanValidatorPluginsConfiguration.class)
public class Knife4jConfig {
    
    @Bean
    public Docket defaultApi() {
        return new Docket(DocumentationType.SWAGGER_2)
                .apiInfo(apiInfo())
                .groupName("default")
                .select()
                .apis(RequestHandlerSelectors.basePackage("com.newsplatform"))
                .paths(PathSelectors.any())
                .build();
    }
    
    private ApiInfo apiInfo() {
        return new ApiInfoBuilder()
                .title("News Platform API")
                .description("Enhanced API documentation with Knife4j")
                .termsOfServiceUrl("https://newsplatform.com")
                .version("1.0")
                .build();
    }
}

Access enhanced UI at: http://localhost:8080/doc.html

11.3 Automated Testing with Postman

Create test collections for all endpoints. Export environment variables for different deployment stages. Implement pre-request scripts for authentication token injection and test scripts for response validation.

Example test script for channel creation:

pm.test("Channel created successfully", function() {
    pm.response.to.have.status(200);
    var jsonData = pm.response.json();
    pm.expect(jsonData.statusCode).to.eql(200);
    pm.expect(jsonData.payload).to.have.property('channelId');
});

12. Deployment Considerations

12.1 Containerization

Prepare Dockerfiles for each microservice with multi-stage builds to minimize image size. Use Docker Compose for local orchestration and Kubernetes for production deployments.

12.2 Monitoring and Observability

Integrate Spring Boot Actuator for health checks and metrics. Configure centralized logging with ELK stack. Implement distributed tracing with Sleuth and Zipkin.

12.3 CI/CD Pipeline

Establish Jenkins or GitLab CI pipelines with stages for:

• Static code analysis (SonarQube)
• Unit and integration testing
• Security scanning
• Artifact building
• Container image creation
• Deployment to test/staging/production