Network Topology and Host Allocation

Establishing bidirectional data synchronization requires compiling the Repcached fork, which extends standard Memcached with real-time state propagation. The virtual IP ensures applications interact with a single logical address regardles of node availability.

Source Preparation and Compilation

Both hosts must build the required event library and the replication-enabled daemon. A minor source modification is necesary to bypass socket I/O vector restrictions on contemporary Linux distributions.

tar -xzf libevent-2.1.12-stable.tar.gz
cd libevent-2.1.12-stable
./configure --prefix=/opt/libevent
make -j$(nproc) && make install

cd ..
tar -xzf memcached-1.2.8-repcached-2.2.1.tar.gz
cd memcached-1.2.8-repcached-2.2.1

# Patch memcached.c to explicitly define IOV_MAX for modern kernels
cat > /tmp/iovec_patch.sed << 'EOF'
/\/\* FreeBSD 4\.x doesn't have IOV_MAX exposed\. \*\//,/^#endif/{
  s/#if defined(__FreeBSD__) || defined(__APPLE__)/\/\/#if defined(__FreeBSD__) || defined(__APPLE__/
  s/# define IOV_MAX 1024/# define IOV_MAX 1024/
  s/#endif$/\/\/#endif/
}
EOF
sed -i -f /tmp/iovec_patch.sed memcached.c

./configure --prefix=/opt/memcached-rep --with-libevent=/opt/libevent --enable-replication
make -j$(nproc) && make install

Runtime Configuration Flags

The compiled binary introduces several replication-specific arguments alongside standard caching parameters:

• -x <peer_ip>: Specifies the remote host for bidirectional data synchronization.
• -X <listen:connect>: Defines the TCP port range dedicated to replication traffic (defaults to 11212).
• -m <mb>: Allocates maximum heap memory for cached objects.
• -d: Detaches the process to run as a background daemon.
• -u <user>: Drops privileges after binding to network interfaces.

Service Initialization

Launch the replication daemons on each host, pointing the -x flag toward the counterpart's physical address.

# Execute on 10.32.161.131
/opt/memcached-rep/bin/memcached -d -u root -m 128 -x 10.32.161.132

# Execute on 10.32.161.132
/opt/memcached-rep/bin/memcached -d -u root -m 128 -x 10.32.161.131

VRRP Failover Orchestration

Keepalived manages the floating address using the Virtual Router Redundancy Protocol. Each node requires a tailored configuration file that defines priority, authentication, and a health-check hook.

Primary Node Configuration (10.32.161.131)

global_defs {
    router_id cache_cluster_01
}

vrrp_script verify_cache_service {
    script "/opt/health/cache_monitor.sh"
    interval 2
    fall 2
    rise 1
}

vrrp_instance VIP_MEMCACHE {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 150
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass secure_passphrase
    }
    virtual_ipaddress {
        10.32.161.130
    }
    track_script {
        verify_cache_service
    }
}

Secondary Node Configuration (10.32.161.132)

global_defs {
    router_id cache_cluster_02
}

vrrp_script verify_cache_service {
    script "/opt/health/cache_monitor.sh"
    interval 2
    fall 2
    rise 1
}

vrrp_instance VIP_MEMCACHE {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass secure_passphrase
    }
    virtual_ipaddress {
        10.32.161.130
    }
    track_script {
        verify_cache_service
    }
}

Activate the VRRP daemon on both systems:

systemctl enable --now keepalived

Health Monitoring Routine

Deploy the following executable to evaluate the socket state. The script executes at fixed intervals defined in the VRRP instance. If the cache port becomes unreachable, the local Keepalived process terminates to trigger VIP migration.

#!/usr/bin/env bash
ACTIVE_LISTENERS=$(ss -tln | grep -c ":11211")

if [[ ${ACTIVE_LISTENERS} -eq 0 ]]; then
    systemctl disable --now keepalived
fi

Assign executable permissions: chmod +x /opt/health/cache_monitor.sh

Cluster Validation

Connect to the virtual gateway and issue diagnostic commands to confirm bidirectional synchronizasion and data persistence. The stats output will display replication metadata alongside standard performance metrics.

$ nc 10.32.161.130 11211
stats
STAT pid 28491
STAT uptime 3042
STAT time 1710552800
STAT version 1.2.8
STAT pointer_size 64
STAT curr_items 1
STAT total_items 1
STAT bytes 96
STAT curr_connections 5
STAT total_connections 10
STAT connection_structures 6
STAT cmd_get 2
STAT cmd_set 2
STAT get_hits 1
STAT get_misses 1
STAT limit_maxbytes 134217728
STAT threads 4
STAT accepting_conns 1
STAT replication MASTER
STAT repcached_version 2.2.1
STAT repcached_qi_free 8191
STAT repcached_wdata 0
STAT repcached_wsize 2048
END

set validation_key 0 300 14
sample_payload
STORED
get validation_key
VALUE validation_key 0 14
sample_payload
END
quit