Serialization Requirements

A serialization algorithm must enable bidirectional conversion: Object → Byte Array → Object. During serialization, Java objects transform into transmissible data (byte[] or JSON), which ultimately converts to byte[]. Deserialization reverses this process, converting inbound data back to Java objects for processing.

Implementing Extensible Serialization

To achieve extensibility, define a serialization interface that standardizes serialization and deserialization methods. Use an enum within this interface to provide different serialization implementations while maintaining a consistent API.

Target Implementation: Support JDK native serialization and JSON (Gson) serialization, with capacity for future extensions. JSON typically produces smaller byte arrays compared to JDK serialization.

Serialization Interface

package network.serialization;

import com.google.gson.*;
import java.io.*;
import java.lang.reflect.Type;
import java.nio.charset.StandardCharsets;

public interface SerializationEngine {
    <T> T deserialize(Class<T> targetType, byte[] data);
    <T> byte[] serialize(T object);
    
    enum Algorithms implements SerializationEngine {
        JavaNative {
            @Override
            public <T> T deserialize(Class<T> targetType, byte[] data) {
                try (ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(data))) {
                    return targetType.cast(ois.readObject());
                } catch (IOException | ClassNotFoundException e) {
                    throw new DeserializationException("Failed to deserialize object", e);
                }
            }

            @Override
            public <T> byte[] serialize(T object) {
                try (ByteArrayOutputStream bos = new ByteArrayOutputStream();
                     ObjectOutputStream oos = new ObjectOutputStream(bos)) {
                    oos.writeObject(object);
                    return bos.toByteArray();
                } catch (IOException e) {
                    throw new SerializationException("Failed to serialize object", e);
                }
            }
        },
        GsonJson {
            private final Gson gson = new GsonBuilder()
                .registerTypeAdapter(Class.class, new ClassTypeAdapter())
                .create();

            @Override
            public <T> T deserialize(Class<T> targetType, byte[] data) {
                String json = new String(data, StandardCharsets.UTF_8);
                return gson.fromJson(json, targetType);
            }

            @Override
            public <T> byte[] serialize(T object) {
                String json = gson.toJson(object);
                return json.getBytes(StandardCharsets.UTF_8);
            }
        }
    }

    class ClassTypeAdapter implements JsonSerializer<Class<?>>, JsonDeserializer<Class<?>> {
        @Override
        public Class<?> deserialize(JsonElement json, Type type, JsonDeserializationContext context) {
            try {
                return Class.forName(json.getAsString());
            } catch (ClassNotFoundException e) {
                throw new JsonParseException(e);
            }
        }

        @Override
        public JsonElement serialize(Class<?> clazz, Type type, JsonSerializationContext context) {
            return new JsonPrimitive(clazz.getName());
        }
    }
}

Protocol Integration

public class ProtocolCodec extends MessageToMessageCodec<ByteBuf, ProtocolMessage> {
    private static final byte[] PROTOCOL_MAGIC = "NETTY".getBytes(StandardCharsets.UTF_8);
    private static final SerializationEngine DEFAULT_SERIALIZER = SerializationEngine.Algorithms.GsonJson;
    
    @Override
    protected void encode(ChannelHandlerContext ctx, ProtocolMessage msg, List<Object> out) {
        ByteBuf buffer = ctx.alloc().buffer();
        buffer.writeBytes(PROTOCOL_MAGIC);
        buffer.writeByte(1); // Protocol version
        buffer.writeByte(DEFAULT_SERIALIZER.ordinal());
        buffer.writeByte(msg.getType().ordinal());
        buffer.writeInt(msg.getSequenceId());
        
        byte[] payload = DEFAULT_SERIALIZER.serialize(msg);
        buffer.writeInt(payload.length);
        buffer.writeBytes(payload);
        
        out.add(buffer);
    }

    @Override
    protected void decode(ChannelHandlerContext ctx, ByteBuf buffer, List<Object> out) {
        buffer.skipBytes(4); // Magic number
        buffer.readByte(); // Version
        byte serializerId = buffer.readByte();
        byte typeId = buffer.readByte();
        int sequenceId = buffer.readInt();
        int length = buffer.readInt();
        
        byte[] data = new byte[length];
        buffer.readBytes(data);
        
        SerializationEngine serializer = SerializationEngine.Algorithms.values()[serializerId];
        ProtocolMessage message = serializer.deserialize(
            ProtocolMessage.MessageType.values()[typeId].getMessageClass(), 
            data
        );
        out.add(message);
    }
}

Network Connection Parameters

Netty's channel configuration options control various aspects of network behavior. Key parameters include:

• CONNECT_TIMEOUT_MILLIS: Connection establishment timeout
• SO_BACKLOG: Server's connection queue size
• ALLOCATOR: ByteBuf allocation strategy
• RCVBUF_ALLOCATOR: Adaptive receive buffer allocator

Connection Timeout Configuration

Client-Side Setup

public class ConnectionTimeoutClient {
    public static void main(String[] args) {
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        
        try {
            Bootstrap bootstrap = new Bootstrap()
                .group(workerGroup)
                .channel(NioSocketChannel.class)
                .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 500)
                .handler(new ChannelInitializer<SocketChannel>() {
                    @Override
                    protected void initChannel(SocketChannel ch) {
                        ch.pipeline().addLast(new LoggingHandler());
                    }
                });
                
            Channel channel = bootstrap.connect("localhost", 8080).sync().channel();
            channel.closeFuture().sync();
        } catch (Exception e) {
            System.err.println("Connection failed: " + e.getMessage());
        } finally {
            workerGroup.shutdownGracefully();
        }
    }
}

Timeout Mechanism

Netty implements connection timeout via scheduled tasks in the event loop. When timeout occurs, the failure is propagated through the Promise mechanism to waiting threads.

Backlog Parameter

Controls the server's connection queue size. The actual queue length is min(backlog, somaxconn). Linux systems use two queues during TCP handshake:

• Semi-connection queue: Requests in SYN_RCVD state
• Full-connection queue: Established connections ready for accept()

Configuration

ServerBootstrap server = new ServerBootstrap()
    .group(bossGroup, workerGroup)
    .channel(NioServerSocketChannel.class)
    .option(ChannelOption.SO_BACKLOG, 1024)
    .childHandler(new ChannelInitializer<SocketChannel>() {
        // ...
    });

Netty's default backlog value is determined by the OS's somaxconn parameter (128 on Linux, 200 on Windows).

File Descriptor Limits

Configure per-process limits using ulimit -n in Linux environments to ensure sufficient descriptors for high-concurrency scenarios.

ByteBuf Configuration

Control memory allocation behavior through JVM parameters:

Example configuration for heap-based unpooled buffers: -Dio.netty.allocator.type=unpooled -Dio.netty.noPreferDirect=true

Netty's default allocator is PooledByteBufAllocator with direct buffer preference when available.