Understanding HTTP Idempotency

HTTP idempotency guarantees that making multiple identical requests produces the same result as a single request. This becomes critical when network failures cause clients to retry opertaions, ensuring that duplicate requests don't alter server state unexpectedly.

The HTTP specification classifies methods by their idempotent properties:

Without idempotency guarantees, a user experiencing poor network connectivity might submit a payment request multiple times, resulting in duplicate charges. Implementing idempotency prevents such scenarios.

Idempotency Key Strategy

The solution involves requiring clients to supply a unique identifier with each request. The server validates this identifier and rejects duplicates.

Client request example:

POST /api/v1/transactions HTTP/1.1
Host: api.example.com
Content-Type: application/json
X-Idempotency-Key: a1b2c3d4-e5f6-7890-abcd-ef1234567890

{
  "user_id": 42,
  "amount": 100.00,
  "currency": "USD"
}

When the server receives this request, it first checks whether the X-Idempotency-Key exists in storage. If found, the request is rejected with a conflict response, preventing duplicate processing.

Middleware Implementation in Kratos

The following example demonstrates idempotency enforcement using Redis as the backing store:

type contextKeyType string

const (
    idempotencyCtxKey contextKeyType = "idempotency-key"
)

func InjectIdempotencyContext(ctx context.Context, key string) context.Context {
    return context.WithValue(ctx, idempotencyCtxKey, key)
}

func IdempotencyGuard(redisClient *redis.Client) middleware.Middleware {
    return func(next middleware.Handler) middleware.Handler {
        return func(ctx context.Context, reqData any) (any, error) {
            var idempotencyKey string

            if serverCtx, ok := http.RequestFromServerContext(ctx); ok {
                idempotencyKey = serverCtx.Header.Get("X-Idempotency-Key")
                
                if idempotencyKey != "" {
                    exists, err := redisClient.Exists(ctx, idempotencyKey).Result()
                    if err != nil {
                        return nil, err
                    }
                    
                    if exists > 0 {
                        return nil, v1.ErrorErrorReasonErrorConflict(
                            "duplicate idempotency key detected",
                        )
                    }
                    
                    ctx = InjectIdempotencyContext(ctx, idempotencyKey)
                }
            }

            result, err := next(ctx, reqData)
            
            if err == nil && idempotencyKey != "" {
                storeErr := redisClient.Set(
                    ctx, 
                    idempotencyKey, 
                    "processed", 
                    24*time.Hour,
                ).Err()
                if storeErr != nil {
                    return nil, storeErr
                }
            }

            return result, err
        }
    }
}

The middleware intercepts incoming requests, extracts the idempotency key from headers, verifies uniqueness against Redis, and records processed keys to block future duplicates.

Verification Tests

func TestIdempotencyProtection(t *testing.T) {
    testKey := "test-uuid-1234-5678"

    verifyResponse := func() *http.Response {
        request, err := http.NewRequest(
            "POST", 
            "http://127.0.0.1:8080/api/orders", 
            strings.NewReader(`{}`),
        )
        if err != nil {
            t.Fatalf("Failed to construct request: %v", err)
        }

        request.Header.Set("X-Idempotency-Key", testKey)
        
        httpClient := &http.Client{}
        response, err := httpClient.Do(request)
        if err != nil {
            t.Fatalf("Request execution failed: %v", err)
        }
        defer response.Body.Close()

        body, _ := io.ReadAll(response.Body)
        t.Logf("Status: %d, Body: %s", response.StatusCode, string(body))
        
        return response
    }

    first := verifyResponse()
    if first.StatusCode != 201 {
        t.Fatalf("Initial request should succeed with 201")
    }

    second := verifyResponse()
    if second.StatusCode != 409 {
        t.Fatalf("Duplicate request should fail with 409 Conflict")
    }
}

=== RUN   TestIdempotencyProtection
    test_file.go:42: Status: 201, Body: {"order_id":"ORD-001","status":"created"}
    test_file.go:42: Status: 409, Body: {"code":409,"reason":"ERROR_REASON_ERROR_CONFLICT","message":"duplicate idempotency key detected"}
--- PASS: TestIdempotencyProtection (0.02s)
PASS

Operational Considerations

When deploying idempotency middleware in production:

1. Key expiration: Set TTL values based on business requirements to prevent unbounded Redis growth
2. Header validation: Require idempotency keys for state-modifying operations
3. Graceful degradation: Decide whether to allow requests without keys or reject them
4. Response caching: Store successful responses alongside keys to return consistent results for retries

References

• REST API Design Rules - Idempotence
• HTTP Request Methods - MDN
• HTTP Status Codes