Introduction to Spring AI Advisors

Spring AI has recently introduced a new feature set in its 1.0.0-M3 milestone release. Although not yet stable, this version brings in powerful new capabilities that allow developers to enhance AI applications through interceptors known as Advisors. These components are designed to modify or enrich the input and output of chat-based AI interactions, enabling reusable and modular logic across different models and use cases.

Advisors are conceptually similar to aspects in Spring AOP, where they intercept and manipulate execution flows. They are especially useful in AI applications for tasks like logging, memory management, input validation, and prompt engineering.

Core Concepts

What is an Advisor?

An Advisor is a component that can intercept and modify the request and resposne flow in a chat-based AI system. The key interface for defining custom behavior is CallAroundAdvisor and StreamAroundAdvisor, depending on whether the application uses synchronous or reactive (streaming) responses.

Key Benefits of Advisors

• Encapsulation of common AI patterns into reusable modules.
• Enhancement of input prompts and output formatting for better model performence.
• Portability across different models and use cases.

How Advisors Work

Advisors operate in a chain, where each advisor can modify the request before it reaches the model and the response before it is returned to the client. This allows for a layered approach to request processing, much like filters in a web application or aspects in AOP.

Here’s a simplified view of the processing flow:

1. The client sends a request with input text and parameters.
2. Each advisor in the chain gets a chance to modify the request.
3. The final request is sent to the LLM (Large Language Model).
4. The response flows back through the advisor chain, where it can be further modified.
5. The final enhanced response is returned to the client.

Built-in Advisors

Spring AI comes with several built-in advisors that demonstrate common use cases:

• MessageChatMemoryAdvisor: Stores chat history in memory to provide context for subsequent interactions.
• PromptChatMemoryAdvisor: Embeds chat history into the system prompt rather than using a messages list.
• QuestionAnswerAdvisor: Implements RAG (Retrieval-Augmented Generation) by retrieving relevant documents before generating a response.
• SafeGuardAdvisor: Filters out sensitive or inappropriate input before it reaches the model.
• SimpleLoggerAdvisor: Logs requests and responses for debugging and monitoring purposes.
• VectorStoreChatMemoryAdvisor: Uses a vector database to store and retrieve chat history for long-term memory.

Managing Conversation IDs

Many of these advisors rely on a chat_memory_conversation_id to track conversation state. It's important to manage this ID properly to avoid data pollution in the vector store or memory. Developers should ensure this ID is unique per user and persisted appropriately.

Creating a Custom Advisor

Let's implement a custom advisor that enhances user queries by prompting the model to re-read the input question. This pattern is inspired by the Re2 technique for improving model accuracy.

public class ReReadAdvisor implements CallAroundAdvisor, StreamAroundAdvisor {
    private static final String PROMPT_TEMPLATE = """
        {re2_query}
        Please read the question again: {re2_query}
    """;

    @Override
    public String getName() {
        return "ReReadAdvisor";
    }

    @Override
    public int getOrder() {
        return 0;
    }

    private AdvisedRequest prepareRequest(AdvisedRequest request) {
        String originalQuery = request.userText();
        Map<string object=""> params = new HashMap<>(request.userParams());
        params.put("re2_query", originalQuery);

        return AdvisedRequest.from(request)
                .withUserText(PROMPT_TEMPLATE)
                .withUserParams(params)
                .build();
    }

    @Override
    public AdvisedResponse aroundCall(AdvisedRequest request, CallAroundAdvisorChain chain) {
        return chain.nextAroundCall(prepareRequest(request));
    }

    @Override
    public Flux<AdvisedResponse> aroundStream(AdvisedRequest request, StreamAroundAdvisorChain chain) {
        return chain.nextAroundStream(prepareRequest(request));
    }
}
</string>

Passing and Updating Parameters

Advisors can be configured with parameters at runtime. These parameters can be used to customize behavior, such as setting a conversation ID or adjusting memory size.

@Bean
MessageChatMemoryAdvisor memoryAdvisor() {
    return new MessageChatMemoryAdvisor(new InMemoryChatMemory(), "default-conversation-id", 10);
}

These parameters can also be overridden dynamically when making a call:

.advisors(memoryAdvisor, loggerAdvisor)
.advisors(config -> config.param("chat_memory_conversation_id", "user-123")
        .param("chat_memory_response_size", 20))

Additionally, parameters can be updated during the execution of an advisor:

@Override
public AdvisedResponse aroundCall(AdvisedRequest request, CallAroundAdvisorChain chain) {
    AdvisedRequest updated = request.updateContext(ctx -> {
        ctx.put("advisor_name", getName());
        ctx.put("timestamp", System.currentTimeMillis());
        return ctx;
    });
    return chain.nextAroundCall(updated);
}