Integrating Spring AI with DeepSeek for Streamed Chat

The Spring AI project offers Java developers a streamlined way to incorporate AI capabilities. This guide demonstrates creating a natural language conversational agent using Spring AI and the DeepSeek V3 model, implemented with streaming responses and context memory.

Prerequisites

• JDK 17 or later
• Maven or Gradle build tool
• A DeepSeek API key (Note: This example uses a SiliconFlow-hosted service)
• Spring Boot 3.2+

Project Setup

Begin by creating a standard Spring Boot project. Add the Spring AI OpenAI starter dependency to your pom.xml. This starter is used because the DeepSeek API adheres to the OpenAI specification.

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
</dependency>

Configure the application properties to point to the DeepSeek service endpoint and provide your API key.

spring.ai.openai.base-url=https://api.siliconflow.cn/
spring.ai.openai.api-key=${YOUR_API_KEY}
spring.ai.openai.chat.options.model=deepseek-ai/DeepSeek-V3
logging.level.org.springframework.ai.chat.client.advisor=DEBUG

Core Server-Side Implementation

Agent Configuration

Define a configuration class to create a ChatClient bean with a default system message, setting the AI's persona.

@Configuration
public class AiAgentConfig {

    @Bean
    public ChatClient agentClient(ChatClient.Builder builder) {
        return builder
                .defaultSystem("You are an intelligent assistant named Spring AI Bot.")
                .build();
    }
}

Streaming Chat Controller

Create a REST controller to handle streaming chat requests. It uses Server-Sent Events (SSE) to send the AI's response tokens in real-time.

@RestController
@CrossOrigin("*")
public class ChatStreamController {

    private final ChatClient chatClient;

    public ChatStreamController(ChatClient chatClient) {
        this.chatClient = chatClient;
    }

    @PostMapping(value = "/api/chat/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
    public Flux<ServerSentEvent<String>> handleStreamingChat(@RequestBody UserQuery query) {
        return chatClient.prompt(query.userMessage())
                .stream()
                .content()
                .map(content -> ServerSentEvent.builder(content).event("token").build())
                .concatWithValues(ServerSentEvent.builder("[STREAM_END]").build())
                .onErrorResume(ex -> Flux.just(
                    ServerSentEvent.builder("Error: " + ex.getMessage()).event("error").build()
                ));
    }

    public record UserQuery(String userId, String userMessage) {}
}

Adding Conversation Memory

To enable multi-turn conversations, Spring AI provides the MessageChatMemoryAdvisor. First, define an in-memory chat memory bean.

@Configuration
public class MemoryConfig {

    @Bean
    public InMemoryChatMemory conversationMemory() {
        return new InMemoryChatMemory();
    }

    @Bean
    public ChatClient contextualAgent(ChatClient.Builder builder, InMemoryChatMemory memory) {
        return builder
                .defaultSystem("You are an intelligent assistant named Spring AI Bot.")
                .build();
    }
}

Modify the controller to include the memory advisor, which appends recent conversation history to each new prompt.

@PostMapping(value = "/api/chat/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<ServerSentEvent<String>> handleStreamingChat(@RequestBody UserQuery query) {
    String sessionId = query.userId();

    return chatClient.prompt(query.userMessage())
            .advisors(new MessageChatMemoryAdvisor(conversationMemory, sessionId, 10))
            .stream()
            .content()
            .map(content -> ServerSentEvent.builder(content).event("token").build())
            .concatWithValues(ServerSentEvent.builder("[STREAM_END]").build())
            .onErrorResume(ex -> Flux.just(
                ServerSentEvent.builder("Error: " + ex.getMessage()).event("error").build()
            ));
}

Frontend Client with Vue.js

The frontend is built with Vue 3 and TypeScript. It manages the chat interface and connects to the SSE endpoint.

Core Chat Logic

The Vue component handles sending messages, processing the streamed response, and managing conversation state.

<script setup lang="ts">
import { ref, reactive, onMounted, nextTick } from 'vue'
import { fetchEventSource } from '@microsoft/fetch-event-source'

interface ChatMessage {
  id: string;
  text: string;
  isFromBot: boolean;
  isStreaming?: boolean;
}

const messages = ref<ChatMessage[]>([]);
const userInput = ref('');
const isProcessing = ref(false);
let streamController: AbortController | null = null;

const sendMessage = async () => {
  if (!userInput.value.trim() || isProcessing.value) return;

  const userText = userInput.value.trim();
  userInput.value = '';

  // Add user message
  messages.value.push({
    id: `usr-${Date.now()}`,
    text: userText,
    isFromBot: false
  });

  // Add placeholder for bot's streaming response
  const botMessage: ChatMessage = reactive({
    id: `bot-${Date.now()}`,
    text: '',
    isFromBot: true,
    isStreaming: true
  });
  messages.value.push(botMessage);

  isProcessing.value = true;
  streamController = new AbortController();

  try {
    await fetchEventSource('http://localhost:8080/api/chat/stream', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        userId: 'demo-user',
        userMessage: userText
      }),
      signal: streamController.signal,
      onopen(response) {
        if (!response.ok) {
          throw new Error(`HTTP ${response.status}`);
        }
      },
      onmessage(event) {
        if (event.data === '[STREAM_END]') {
          botMessage.isStreaming = false;
          return;
        }
        if (event.event === 'token') {
          botMessage.text += event.data;
        }
      },
      onclose() {
        isProcessing.value = false;
      },
      onerror(err) {
        botMessage.text = `Error: ${err.message}`;
        botMessage.isStreaming = false;
        isProcessing.value = false;
        throw err;
      }
    });
  } catch (err) {
    console.error('Stream error:', err);
    isProcessing.value = false;
  }
};

const stopGeneration = () => {
  if (streamController) {
    streamController.abort();
    isProcessing.value = false;
    const lastMsg = messages.value[messages.value.length - 1];
    if (lastMsg.isFromBot && lastMsg.isStreaming) {
      lastMsg.isStreaming = false;
    }
  }
};

onMounted(() => {
  // Focus the input field on load
  const inputEl = document.querySelector('input');
  inputEl?.focus();
});
</script>

<template>
  <div class="chat-container">
    <div class="message-list">
      <div v-for="msg in messages" :key="msg.id" class="message" :class="{ bot: msg.isFromBot }">
        {{ msg.text }}
        <span v-if="msg.isStreaming" class="streaming-cursor">▌</span>
      </div>
    </div>
    <div class="input-area">
      <input v-model="userInput" @keyup.enter="sendMessage" :disabled="isProcessing" placeholder="Type a message..." />
      <button @click="sendMessage" :disabled="isProcessing">Send</button>
      <button v-if="isProcessing" @click="stopGeneration">Stop</button>
    </div>
  </div>
</template>