1. 概述
搜索是软件领域的基础概念,核心目标是在海量数据中快速定位相关信息。本质是在数据集合中查找特定项的过程。
本教程将演示如何结合Spring AI、PGVector和Ollama构建语义搜索系统。
2. 背景知识
语义搜索是一种基于语义理解的高级搜索技术,通过分析词语含义返回最相关结果。构建语义搜索应用需要掌握以下核心概念:
- 词嵌入(Word Embeddings):将词语转换为数值向量的表示方法,使语义相近的词语在向量空间中距离更近。这种表示形式可直接用于机器学习模型。
- 语义相似度(Semantic Similarity):衡量两段文本在语义层面的接近程度,适用于词语、句子或文档间的比较。
- 向量空间模型(Vector Space Model):将文本表示为高维空间向量的数学模型。每个词语对应一个向量,相似度通过向量间距离计算。
- 余弦相似度(Cosine Similarity):计算两个非零向量夹角余弦值的相似度度量方法,常用于向量空间模型中的相似性评估。
接下来我们动手实现一个语义搜索应用。
3. 准备工作
首先需安装Docker来运行PGVector和Ollama服务。
然后在Spring应用中添加以下依赖:
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-ollama-spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-pgvector-store-spring-boot-starter</artifactId>
</dependency>
同时添加Spring Boot Docker Compose支持来管理容器:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-docker-compose</artifactId>
<version>3.1.1</version>
</dependency>
创建docker-compose.yml文件定义服务:
services:
postgres:
image: pgvector/pgvector:pg17
environment:
POSTGRES_DB: vectordb
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
ports:
- "5434:5432"
healthcheck:
test: [ "CMD-SHELL", "pg_isready -U postgres" ]
interval: 10s
timeout: 5s
retries: 5
ollama:
image: ollama/ollama:latest
ports:
- "11435:11434"
volumes:
- ollama_data:/root/.ollama
healthcheck:
test: [ "CMD", "curl", "-f", "http://localhost:11435/api/health" ]
interval: 10s
timeout: 5s
retries: 10
volumes:
ollama_data:
4. 应用配置
在application.yml中配置Spring Boot应用:
spring:
ai:
ollama:
init:
pull-model-strategy: when_missing
chat:
include: true
embedding:
options:
model: nomic-embed-text
vectorstore:
pgvector:
initialize-schema: true
dimensions: 768
index-type: hnsw
docker:
compose:
file: docker-compose.yml
enabled: true
datasource:
url: jdbc:postgresql://localhost:5434/vectordb
username: postgres
password: postgres
driver-class-name: org.postgresql.Driver
jpa:
database-platform: org.hibernate.dialect.PostgreSQLDialect
✅ 关键配置说明:
- 使用nomic-embed-text作为Ollama嵌入模型(缺失时自动下载)
- PGVector配置要点:
initialize-schema: true
自动初始化数据库结构dimensions: 768
匹配常见嵌入维度index-type: hnsw
使用HNSW索引加速近似最近邻搜索
5. 实现语义搜索
基础设施就绪后,我们构建一个智能图书搜索引擎,支持基于内容语义的图书检索。
先定义图书实体类:
public record Book(String title, String author, String description) {
}
5.1 数据准备
向PGVector存储样本图书数据:
void run() {
var books = List.of(
new Book("The Great Gatsby", "F. Scott Fitzgerald", "The Great Gatsby is a 1925 novel by American writer F. Scott Fitzgerald. Set in the Jazz Age on Long Island, near New York City, the novel depicts first-person narrator Nick Carraway's interactions with mysterious millionaire Jay Gatsby and Gatsby's obsession to reunite with his former lover, Daisy Buchanan."),
new Book("To Kill a Mockingbird", "Harper Lee", "To Kill a Mockingbird is a novel by the American author Harper Lee. It was published in 1960 and was instantly successful. In the United States, it is widely read in high schools and middle schools."),
new Book("1984", "George Orwell", "Nineteen Eighty-Four: A Novel, often referred to as 1984, is a dystopian social science fiction novel by the English novelist George Orwell. It was published on 8 June 1949 by Secker & Warburg as Orwell's ninth and final book completed in his lifetime."),
new Book("The Catcher in the Rye", "J. D. Salinger", "The Catcher in the Rye is a novel by J. D. Salinger, partially published in serial form in 1945–1946 and as a novel in 1951. It was originally intended for adults but is often read by adolescents for its themes of angst, alienation, and as a critique on superficiality in society."),
new Book("Lord of the Flies", "William Golding", "Lord of the Flies is a 1954 novel by Nobel Prize-winning British author William Golding. The book focuses on a group of British")
);
List<Document> documents = books.stream()
.map(book -> new Document(book.toString()))
.toList();
vectorStore.add(documents);
}
5.2 基础语义搜索
创建控制器实现搜索接口:
@RequestMapping("/books")
class BookSearchController {
final VectorStore vectorStore;
final ChatClient chatClient;
BookSearchController(VectorStore vectorStore, ChatClient.Builder chatClientBuilder) {
this.vectorStore = vectorStore;
this.chatClient = chatClientBuilder.build();
}
// ...
}
添加语义搜索接口:
@PostMapping("/search")
List<String> semanticSearch(@RequestBody String query) {
return vectorStore.similaritySearch(SearchRequest.builder()
.query(query)
.topK(3)
.build())
.stream()
.map(Document::getText)
.toList();
}
⚠️ 核心逻辑:使用VectorStore#similaritySearch
执行语义相似度搜索,返回Top 3结果。
测试搜索功能:
curl -X POST --data "1984" http://localhost:8080/books/search
响应包含精确匹配和部分匹配结果:
[
"Book[title=1984, author=George Orwell, description=Nineteen Eighty-Four: A Novel, often referred to as 1984, is a dystopian social science fiction novel by the English novelist George Orwell.]",
"Book[title=The Catcher in the Rye, author=J. D. Salinger, description=The Catcher in the Rye is a novel by J. D. Salinger, partially published in serial form in 1945–1946 and as a novel in 1951.]",
"Book[title=To Kill a Mockingbird, author=Harper Lee, description=To Kill a Mockingbird is a novel by the American author Harper Lee.]"
]
5.3 增强语义搜索
集成Ollama优化搜索结果,处理流程:
- 从向量库获取Top 3匹配结果
- 将结果输入Ollama生成上下文感知的响应
- 返回经过总结和改写的自然语言回答
添加增强搜索接口:
@PostMapping("/enhanced-search")
String enhancedSearch(@RequestBody String query) {
String context = vectorStore.similaritySearch(SearchRequest.builder()
.query(query)
.topK(3)
.build())
.stream()
.map(Document::getText)
.reduce("", (a, b) -> a + b + "\n");
return chatClient.prompt()
.system(context)
.user(query)
.call()
.content();
}
测试增强搜索:
curl -X POST --data "1984" http://localhost:8080/books/enhanced-search
✅ 与基础搜索对比:
- 基础搜索:返回原始图书描述列表
- 增强搜索:Ollama智能整合信息,聚焦最相关结果(如"1984"的详细摘要),提供更自然的类人交互体验
6. 总结
本文演示了如何使用Spring AI、PGVector和Ollama构建语义搜索系统,实现了两种搜索模式:
- 基础语义搜索:直接返回向量库相似度匹配结果
- 增强语义搜索:结合LLM对结果进行智能优化和上下文增强
这种组合方案在保持搜索准确性的同时,显著提升了结果的可读性和实用性。