Skip to content

Milvus 单机部署指南 ​

本指南将介绍如何在单机环境中快速部署 Milvus 向量数据库,适合开发测试、学习和小规模应用场景。

概述 ​

单机部署特点 ​

  • 快速部署:几分钟内完成安装
  • 资源占用少:适合开发测试环境
  • 易于维护:配置简单,易于排查问题
  • 功能完整:支持所有核心功能

适用场景 ​

  • 开发和测试环境
  • 学习和实验
  • 小规模应用(百万级向量)
  • 原型验证

环境要求 ​

硬件要求 ​

最低配置 ​

  • CPU: 4核心
  • 内存: 8GB
  • 存储: 50GB SSD
  • 网络: 100Mbps

推荐配置 ​

  • CPU: 8核心
  • 内存: 16GB
  • 存储: 200GB SSD
  • 网络: 1Gbps

软件要求 ​

  • 操作系统: Linux (Ubuntu 18.04+) / macOS / Windows
  • Docker: 19.03+
  • Docker Compose: 1.25+(可选)

部署方式 ​

方式一:Docker 部署(推荐) ​

1. 拉取镜像 ​

bash
docker pull milvusdb/milvus:v2.3.3

2. 启动 Milvus ​

bash
docker run -d \
  --name milvus-standalone \
  -p 19530:19530 \
  -p 9091:9091 \
  -v $(pwd)/milvus:/var/lib/milvus \
  milvusdb/milvus:v2.3.3 \
  milvus run standalone

3. 验证运行状态 ​

bash
docker ps | grep milvus
docker logs milvus-standalone

方式二:Docker Compose 部署 ​

1. 创建配置文件 ​

创建 docker-compose.yml:

yaml
version: '3.5'

services:
  etcd:
    container_name: milvus-etcd
    image: quay.io/coreos/etcd:v3.5.5
    environment:
      - ETCD_AUTO_COMPACTION_MODE=revision
      - ETCD_AUTO_COMPACTION_RETENTION=1000
      - ETCD_QUOTA_BACKEND_BYTES=4294967296
      - ETCD_SNAPSHOT_COUNT=50000
    volumes:
      - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/etcd:/etcd
    command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd
    healthcheck:
      test: ["CMD", "etcdctl", "endpoint", "health"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio:
    container_name: milvus-minio
    image: minio/minio:RELEASE.2023-03-20T20-16-18Z
    environment:
      MINIO_ACCESS_KEY: minioadmin
      MINIO_SECRET_KEY: minioadmin
    ports:
      - "9001:9001"
      - "9000:9000"
    volumes:
      - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/minio:/minio_data
    command: minio server /minio_data
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  standalone:
    container_name: milvus-standalone
    image: milvusdb/milvus:v2.3.3
    command: ["milvus", "run", "standalone"]
    security_opt:
    - seccomp:unconfined
    environment:
      ETCD_ENDPOINTS: etcd:2379
      MINIO_ADDRESS: minio:9000
    volumes:
      - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/milvus:/var/lib/milvus
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9091/healthz"]
      interval: 30s
      start_period: 90s
      timeout: 20s
      retries: 3
    ports:
      - "19530:19530"
      - "9091:9091"
    depends_on:
      - "etcd"
      - "minio"

networks:
  default:
    name: milvus

2. 启动服务 ​

bash
docker-compose up -d

3. 查看状态 ​

bash
docker-compose ps
docker-compose logs -f standalone

4. 停止服务 ​

bash
docker-compose down

方式三:二进制部署 ​

1. 下载安装包 ​

bash
wget https://github.com/milvus-io/milvus/releases/download/v2.3.3/milvus-standalone-linux.tar.gz
tar -xzf milvus-standalone-linux.tar.gz

2. 安装依赖 ​

bash
sudo apt-get update
sudo apt-get install -y etcd-server minio

3. 启动 etcd ​

bash
etcd --listen-client-urls=http://0.0.0.0:2379 \
     --advertise-client-urls=http://localhost:2379 &

4. 启动 MinIO ​

bash
minio server /data --address ":9000" &

5. 启动 Milvus ​

bash
export ETCD_ENDPOINTS=localhost:2379
export MINIO_ADDRESS=localhost:9000
./milvus run standalone

基础使用 ​

安装 Python SDK ​

bash
pip install pymilvus

连接 Milvus ​

python
from pymilvus import connections, utility

connections.connect(
    alias="default",
    host='localhost',
    port='19530'
)

print(f"Milvus version: {utility.get_server_version()}")

创建集合 ​

python
from pymilvus import Collection, FieldSchema, CollectionSchema, DataType

fields = [
    FieldSchema(name="id", dtype=DataType.INT64, is_primary=True, auto_id=False),
    FieldSchema(name="embedding", dtype=DataType.FLOAT_VECTOR, dim=128)
]

schema = CollectionSchema(fields=fields, description="示例集合")

collection = Collection(
    name="example_collection",
    schema=schema
)

print(f"集合 {collection.name} 创建成功")

插入数据 ​

python
import numpy as np

num_entities = 10000

entities = [
    list(range(num_entities)),
    [[np.random.random() for _ in range(128)] for _ in range(num_entities)]
]

insert_result = collection.insert(entities)

print(f"插入了 {collection.num_entities} 条数据")

创建索引 ​

python
index_params = {
    "metric_type": "L2",
    "index_type": "IVF_FLAT",
    "params": {"nlist": 128}
}

collection.create_index(
    field_name="embedding",
    index_params=index_params
)

print("索引创建成功")

加载集合 ​

python
collection.load()

向量检索 ​

python
search_params = {
    "metric_type": "L2",
    "params": {"nprobe": 10}
}

search_vector = [[np.random.random() for _ in range(128)]]

results = collection.search(
    data=search_vector,
    anns_field="embedding",
    param=search_params,
    limit=10,
    expr=None
)

for hits in results:
    for hit in hits:
        print(f"ID: {hit.id}, Distance: {hit.distance}")

配置优化 ​

内存配置 ​

创建配置文件 milvus.yaml:

yaml
standalone:
  memory:
    limit: 8GB

queryNode:
  segcore:
    chunkRows: 1024
  cache:
    enabled: true
    memoryLimit: 4GB

dataNode:
  flush:
    insertBufferSize: 16MB
    insertBufferPoolSize: 1GB

启动时指定配置文件:

bash
docker run -d \
  --name milvus-standalone \
  -p 19530:19530 \
  -v $(pwd)/milvus.yaml:/milvus/configs/milvus.yaml \
  -v $(pwd)/milvus:/var/lib/milvus \
  milvusdb/milvus:v2.3.3 \
  milvus run standalone

存储配置 ​

使用外部存储 ​

yaml
minio:
  address: your-minio-server.com
  port: 9000
  accessKeyID: your-access-key
  secretAccessKey: your-secret-key
  useSSL: true
  bucketName: milvus-data

数据压缩 ​

yaml
dataCoord:
  enableCompaction: true
  compaction:
    enableAutoCompaction: true

性能调优 ​

索引优化 ​

python
index_params = {
    "metric_type": "L2",
    "index_type": "HNSW",
    "params": {
        "M": 16,
        "efConstruction": 256
    }
}

collection.create_index(
    field_name="embedding",
    index_params=index_params
)

查询优化 ​

python
search_params = {
    "metric_type": "L2",
    "params": {
        "ef": 64
    }
}

results = collection.search(
    data=search_vectors,
    anns_field="embedding",
    param=search_params,
    limit=100
)

数据管理 ​

数据导入 ​

从文件导入 ​

python
import pandas as pd

df = pd.read_csv('vectors.csv')

entities = [
    df['id'].tolist(),
    df['embedding'].apply(eval).tolist()
]

collection.insert(entities)

批量导入 ​

python
import numpy as np

batch_size = 10000
total_records = 100000

for i in range(0, total_records, batch_size):
    batch = [
        list(range(i, i + batch_size)),
        np.random.random((batch_size, 128)).tolist()
    ]
    collection.insert(batch)
    print(f"已插入 {i + batch_size} 条记录")

数据导出 ​

python
results = collection.query(
    expr="id >= 0",
    output_fields=["id", "embedding"]
)

import json
with open('export.json', 'w') as f:
    json.dump(results, f)

数据备份 ​

使用 Milvus Backup ​

bash
docker run --rm \
  -v $(pwd)/backup:/backup \
  -e MILVUS_ADDRESS=localhost:19530 \
  milvusdb/milvus-backup:latest \
  milvus-backup create -n backup-$(date +%Y%m%d)

监控与日志 ​

健康检查 ​

bash
curl http://localhost:9091/healthz

性能指标 ​

访问 Prometheus 指标:

bash
curl http://localhost:9091/metrics

日志查看 ​

bash
docker logs milvus-standalone -f

配置日志级别 ​

yaml
log:
  level: debug
  file:
    rootPath: /var/lib/milvus/logs
    maxSize: 300
    maxAge: 10
    maxBackups: 20

升级迁移 ​

版本升级 ​

1. 备份数据 ​

bash
docker exec milvus-standalone \
  milvus-backup create -n backup-before-upgrade

2. 停止服务 ​

bash
docker stop milvus-standalone
docker rm milvus-standalone

3. 拉取新版本 ​

bash
docker pull milvusdb/milvus:v2.3.4

4. 启动新版本 ​

bash
docker run -d \
  --name milvus-standalone \
  -p 19530:19530 \
  -v $(pwd)/milvus:/var/lib/milvus \
  milvusdb/milvus:v2.3.4 \
  milvus run standalone

集群迁移 ​

1. 导出数据 ​

python
from pymilvus import connections, Collection

connections.connect(alias="source", host='source-host', port='19530')
collection = Collection("example_collection")

results = collection.query(
    expr="",
    output_fields=["*"],
    limit=10000
)

2. 导入到新环境 ​

python
connections.connect(alias="target", host='target-host', port='19530')

target_collection = Collection("example_collection")
target_collection.insert(results)

常见问题 ​

1. 容器启动失败 ​

问题: Docker 容器无法启动

解决方案:

bash
docker logs milvus-standalone

docker run --rm milvusdb/milvus:v2.3.3 milvus run standalone --help

2. 连接超时 ​

问题: 无法连接到 Milvus

解决方案:

bash
docker exec -it milvus-standalone bash
curl localhost:19530/v1/vector/collections

netstat -tlnp | grep 19530

3. 内存不足 ​

问题: OOM (Out of Memory) 错误

解决方案:

yaml
standalone:
  memory:
    limit: 16GB

4. 性能问题 ​

问题: 查询速度慢

解决方案:

  • 优化索引参数
  • 增加内存配置
  • 使用更快的存储

最佳实践 ​

1. 开发环境 ​

  • 使用 Docker 部署
  • 配置合适的资源限制
  • 定期清理测试数据

2. 数据管理 ​

  • 合理设计集合结构
  • 选择合适的索引类型
  • 定期备份数据

3. 性能优化 ​

  • 使用批量操作
  • 优化索引参数
  • 监控系统资源

4. 安全建议 ​

  • 不要暴露到公网
  • 配置访问控制
  • 定期更新版本

参考资料 ​

官方文档 ​

相关教程 ​


提示

单机部署适合开发测试,生产环境建议使用集群部署以获得更好的性能和可靠性。

信息

本指南基于 Milvus 2.3 版本编写,如需其他版本,请参考官方文档。