DAMO-YOLO手机检测镜像CI流水线：GitHub Actions自动化测试与部署

DAMO-YOLO手机检测镜像CI流水线GitHub Actions自动化测试与部署1. 项目简介与核心价值你有没有遇到过这样的场景开发了一个AI应用每次更新代码后都要手动登录服务器、拉取代码、安装依赖、重启服务。整个过程繁琐耗时还容易出错。特别是当团队协作时不同成员的开发环境差异、代码合并冲突、部署遗漏等问题更是让人头疼。今天我要分享的就是如何为DAMO-YOLO手机检测镜像搭建一套完整的CI/CD流水线。通过GitHub Actions实现代码推送后自动测试、自动构建、自动部署的全流程自动化。这套方案不仅适用于手机检测项目任何基于Python的AI应用都可以参考。1.1 为什么需要自动化流水线在介绍具体实现之前我们先看看传统部署方式的问题手动部署的痛点效率低下每次更新都要重复执行相同的部署步骤容易出错人工操作难免遗漏步骤或配置错误环境不一致开发、测试、生产环境差异导致“在我机器上能跑”的问题缺乏记录部署过程没有日志出问题难以追溯协作困难多人开发时代码质量参差不齐自动化流水线的优势一键部署代码推送后自动完成所有流程质量保证自动运行测试确保代码质量环境一致每次都在干净的环境中构建完整日志每个步骤都有详细记录团队协作统一部署标准减少沟通成本1.2 DAMO-YOLO手机检测项目回顾我们的手机检测系统基于阿里巴巴达摩院的DAMO-YOLO模型具有“小、快、省”的特点模型小巧仅125MB适合移动端和边缘设备推理快速单张图片约3.83msT4 GPU准确率高88.8%的AP0.5准确率部署简单基于Gradio的Web界面开箱即用项目结构如下phone-detection/ ├── app.py # 主程序 ├── requirements.txt # 依赖列表 ├── tests/ # 测试目录 ├── .github/workflows/ # GitHub Actions配置 └── deployment/ # 部署脚本2. GitHub Actions基础配置2.1 什么是GitHub ActionsGitHub Actions是GitHub提供的持续集成和持续部署CI/CD平台。它允许你在代码仓库中创建自动化工作流响应各种事件如代码推送、Pull Request等。核心概念Workflow工作流自动化的完整流程存储在.github/workflows/目录Job作业工作流中的任务单元可以并行或顺序执行Step步骤作业中的具体操作步骤Action动作可重用的代码单元如检查代码、运行测试等2.2 创建基础工作流文件首先在项目根目录创建.github/workflows/ci-cd.yml文件name: CI/CD Pipeline on: push: branches: [ main, develop ] pull_request: branches: [ main ] jobs: test: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkoutv3 - name: Set up Python uses: actions/setup-pythonv4 with: python-version: 3.11 - name: Install dependencies run: | python -m pip install --upgrade pip pip install -r requirements.txt pip install pytest pytest-cov - name: Run tests run: | pytest tests/ --covapp --cov-reportxml - name: Upload coverage uses: codecov/codecov-actionv3 with: file: ./coverage.xml这个基础配置实现了在代码推送到main或develop分支时触发在Pull Request时也触发测试使用Ubuntu最新版作为运行环境安装Python 3.11和项目依赖运行测试并生成覆盖率报告上传覆盖率到Codecov3. 完整的CI/CD流水线设计3.1 多阶段流水线架构一个完整的CI/CD流水线应该包含多个阶段每个阶段有明确的目标name: DAMO-YOLO CI/CD Pipeline on: push: branches: [ main ] pull_request: branches: [ main ] jobs: # 第一阶段代码质量检查 lint-and-test: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkoutv3 - name: Set up Python uses: actions/setup-pythonv4 with: python-version: 3.11 - name: Install dependencies run: | python -m pip install --upgrade pip pip install -r requirements-dev.txt - name: Code linting run: | pip install black flake8 black --check . flake8 . - name: Run unit tests run: | pip install pytest pytest-cov pytest tests/unit/ -v --covapp --cov-reportxml - name: Run integration tests run: | pytest tests/integration/ -v # 第二阶段构建和推送Docker镜像 build-and-push: needs: lint-and-test runs-on: ubuntu-latest if: github.event_name push github.ref refs/heads/main steps: - name: Checkout code uses: actions/checkoutv3 - name: Set up Docker Buildx uses: docker/setup-buildx-actionv2 - name: Log in to Docker Hub uses: docker/login-actionv2 with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} - name: Build and push uses: docker/build-push-actionv4 with: context: . push: true tags: | yourusername/damo-yolo-phone-detection:latest yourusername/damo-yolo-phone-detection:${{ github.sha }} # 第三阶段自动部署到服务器 deploy: needs: build-and-push runs-on: ubuntu-latest steps: - name: Deploy to production uses: appleboy/ssh-actionv0.1.5 with: host: ${{ secrets.SERVER_HOST }} username: ${{ secrets.SERVER_USERNAME }} key: ${{ secrets.SERVER_SSH_KEY }} script: | cd /root/phone-detection git pull origin main docker-compose down docker-compose pull docker-compose up -d echo Deployment completed at $(date)3.2 测试策略设计测试是CI/CD流水线的核心好的测试策略能确保代码质量tests/目录结构tests/ ├── unit/ # 单元测试 │ ├── test_models.py # 模型相关测试 │ ├── test_utils.py # 工具函数测试 │ └── test_validators.py # 数据验证测试 ├── integration/ # 集成测试 │ ├── test_api.py # API接口测试 │ └── test_detection.py # 检测功能测试 ├── conftest.py # pytest配置 └── test_data/ # 测试数据 ├── test_images/ # 测试图片 └── expected_results/ # 预期结果单元测试示例test_models.pyimport pytest import torch from app.models.detector import PhoneDetector class TestPhoneDetector: pytest.fixture def detector(self): 创建检测器实例 return PhoneDetector() def test_model_loading(self, detector): 测试模型加载 assert detector.model is not None assert detector.device torch.device(cuda if torch.cuda.is_available() else cpu) def test_preprocess(self, detector): 测试图像预处理 import numpy as np test_image np.random.randint(0, 255, (480, 640, 3), dtypenp.uint8) processed detector.preprocess(test_image) assert processed.shape (1, 3, 640, 640) assert processed.dtype torch.float32 def test_detection_output_format(self, detector): 测试检测输出格式 import numpy as np test_image np.random.randint(0, 255, (480, 640, 3), dtypenp.uint8) results detector.detect(test_image) assert isinstance(results, list) if len(results) 0: for result in results: assert bbox in result assert confidence in result assert class_name in result assert result[class_name] phone集成测试示例test_detection.pyimport pytest from app import create_app import json class TestDetectionAPI: pytest.fixture def client(self): 创建测试客户端 app create_app() app.config[TESTING] True with app.test_client() as client: yield client def test_health_check(self, client): 测试健康检查接口 response client.get(/health) assert response.status_code 200 data json.loads(response.data) assert data[status] healthy def test_detection_endpoint(self, client): 测试检测接口 # 准备测试图片 import io from PIL import Image import numpy as np # 创建测试图片 test_image Image.fromarray(np.random.randint(0, 255, (480, 640, 3), dtypenp.uint8)) img_byte_arr io.BytesIO() test_image.save(img_byte_arr, formatPNG) img_byte_arr.seek(0) # 发送请求 data { image: (img_byte_arr, test.png) } response client.post(/detect, datadata, content_typemultipart/form-data) assert response.status_code 200 data json.loads(response.data) assert detections in data assert processing_time in data4. 高级功能与优化4.1 矩阵测试多环境测试为了确保代码在不同环境下的兼容性我们可以使用矩阵测试test-matrix: runs-on: ${{ matrix.os }} strategy: matrix: os: [ubuntu-latest, windows-latest] python-version: [3.9, 3.10, 3.11] include: - os: ubuntu-latest python-version: 3.11 test-type: full - os: windows-latest python-version: 3.10 test-type: basic steps: - name: Checkout code uses: actions/checkoutv3 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-pythonv4 with: python-version: ${{ matrix.python-version }} - name: Install dependencies run: | python -m pip install --upgrade pip pip install -r requirements.txt if [ ${{ matrix.test-type }} full ]; then pip install -r requirements-dev.txt fi - name: Run tests run: | if [ ${{ matrix.test-type }} full ]; then pytest tests/ -v --covapp --cov-reportxml else pytest tests/unit/ -v fi4.2 缓存优化依赖安装通常是CI中最耗时的步骤使用缓存可以大幅提升速度- name: Cache pip packages uses: actions/cachev3 with: path: ~/.cache/pip key: ${{ runner.os }}-pip-${{ hashFiles(requirements.txt) }} restore-keys: | ${{ runner.os }}-pip- - name: Cache Docker layers uses: actions/cachev3 with: path: /tmp/.buildx-cache key: ${{ runner.os }}-buildx-${{ github.sha }} restore-keys: | ${{ runner.os }}-buildx-4.3 安全扫描与代码质量集成安全扫描工具确保代码安全性security-scan: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkoutv3 - name: Run Bandit (安全漏洞扫描) run: | pip install bandit bandit -r app/ -f json -o bandit-report.json || true - name: Run Safety (依赖漏洞检查) run: | pip install safety safety check -r requirements.txt --json safety-report.json || true - name: Upload security reports uses: actions/upload-artifactv3 with: name: security-reports path: | bandit-report.json safety-report.json5. 部署策略与监控5.1 Docker化部署创建Dockerfile和docker-compose.yml文件实现容器化部署Dockerfile:FROM python:3.11-slim WORKDIR /app # 安装系统依赖 RUN apt-get update apt-get install -y \ libgl1-mesa-glx \ libglib2.0-0 \ rm -rf /var/lib/apt/lists/* # 复制依赖文件 COPY requirements.txt . # 安装Python依赖 RUN pip install --no-cache-dir -r requirements.txt # 复制应用代码 COPY . . # 创建非root用户 RUN useradd -m -u 1000 appuser chown -R appuser:appuser /app USER appuser # 暴露端口 EXPOSE 7860 # 启动命令 CMD [python, app.py]docker-compose.yml:version: 3.8 services: phone-detection: build: . image: yourusername/damo-yolo-phone-detection:latest container_name: phone-detection ports: - 7860:7860 environment: - PYTHONUNBUFFERED1 - GRADIO_SERVER_NAME0.0.0.0 volumes: - ./logs:/app/logs - ./models:/app/models restart: unless-stopped healthcheck: test: [CMD, curl, -f, http://localhost:7860/health] interval: 30s timeout: 10s retries: 35.2 部署脚本创建自动化部署脚本deployment/deploy.sh:#!/bin/bash set -e # 遇到错误立即退出 echo 开始部署 DAMO-YOLO 手机检测系统... # 检查Docker是否安装 if ! command -v docker /dev/null; then echo ❌ Docker未安装请先安装Docker exit 1 fi # 检查docker-compose是否安装 if ! command -v docker-compose /dev/null; then echo ❌ docker-compose未安装请先安装docker-compose exit 1 fi # 拉取最新镜像 echo 拉取最新镜像... docker-compose pull # 停止旧容器 echo 停止旧容器... docker-compose down # 启动新容器 echo 启动新容器... docker-compose up -d # 等待服务启动 echo ⏳ 等待服务启动... sleep 10 # 检查服务状态 if curl -f http://localhost:7860/health /dev/null 21; then echo ✅ 部署成功服务已启动 echo 访问地址: http://localhost:7860 else echo ❌ 服务启动失败请检查日志 docker-compose logs exit 1 fi # 清理旧镜像 echo 清理未使用的镜像... docker image prune -f echo 部署完成5.3 监控与告警集成监控系统实时了解服务状态monitoring: runs-on: ubuntu-latest if: always() # 无论成功失败都运行 steps: - name: Check service health run: | # 测试服务健康状态 response$(curl -s -o /dev/null -w %{http_code} http://${{ secrets.SERVER_HOST }}:7860/health) if [ $response -eq 200 ]; then echo ✅ 服务运行正常 else echo ❌ 服务异常HTTP状态码: $response # 发送告警通知 curl -X POST https://api.alert-service.com/notify \ -H Content-Type: application/json \ -d { service: phone-detection, status: down, timestamp: $(date -Iseconds) } fi - name: Performance monitoring run: | # 测试接口性能 start_time$(date %s%3N) curl -s http://${{ secrets.SERVER_HOST }}:7860/health /dev/null end_time$(date %s%3N) response_time$((end_time - start_time)) echo ⏱️ 接口响应时间: ${response_time}ms if [ $response_time -gt 1000 ]; then echo ⚠️ 响应时间超过1秒可能需要优化 fi6. 实战案例完整的CI/CD流水线6.1 完整的工作流配置下面是一个完整的、生产可用的CI/CD流水线配置name: Production CI/CD Pipeline on: push: branches: [ main ] pull_request: branches: [ main, develop ] schedule: - cron: 0 2 * * * # 每天凌晨2点运行 env: REGISTRY: docker.io IMAGE_NAME: ${{ secrets.DOCKER_USERNAME }}/damo-yolo-phone-detection jobs: # 代码质量检查 quality-check: name: Code Quality Check runs-on: ubuntu-latest steps: - uses: actions/checkoutv3 - name: Set up Python uses: actions/setup-pythonv4 with: python-version: 3.11 - name: Cache dependencies uses: actions/cachev3 with: path: ~/.cache/pip key: ${{ runner.os }}-pip-${{ hashFiles(requirements*.txt) }} - name: Install dependencies run: | python -m pip install --upgrade pip pip install -r requirements-dev.txt - name: Code formatting check run: | black --check --diff . - name: Linting run: | flake8 . --count --selectE9,F63,F7,F82 --show-source --statistics flake8 . --count --exit-zero --max-complexity10 --max-line-length127 --statistics - name: Type checking run: | pip install mypy mypy app/ --ignore-missing-imports - name: Security scan run: | pip install bandit safety bandit -r app/ -ll safety check -r requirements.txt # 测试套件 test-suite: name: Test Suite runs-on: ubuntu-latest needs: quality-check strategy: matrix: python-version: [3.9, 3.10, 3.11] steps: - uses: actions/checkoutv3 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-pythonv4 with: python-version: ${{ matrix.python-version }} - name: Install dependencies run: | python -m pip install --upgrade pip pip install -r requirements.txt pip install pytest pytest-cov pytest-xdist - name: Run unit tests run: | pytest tests/unit/ -v --covapp --cov-reportxml --junitxmlunit-test-results.xml - name: Run integration tests run: | pytest tests/integration/ -v --junitxmlintegration-test-results.xml - name: Upload test results uses: actions/upload-artifactv3 with: name: test-results-${{ matrix.python-version }} path: | unit-test-results.xml integration-test-results.xml coverage.xml # 构建和推送Docker镜像 build-and-push: name: Build and Push Docker Image runs-on: ubuntu-latest needs: test-suite if: github.event_name push github.ref refs/heads/main steps: - uses: actions/checkoutv3 - name: Set up Docker Buildx uses: docker/setup-buildx-actionv2 - name: Log in to Docker Hub uses: docker/login-actionv2 with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} - name: Extract metadata id: meta uses: docker/metadata-actionv4 with: images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} tags: | typesha,prefix{{branch}}- typeref,eventbranch typeref,eventpr typesemver,pattern{{version}} typesemver,pattern{{major}}.{{minor}} - name: Build and push uses: docker/build-push-actionv4 with: context: . push: true tags: ${{ steps.meta.outputs.tags }} labels: ${{ steps.meta.outputs.labels }} cache-from: typegha cache-to: typegha,modemax # 部署到生产环境 deploy-production: name: Deploy to Production runs-on: ubuntu-latest needs: build-and-push if: github.event_name push github.ref refs/heads/main steps: - name: Deploy via SSH uses: appleboy/ssh-actionv0.1.5 with: host: ${{ secrets.PRODUCTION_HOST }} username: ${{ secrets.PRODUCTION_USER }} key: ${{ secrets.PRODUCTION_SSH_KEY }} script: | set -e echo 开始部署到生产环境... # 进入项目目录 cd /opt/phone-detection # 拉取最新代码 git fetch origin git reset --hard origin/main # 更新镜像 docker-compose pull # 重启服务 docker-compose down docker-compose up -d # 等待服务启动 sleep 15 # 健康检查 if curl -f http://localhost:7860/health; then echo ✅ 部署成功 # 发送部署成功通知 curl -X POST ${{ secrets.SLACK_WEBHOOK }} \ -H Content-Type: application/json \ -d { text: DAMO-YOLO手机检测系统部署成功\n版本: ${{ github.sha }}\n时间: $(date)\n环境: 生产 } else echo ❌ 部署失败服务未启动 exit 1 fi - name: Rollback on failure if: failure() run: | echo ⚠️ 部署失败执行回滚... # 这里可以添加回滚逻辑 curl -X POST ${{ secrets.SLACK_WEBHOOK }} \ -H Content-Type: application/json \ -d { text: ❌ DAMO-YOLO手机检测系统部署失败\n请立即检查\n提交: ${{ github.sha }}\n时间: $(date) }6.2 环境变量配置在GitHub仓库的Settings → Secrets中配置以下环境变量DOCKER_USERNAME: 你的Docker Hub用户名 DOCKER_PASSWORD: 你的Docker Hub密码或访问令牌 PRODUCTION_HOST: 生产服务器IP地址 PRODUCTION_USER: 服务器用户名 PRODUCTION_SSH_KEY: SSH私钥 SLACK_WEBHOOK: Slack通知Webhook URL可选6.3 测试数据管理创建测试数据管理脚本确保测试的一致性和可重复性tests/conftest.py:import pytest import os import sys from pathlib import Path # 添加项目根目录到Python路径 project_root Path(__file__).parent.parent sys.path.insert(0, str(project_root)) pytest.fixture(scopesession) def test_data_dir(): 返回测试数据目录 return project_root / tests / test_data pytest.fixture(scopesession) def sample_images(test_data_dir): 返回样本图片路径 images_dir test_data_dir / test_images return list(images_dir.glob(*.jpg)) list(images_dir.glob(*.png)) pytest.fixture(scopesession) def expected_results(test_data_dir): 返回预期结果 import json results_file test_data_dir / expected_results / detection_results.json with open(results_file, r) as f: return json.load(f) pytest.fixture def mock_image(): 创建模拟图片 import numpy as np return np.random.randint(0, 255, (480, 640, 3), dtypenp.uint8) pytest.fixture def phone_detector(): 创建手机检测器实例 from app.models.detector import PhoneDetector return PhoneDetector()7. 总结与最佳实践7.1 流水线带来的价值通过实施这套CI/CD流水线我们获得了以下收益开发效率提升代码提交后自动运行测试即时反馈问题自动构建Docker镜像确保环境一致性一键部署到生产环境减少人工操作代码质量有保障减少生产环境问题运维成本降低部署过程标准化减少人为错误完整的部署日志便于问题排查自动回滚机制降低故障影响监控告警集成及时发现问题团队协作改进统一的代码质量标准透明的部署流程自动化的代码审查可追溯的版本管理7.2 最佳实践建议基于我们的实践经验总结以下几点最佳实践渐进式实施不要一次性实现所有功能先从基础测试开始逐步添加更多阶段快速反馈确保测试阶段快速完成开发人员能及时获得反馈失败快速在流水线早期发现并报告问题避免问题传递到后续阶段环境一致性使用Docker确保开发、测试、生产环境一致安全第一妥善保管密钥和凭证使用GitHub Secrets管理敏感信息监控告警集成监控系统及时发现问题并通知相关人员文档完善为每个流水线阶段编写清晰的文档方便团队理解和使用7.3 后续优化方向这套流水线还可以进一步优化性能测试集成添加性能测试阶段确保新版本不会降低系统性能蓝绿部署实现零停机部署提高系统可用性多环境部署支持开发、测试、预生产、生产多环境部署自动化回滚当部署失败时自动回滚到上一个稳定版本成本优化优化CI/CD运行时间降低云计算成本移动端测试集成移动端自动化测试确保在手机上的兼容性7.4 开始行动如果你还没有为你的AI项目设置CI/CD流水线现在就是开始的好时机从简单开始先设置基本的测试流水线逐步完善根据项目需求逐步添加更多功能团队培训确保团队成员了解和使用流水线持续改进定期回顾和优化流水线配置记住好的CI/CD流水线不是一蹴而就的而是随着项目发展不断演进和完善的。从今天开始让你的部署过程变得更加简单、可靠和高效。获取更多AI镜像想探索更多AI镜像和应用场景访问 CSDN星图镜像广场提供丰富的预置镜像覆盖大模型推理、图像生成、视频生成、模型微调等多个领域支持一键部署。