K100_AI两卡全离线部署超长音视频说话人角色确认转录系统（多轮推理）

一、引言

之前我发表的一篇文章https://blog.csdn.net/pla88888888/article/details/162817260（K100_AI两卡全离线部署音视频说话人角色确认转录系统），采用SoulX-Transcriber多模态大模型实现，在实测中发现超长音视频文件分析时会出现重复输出。这次我尝试用Speaker-Reasoner-4194h多模态大模型来解决该问题，自编程序思路来源于官方代码，代码链接为：https://github.com/ASLP-lab/Speaker-Reasoner/tree/main，如果您觉得该大模型对本工作有用，请引用：

@article{lin2026speakerreasoner,

  title={Speaker-Reasoner: Scaling Interaction Turns and Reasoning Patterns for Timestamped Speaker-Attributed ASR},

  author={Zhennan Lin and Shuai Wang and Zhaokai Sun and Pengyuan Xie and Chuan Xie and Jie Liu and Qiang Zhang and Lei Xie},

  year={2026},

  eprint={2604.03074},

  archivePrefix={arXiv},

  primaryClass={eess.AS},

  url={https://arxiv.org/abs/2604.03074},

}

二、方案设计

1. 项目背景与目标

在会议记录、访谈整理等场景中，精确的说话人分离与时间轴对齐是核心需求。本系统旨在利用多模态大模型“Speaker-Reasoner-4194h”的音频理解与推理能力，通过多轮对话机制，实现端到端的智能转录：用户只需上传音频或视频文件，即可自动获得带有说话人ID、性别、精确时间戳的完整转写文本。

2. 系统架构

系统采用B/S架构，前端基于HTML模板提供简洁的上传界面；后端基于Flask框架，负责请求接收、任务调度与结果返回。核心组件包括：

文件预处理模块：使用ffmpeg将任意格式视频/音频统一转码为16kHz单声道WAV，保证模型输入一致性。

推理引擎：通过HTTP接口调用自部署的vLLM服务，承载Speaker-Reasoner-4194h模型。

多轮对话控制模块：实现自适应温度调节、重复输出检测、最大重试与边界校验，确保推理鲁棒性。

转录解析模块：支持秒数、HH:MM:SS、MM:SS.ss等多类时间戳格式，并过滤占位符等无效行，输出结构化片段。

3. 核心工作流程

（1）用户通过Web页面上传文件，后端进行安全校验并保存为临时文件。

（2）ffmpeg转换为16kHz单声道WAV格式，创建临时工作目录用于存放后续音频片段。

（3）发起首轮对话：模型接收完整音频与初始提示，被要求概括整体并给出第一个片段的起止时间边界。

（4）进入循环：每轮解析模型返回的<box>标签获取边界，用ffmpeg精确切出片段音频，构造“当前片段作为O_turn”的新消息，驱动模型逐步推导该片段的说话人、性别与文本。

（5）模型在最终回复中输出<answer>标签，包含所有片段的完整转录。若未正常结束，则从最后一轮提取后备结果。

（6）解析转录文本，转换为带有start、end、speaker、text的标准JSON分段数组，连同原始文本一并返回前端展示。

（7）全程自动清理所有临时文件，保证服务器空间安全。

4. 关键技术细节

鲁棒性设计：单轮最多重试5次，前两次使用极低温度（0.01）和受限采样以抑制随机性，后续逐步放宽；引入重复惩罚系数，并检测连续重复10次以上的模式，一旦发现即触发重试。

边界校验：严格检查<box>中结束时间必须大于起始时间，否则终止对话，防止无效片段。

资源管理：上传限制500MB，所有临时文件使用try-finally确保释放，避免残留。

多格式兼容：解析模块同时支持Speaker [0.0, 3.68]: text和[00:01:23 -> 00:01:45] Speaker1: text等，并通过统一的秒数→时间码转换函数保证输出一致。

5. 部署与展望

系统部署于Flask内置服务器，监听5000端口，可与vLLM服务同机或分布式部署。后续可扩展为异步任务队列，支持更大规模并发处理，并增加说话人声纹注册与自动命名等功能，进一步提升转录体验。

下图展示了整体数据流：

三、实施方法及代码

3.1硬件环境

本方案的硬件平台为一台H3C服务器，配置如下：

组件	规格
CPU	2×海光74902.7GHz64C
内存	16×32GDDR5
GPU	8×海光DCU64GBK100_AI

需占用两张K100_AI显卡

3.2软件栈

本方案的软件栈基于Docker容器化技术构建，使用经过海光DCU适配的vLLM推理镜像：

镜像（Speaker-Reasoner-4194h）：
harbor.sourcefind.cn:5443/dcu/admin/base/custom:vllm0.15.1-ubuntu22.04-dtk26.04-0130-py3.10-20260220

该镜像基于vLLM0.15.1推理框架、DTK26.04，Python3.10环境，双卡部署Speaker-Reasoner-4194h多模态大模型。

软件项目：

1、Speaker-Reasoner-4194h环境部署参照https://developer.sourcefind.cn/modelzoo/list/qwen3-omni_vllm/detail?post_id=3563caba-356c-11f1-be71-0242ac150003里面操作步骤，项目下载链接为：https://developer.sourcefind.cn/codes/modelzoo/qwen3-omni_vllm/-/archive/v1.0/qwen3-omni_vllm-v1.0.zip。

2、离线音视频说话人角色确认转录系统自编程序代码如下：

Speaker-Reasoner-web-2.py代码如下：

import os
import re
import tempfile
import subprocess
import requests
import logging
from flask import Flask, request, jsonify, render_template
from werkzeug.utils import secure_filename
app = Flask(__name__)
app.config['MAX_CONTENT_LENGTH'] = 500 * 1024 * 1024  # 500 MB
VLLM_URL = "http://localhost:8082/v1/chat/completions"
MODEL_NAME = "Speaker-Reasoner-4194h"
# 提示词（来自 plugin/prompt_4194h.json）
INITIAL_USER_TEXT = (
    "分析这段音频$X$，将其划分为合适的片段。对于每个片段，"
    "请逐步推导说话人ID、性别、时间戳及转录内容。"
    "现在，请先概括音频整体情况，并给出第一个片段的边界。"
)
TURN_USER_TEXT = "当前片段作为O_{turn}。"
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
def _seconds_to_hms(seconds_str: str) -> str:
    """将秒数字符串转换为 HH:MM:SS.sss 格式，便于前端统一处理"""
    try:
        total_seconds = float(seconds_str)
    except ValueError:
        return seconds_str  # 转换失败则原样返回
    hours = int(total_seconds // 3600)
    minutes = int((total_seconds % 3600) // 60)
    secs = total_seconds % 60
    return f"{hours:02d}:{minutes:02d}:{secs:06.3f}"
def parse_transcript(text: str):
    """
    解析转录文本，支持多种时间戳格式，过滤无意义的占位行。
    增加支持：Speaker [秒数, 秒数]: 内容
    """
    lines = text.strip().split('n')
    segments = []
    # 匹配 --:--:-- --> --:--:-- 占位行
    placeholder_pattern = re.compile(r'^--:--:--s*(?:-->|→)s*--:--:--$')
    # 新增：匹配 "Female 1 [0.0, 3.68]: 王姐，我爸妈来看你了。" 格式
    pattern_sec = re.compile(
        r'(.+?)s*[s*(d+(?:.d+)?)s*,s*(d+(?:.d+)?)s*]s*:s*(.*)'
    )
    # 原有格式
    pattern_hms = re.compile(r'[(d{2}:d{2}:d{2})s*->s*(d{2}:d{2}:d{2})]s*(Speakerd+):s*(.*)')
    pattern_ms  = re.compile(r'[(d{2}:d{2}.d+)s*-->s*(d{2}:d{2}.d+)]s*(Speakerd+):s*(.*)')
    pattern_no_time = re.compile(r'(Speakerd+):s*(.*)')
    for line in lines:
        line = line.strip()
        if not line:
            continue
        if placeholder_pattern.match(line):
            continue
        # 1. 秒数格式
        match = pattern_sec.match(line)
        if match:
            speaker, start_sec, end_sec, content = match.groups()
            segments.append({
                'start': _seconds_to_hms(start_sec),
                'end': _seconds_to_hms(end_sec),
                'speaker': speaker.strip(),
                'text': content.strip()
            })
            continue
        # 2. HH:MM:SS 格式
        match = pattern_hms.match(line)
        if match:
            start, end, speaker, content = match.groups()
            segments.append({'start': start, 'end': end, 'speaker': speaker, 'text': content})
            continue
        # 3. MM:SS.ss 格式
        match = pattern_ms.match(line)
        if match:
            start, end, speaker, content = match.groups()
            segments.append({'start': start, 'end': end, 'speaker': speaker, 'text': content})
            continue
        # 4. 仅有 Speaker 无时间
        match = pattern_no_time.match(line)
        if match:
            speaker, content = match.groups()
            segments.append({'start': None, 'end': None, 'speaker': speaker, 'text': content})
        else:
            segments.append({'start': None, 'end': None, 'speaker': None, 'text': line})
    return segments
def cut_audio_segment(wav_path: str, start_sec: float, end_sec: float, tmp_dir: str) -> str:
    """使用 ffmpeg 从 wav 文件中切出一段音频"""
    segment_path = os.path.join(
        tmp_dir,
        f"{os.path.splitext(os.path.basename(wav_path))[0]}_{start_sec:.3f}_{end_sec:.3f}.wav"
    )
    cmd = [
        'ffmpeg',
        '-i', wav_path,
        '-ss', str(start_sec),
        '-to', str(end_sec),
        '-c:a', 'pcm_s16le',
        '-y',
        segment_path
    ]
    subprocess.run(cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    return segment_path
def call_speaker_reasoner(wav_path: str, tmp_dir: str) -> str:
    """
    通过 HTTP 调用 vLLM 服务完成多轮对话，
    返回最终 <answer> 标签内的文本。
    """
    messages = [
        {
            "role": "user",
            "content": [
                {"type": "audio_url", "audio_url": {"url": f"file://{os.path.abspath(wav_path)}"}},
                {"type": "text", "text": INITIAL_USER_TEXT}
            ]
        }
    ]
    max_turns = 100
    max_retries_per_turn = 5
    current_turn = 0
    answer = None
    last_response = ""
    segment_files = []
    while current_turn < max_turns:
        turn_success = False
        content = ""  # 用于记录本轮响应
        for attempt in range(max_retries_per_turn):
            try:
                # 动态调整温度
                if attempt < 2:
                    temperature = 0.01
                    top_p = 0.1
                    top_k = 1
                else:
                    temperature = min(0.1 + (attempt - 2) * 0.1, 1.0)
                    top_p = 0.9
                    top_k = -1
                # 增加 repetition_penalty 以减少重复输出
                repetition_penalty = 1.1 + attempt * 0.05
                payload = {
                    "model": MODEL_NAME,
                    "messages": messages,
                    "temperature": temperature,
                    "top_p": top_p,
                    "top_k": top_k,
                    "max_tokens": 8192,
                    "repetition_penalty": repetition_penalty
                }
                resp = requests.post(VLLM_URL, json=payload, timeout=600)
                resp.raise_for_status()
                result = resp.json()
                content = result['choices'][0]['message']['content']
                # 简单重复检测
                if re.search(r'(.+?)1{10,}', content.strip()):
                    raise RuntimeError("Repeated output detected")
                messages.append({"role": "assistant", "content": content})
                # 检查 <answer>
                answer_match = re.search(r'<answer>(.*?)</answer>', content, re.DOTALL)
                if answer_match:
                    answer = answer_match.group(1).strip()
                    current_turn = max_turns  # 跳出外层循环
                    turn_success = True
                    break
                # 检查 <box>，获取下一段边界
                box_match = re.search(r'<box>[s*(d+(?:.d+)?)s*,s*(d+(?:.d+)?)s*]</box>', content)
                if not box_match:
                    # 尝试宽松匹配：任何 [数字, 数字] 形式
                    fallback = re.search(r'[s*(d+(?:.d+)?)s*,s*(d+(?:.d+)?)s*]', content)
                    if fallback:
                        start_time = float(fallback.group(1))
                        end_time = float(fallback.group(2))
                    else:
                        # 彻底找不到边界，视为对话结束
                        logger.info("No <box> or time boundary found, ending conversation.")
                        turn_success = True
                        current_turn = max_turns
                        break
                else:
                    start_time = float(box_match.group(1))
                    end_time = float(box_match.group(2))
                # 验证边界有效性（结束时间必须大于起始时间）
                if end_time <= start_time:
                    logger.warning("Invalid time boundary: [%s, %s]. Ending conversation.", start_time, end_time)
                    turn_success = True
                    current_turn = max_turns
                    break
                # 切出音频片段
                segment_path = cut_audio_segment(wav_path, start_time, end_time, tmp_dir)
                segment_files.append(segment_path)
                turn_text = TURN_USER_TEXT.format(turn=current_turn + 1)
                messages.append({
                    "role": "user",
                    "content": [
                        {"type": "audio_url", "audio_url": {"url": f"file://{os.path.abspath(segment_path)}"}},
                        {"type": "text", "text": turn_text}
                    ]
                })
                turn_success = True
                current_turn += 1
                break
            except Exception as e:
                logger.warning("Turn %d, attempt %d failed: %s", current_turn, attempt, e)
                last_response = content if content else last_response
                # 回滚可能已添加的错误 assistant 消息
                if messages and messages[-1]['role'] == 'assistant':
                    messages.pop()
                # 如果是重复输出错误，可以额外记录详细内容以便调试
                if "Repeated output" in str(e):
                    logger.debug("Repeated content sample: %s", content[:200] if content else "N/A")
        if not turn_success:
            logger.error("Turn %d failed after all retries.", current_turn)
            break
    # 若未获得 <answer>，尝试从最后一条 assistant 消息提取
    if answer is None:
        if messages and messages[-1]['role'] == 'assistant':
            last_content = messages[-1]['content']
        else:
            last_content = last_response
        answer_match = re.search(r'<answer>(.*?)</answer>', last_content, re.DOTALL)
        if answer_match:
            answer = answer_match.group(1).strip()
        else:
            # 若根本没有 <answer>，则返回全部最后回复作为后备
            answer = last_content.strip() if last_content else ""
    # 清理临时片段
    for f in segment_files:
        if os.path.exists(f):
            os.unlink(f)
    return answer
@app.route('/')
def index():
    return render_template('index.html')
@app.route('/transcribe', methods=['POST'])
def transcribe():
    if 'video' not in request.files:
        return jsonify({'error': 'No video/audio file provided'}), 400
    video_file = request.files['video']
    if video_file.filename == '':
        return jsonify({'error': 'Empty filename'}), 400
    filename = secure_filename(video_file.filename)
    input_tmp = None
    wav_path = None
    try:
        # 1. 保存上传文件
        suffix = os.path.splitext(filename)[1] or '.tmp'
        input_tmp = tempfile.NamedTemporaryFile(delete=False, suffix=suffix)
        video_file.save(input_tmp.name)
        input_tmp.close()
        # 2. 创建 WAV 临时文件（16kHz 单声道）
        wav_tmp = tempfile.NamedTemporaryFile(delete=False, suffix='.wav')
        wav_path = wav_tmp.name
        wav_tmp.close()
        # 3. ffmpeg 转码
        cmd = [
            'ffmpeg',
            '-i', input_tmp.name,
            '-vn',
            '-acodec', 'pcm_s16le',
            '-ar', '16000',
            '-ac', '1',
            '-y',
            wav_path
        ]
        subprocess.run(cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        # 4. 创建临时目录存放可能的音频片段
        tmp_dir = tempfile.mkdtemp()
        # 5. 调用 Speaker-Reasoner 多轮推理
        try:
            raw_answer = call_speaker_reasoner(wav_path, tmp_dir)
        finally:
            # 清理临时目录
            if os.path.exists(tmp_dir):
                for f in os.listdir(tmp_dir):
                    fp = os.path.join(tmp_dir, f)
                    try:
                        os.unlink(fp)
                    except Exception:
                        pass
                os.rmdir(tmp_dir)
        segments = parse_transcript(raw_answer)
        return jsonify({
            'success': True,
            'segments': segments,
            'raw_text': raw_answer
        })
    except subprocess.CalledProcessError as e:
        error_msg = e.stderr.decode() if e.stderr else 'ffmpeg failed'
        return jsonify({'error': f'Audio conversion failed: {error_msg}'}), 500
    except Exception as e:
        logger.exception("Transcription error")
        return jsonify({'error': str(e)}), 500
    finally:
        # 清理临时文件
        if input_tmp and os.path.exists(input_tmp.name):
            os.unlink(input_tmp.name)
        if wav_path and os.path.exists(wav_path):
            os.unlink(wav_path)
if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000, debug=True)

templates/index.html代码如下：

<!DOCTYPE html>
<html lang="zh-CN">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>超长离线音视频说话人角色确认转录系统（强推理版）</title>
    <style>
        * {
            box-sizing: border-box;
            margin: 0;
            padding: 0;
        }
        body {
            font-family: 'Segoe UI', Roboto, system-ui, sans-serif;
            background: #f5f7fa;
            color: #1e293b;
            padding: 2rem;
            display: flex;
            flex-direction: column;
            align-items: center;
        }
        .container {
            max-width: 1400px;
            width: 100%;
            background: white;
            border-radius: 24px;
            box-shadow: 0 20px 40px -12px rgba(0,0,0,0.2);
            padding: 2rem;
        }
        h1 {
            font-weight: 600;
            margin-bottom: 1.5rem;
            display: flex;
            align-items: center;
            gap: 0.5rem;
        }
        .upload-area {
            border: 2px dashed #cbd5e1;
            border-radius: 16px;
            padding: 2rem;
            text-align: center;
            margin-bottom: 2rem;
            transition: background 0.2s;
            cursor: pointer;
        }
        .upload-area:hover {
            background: #f8fafc;
        }
        .upload-area input[type="file"] {
            display: none;
        }
        .upload-label {
            font-size: 1.1rem;
            color: #475569;
        }
        .upload-label strong {
            color: #2563eb;
        }
        .player-wrapper {
            display: flex;
            flex-direction: row;
            gap: 2rem;
            margin-top: 1rem;
        }
        .video-section {
            flex: 1.2;
            min-width: 0;
        }
        .video-section video {
            width: 100%;
            border-radius: 12px;
            background: #0f172a;
            display: block;
        }
        .transcript-section {
            flex: 1.5;
            max-height: 500px;
            overflow-y: auto;
            background: #f1f5f9;
            border-radius: 12px;
            padding: 0.5rem 0;
            scroll-behavior: smooth;
        }
        .transcript-line {
            padding: 0.75rem 1.2rem;
            border-left: 3px solid transparent;
            cursor: pointer;
            transition: background 0.15s, border-color 0.15s;
            font-size: 0.95rem;
            line-height: 1.6;
            display: flex;
            align-items: baseline;
            gap: 0.8rem;
        }
        .transcript-line:hover {
            background: #e2e8f0;
        }
        .transcript-line.active {
            background: #dbeafe;
            border-left-color: #2563eb;
        }
        .transcript-line .time {
            font-family: 'JetBrains Mono', monospace;
            font-size: 0.8rem;
            color: #64748b;
            white-space: nowrap;
            min-width: 120px;
            flex-shrink: 0;
            margin-right: 0.5rem;
        }
        .transcript-line .speaker {
            font-weight: 600;
            color: #7c3aed !important;
            margin-right: 0.25rem;
            white-space: nowrap;
            flex-shrink: 0;
        }
        .transcript-line .text {
            word-break: break-word;
        }
        .status {
            margin-top: 1rem;
            padding: 0.75rem 1rem;
            background: #f1f5f9;
            border-radius: 8px;
            color: #475569;
            font-size: 0.95rem;
        }
        .status .error {
            color: #dc2626;
        }
        .status .success {
            color: #16a34a;
        }
        .loader {
            display: inline-block;
            width: 16px;
            height: 16px;
            border: 3px solid #e2e8f0;
            border-top-color: #2563eb;
            border-radius: 50%;
            animation: spin 0.6s linear infinite;
            margin-right: 0.5rem;
            vertical-align: middle;
        }
        @keyframes spin {
            to { transform: rotate(360deg); }
        }
        @media (max-width: 768px) {
            .player-wrapper {
                flex-direction: column;
            }
            .transcript-section {
                max-height: 300px;
            }
        }
    </style>
</head>
<body>
    <div class="container">
        <h1>🎤 超长离线音视频说话人角色确认转录系统（强推理版）</h1>
        <div class="upload-area" id="uploadArea">
            <div class="upload-label">
                <strong>点击选择音视频文件</strong> 或拖拽到这里<br>
                <span style="font-size:0.9rem; color:#94a3b8;">支持WAV / MP3 / MP4 / MOV / AVI 等格式</span>
            </div>
            <input type="file" id="videoInput" accept="audio/*,video/*">
        </div>
        <div id="status" class="status">📁 请上传视频文件以开始转录</div>
        <div class="player-wrapper" id="playerWrapper" style="display:none;">
            <div class="video-section">
                <video id="videoPlayer" controls></video>
            </div>
            <div class="transcript-section" id="transcriptContainer">
                <!-- 转录行将动态渲染在这里 -->
            </div>
        </div>
    </div>
    <script>
    (function() {
        const videoInput = document.getElementById('videoInput');
        const uploadArea = document.getElementById('uploadArea');
        const statusDiv = document.getElementById('status');
        const playerWrapper = document.getElementById('playerWrapper');
        const videoPlayer = document.getElementById('videoPlayer');
        const transcriptContainer = document.getElementById('transcriptContainer');
        let segments = [];
        let timeUpdateHandler = null;
        // ============= 上传区域 =============
        uploadArea.addEventListener('click', () => videoInput.click());
        uploadArea.addEventListener('dragover', (e) => {
            e.preventDefault();
            uploadArea.style.borderColor = '#2563eb';
        });
        uploadArea.addEventListener('dragleave', () => {
            uploadArea.style.borderColor = '#cbd5e1';
        });
        uploadArea.addEventListener('drop', (e) => {
            e.preventDefault();
            uploadArea.style.borderColor = '#cbd5e1';
            if (e.dataTransfer.files.length) {
                videoInput.files = e.dataTransfer.files;
                videoInput.dispatchEvent(new Event('change'));
            }
        });
        // ============= 文件选择与转录 =============
        videoInput.addEventListener('change', async function() {
            const file = this.files[0];
            if (!file) return;
            if (timeUpdateHandler) {
                videoPlayer.removeEventListener('timeupdate', timeUpdateHandler);
                timeUpdateHandler = null;
            }
            const url = URL.createObjectURL(file);
            videoPlayer.src = url;
            videoPlayer.style.display = 'block';
            playerWrapper.style.display = 'flex';
            statusDiv.innerHTML = '⏳ 正在上传并转录，请稍候...';
            const formData = new FormData();
            formData.append('video', file);
            try {
                const response = await fetch('/transcribe', {
                    method: 'POST',
                    body: formData
                });
                const data = await response.json();
                if (!response.ok || !data.success) {
                    throw new Error(data.error || '转录失败');
                }
                segments = data.segments || [];
                renderTranscript(segments);
                statusDiv.innerHTML = '✅ 转录完成，点击句子可跳转播放';
                timeUpdateHandler = syncHighlight;
                videoPlayer.addEventListener('timeupdate', timeUpdateHandler);
            } catch (err) {
                statusDiv.innerHTML = `❌ 错误：${err.message}`;
                console.error(err);
            }
        });
        // ============= 事件委托：点击行跳转 =============
        transcriptContainer.addEventListener('click', (event) => {
            const line = event.target.closest('.transcript-line');
            if (!line) return;
            const startStr = line.dataset.start;
            console.log('点击行 data-start:', startStr);
            if (!startStr) {
                console.log('此行无有效时间戳，无法跳转');
                return;
            }
            const seconds = timeToSeconds(startStr);
            console.log(`跳转到 ${startStr} (${seconds}秒)`);
            if (isNaN(seconds) || seconds < 0) {
                console.error('时间解析失败');
                return;
            }
            const doSeek = () => {
                videoPlayer.currentTime = seconds;
                videoPlayer.play().catch(e => {
                    console.warn('播放被阻止，尝试静音', e);
                    videoPlayer.muted = true;
                    videoPlayer.play();
                });
            };
            if (videoPlayer.readyState >= 1) {
                doSeek();
            } else {
                videoPlayer.addEventListener('loadedmetadata', function ready() {
                    videoPlayer.removeEventListener('loadedmetadata', ready);
                    doSeek();
                });
            }
        });
        // ============= 渲染转录文本 =============
        function renderTranscript(segments) {
            transcriptContainer.innerHTML = '';
            if (!segments || segments.length === 0) {
                transcriptContainer.innerHTML = '<div style="padding:1rem;color:#94a3b8;">未识别到语音内容</div>';
                return;
            }
            segments.forEach((seg, index) => {
                const line = document.createElement('div');
                line.className = 'transcript-line';
                line.dataset.index = index;
                let start = seg.start;
                let end = seg.end;
                let speaker = seg.speaker;
                let text = seg.text || '';
                // 如果后端未提供 start/end，尝试从文本中提取（兼容老格式）
                if (!start) {
                    const timeRe = /[?(d{2}:d{2}(?::d{2})?(?:.d+)?)s*(?:-->|→|->)s*(d{2}:d{2}(?::d{2})?(?:.d+)?)]?/;
                    const timeMatch = text.match(timeRe);
                    if (timeMatch) {
                        start = timeMatch[1];
                        end = timeMatch[2];
                        text = text.replace(timeMatch[0], '').trim();
                    }
                }
                // 尝试从文本中提取 Speaker 标签（兼容 Speaker X: 格式）
                const spMatch = text.match(/^(Speakers*d+|Females*d+|Males*d+)s*:s*(.*)/i);
                if (spMatch) {
                    speaker = spMatch[1];
                    text = spMatch[2];
                }
                // 格式化显示时间：若 start/end 是纯秒数，转为 MM:SS.ss
                function formatTimeDisplay(timeStr) {
                    if (!timeStr) return '';
                    // 包含冒号，可能是 HH:MM:SS 或 MM:SS，原样返回
                    if (timeStr.includes(':')) return timeStr;
                    // 纯数字（秒）
                    const sec = parseFloat(timeStr);
                    if (isNaN(sec)) return timeStr;
                    const mins = Math.floor(sec / 60);
                    const remainSec = (sec % 60).toFixed(2);
                    return `${String(mins).padStart(2, '0')}:${String(remainSec).padStart(5, '0')}`;
                }
                const displayStart = formatTimeDisplay(start);
                const displayEnd = formatTimeDisplay(end);
                // 设置 data 属性用于跳转（保留原始值，方便 timeToSeconds 处理）
                if (start && end) {
                    line.dataset.start = start;   // 可能是 HH:MM:SS 或纯秒数字符串
                    line.dataset.end = end;
                    line.style.cursor = 'pointer';
                    const timeSpan = document.createElement('span');
                    timeSpan.className = 'time';
                    timeSpan.textContent = `${displayStart} → ${displayEnd}`;
                    line.appendChild(timeSpan);
                } else {
                    line.style.cursor = 'default';
                }
                // 说话人标签（紫色）
                if (speaker) {
                    const speakerSpan = document.createElement('span');
                    speakerSpan.className = 'speaker';
                    speakerSpan.textContent = speaker + ': ';
                    line.appendChild(speakerSpan);
                }
                // 文本内容
                const textSpan = document.createElement('span');
                textSpan.className = 'text';
                textSpan.textContent = text;
                line.appendChild(textSpan);
                transcriptContainer.appendChild(line);
            });
        }
        // ============= 同步高亮与滚动 =============
        function syncHighlight() {
            const currentTime = videoPlayer.currentTime;
            const lines = transcriptContainer.querySelectorAll('.transcript-line');
            let activeIndex = -1;
            for (let i = 0; i < segments.length; i++) {
                const seg = segments[i];
                if (seg.start && seg.end) {
                    const start = timeToSeconds(seg.start);
                    const end = timeToSeconds(seg.end);
                    if (currentTime >= start && currentTime <= end) {
                        activeIndex = i;
                        break;
                    }
                }
            }
            if (activeIndex === -1 && segments.length > 0) {
                for (let i = segments.length - 1; i >= 0; i--) {
                    const seg = segments[i];
                    if (seg.start) {
                        const start = timeToSeconds(seg.start);
                        if (currentTime >= start) {
                            activeIndex = i;
                            break;
                        }
                    }
                }
            }
            lines.forEach((line, idx) => {
                line.classList.toggle('active', idx === activeIndex);
            });
            if (activeIndex !== -1 && lines[activeIndex]) {
                lines[activeIndex].scrollIntoView({ block: 'center', behavior: 'smooth' });
            }
        }
        // ============= 工具函数 =============
        function timeToSeconds(timeStr) {
            if (!timeStr) return 0;
            // 如果是纯数字（秒），直接返回
            if (!timeStr.includes(':')) {
                const sec = parseFloat(timeStr);
                return isNaN(sec) ? 0 : sec;
            }
            const parts = timeStr.split(':');
            if (parts.length === 3) {
                return parseInt(parts[0]) * 3600 + parseInt(parts[1]) * 60 + parseFloat(parts[2]);
            } else if (parts.length === 2) {
                return parseInt(parts[0]) * 60 + parseFloat(parts[1]);
            }
            return 0;
        }
        window.addEventListener('beforeunload', () => {
            if (videoPlayer.src) {
                URL.revokeObjectURL(videoPlayer.src);
            }
            if (timeUpdateHandler) {
                videoPlayer.removeEventListener('timeupdate', timeUpdateHandler);
            }
        });
    })();
    </script>
</body>
</html>

3.3模型下载与准备

Speaker-Reasoner-4194h（面向带时间戳说话人归属自动语音识别的交互轮次与推理模式扩展多模态大模型）：

下载链接：https://modelscope.cn/models/ASLP-lab/Speaker-Reasoner-4194h/files

3.4模型启动参数

Speaker-Reasoner-4194h启动参数：

cat SoulX-Transcriber.sh 
vllm serve "/home/models/Speaker-Reasoner-4194h" 
    --port 8082 
    --served-model-name Speaker-Reasoner-4194h 
    --trust-remote-code 
    --tensor-parallel-size 2 
    --dtype bfloat16 
    --max-model-len 65536 
    --gpu-memory-utilization 0.95 
    --allowed-local-media-path / 
--chat-template  /home/models/qwen3-omni_vllm-main/Speaker-Reasoner-web/chat_template.jinja
cat /home/models/qwen3-omni_vllm-main/Speaker-Reasoner-web/chat_template.jinja
{% for message in messages %}{{ '<|im_start|>' + message['role'] + 'n' }}{% if message['content'] is string %}{{ message['content'] }}{% else %}{% for item in message['content'] %}{% if item['type'] == 'text' %}{{ item['text'] }}{% elif item['type'] == 'audio' %}{{ '<|audio_start|><|audio_pad|><|audio_end|>' }}{% elif item['type'] == 'image' %}{{ '<|vision_start|><|image_pad|><|vision_end|>' }}{% elif item['type'] == 'video' %}{{ '<|vision_start|><|video_pad|><|vision_end|>' }}{% endif %}{% endfor %}{% endif %}{{ '<|im_end|>n' }}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistantn' }}{% endif %}

四、运行测试

1、启动Speaker-Reasoner-4194h

docker exec -it Speaker-Reasoner-4194h bash
cd /home/models/
nohup ./Speaker-Reasoner-4194h.sh &

2、启动超长离线音视频说话人角色确认转录系统自编程序

python Speaker-Reasoner-web-2.py 
 * Serving Flask app 'Speaker-Reasoner-web-2'
 * Debug mode: on
INFO:werkzeug:WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:5000
 * Running on http://192.168.222.65:5000
INFO:werkzeug:Press CTRL+C to quit
INFO:werkzeug: * Restarting with stat
WARNING:werkzeug: * Debugger is active!
INFO:werkzeug: * Debugger PIN: 105-818-056

3、访问超长离线音视频说话人角色确认转录系统自编程序

用谷歌浏览器访问

转录完成后，点击右边有时间段的文本记录，播放器会自动跳转到指定时间位置进行播放