Speech Evaluation & Data Tooling at ByteDance

This case describes the workflow in general terms and does not show internal interfaces, real business data, or implementation details.

本案例以通用方式描述工作流程，不展示内部界面、真实业务数据或具体实现。

Speech evaluation set iteration

Speech evaluation sets need to evolve with model capabilities and real-world use. For this release, we redesigned the capability taxonomy and questions, adding complex scenarios and recurring Bad Cases. The set grew to three times the size of the previous release.

I owned question selection, adaptation for speech, audio synthesis, and delivery review. I also designed a quality workflow combining automatic pre-labeling (PE), human annotation, automated checks, and manual sampling. I worked with product and algorithm colleagues on the foundational and applied capability taxonomy, and with engineering on evaluation runs.

Automatic pre-labeling was evaluated on hundreds of stratified samples and reached about 88.6% overall agreement with human labels. Agreement was about 82% on difficult Bad Cases and up to 97% on regular sets. Despite tripling the volume, we reduced the build cycle by 20%.

语音评测集迭代

语音评测集需要随模型能力和使用场景持续迭代。这一版重构了能力分类和题目设计，新增复杂场景与高频 Bad Case，题量扩展到上一版的三倍。

我负责题目筛选与语音化改造、音频合成和交付验收，并搭建「自动预标（PE）—人工标注—自动质检—人工抽检」质量流程。我与产品、算法同事共同完善「基础能力 + 应用能力」分类框架，并与工程同事协作完成评测执行。

在数百条分层样本上验证自动预标，与人工标签总体一致率约 88.6%；高难 Bad Case 约 82%，常规集最高约 97%。在题量扩为三倍的前提下，建设周期缩短 20%。

Evaluation build process评测集建设流程

Select and adapt candidate questions筛选并语音化改造候选题目
Synthesize and validate audio合成并校验音频
Automatic pre-labeling (PE)自动预标（PE）
Human annotation人工标注
Automated QC and human sampling自动质检 + 人工抽检
Evaluation runs with engineering与工程协同完成评测执行

My work modules我负责的模块

Evaluation-set design评测集设计

Selection and adaptation题目筛选与语音化改造
Adaptation rules适配规则
Capability taxonomy collaboration协同完善能力分类

Data production and delivery数据生产与交付

Audio synthesis音频合成
Supplier specification and management供应商规范与管理
Delivery review交付验收

Annotation and quality control标注与质量控制

Automatic pre-labeling (PE)自动预标（PE）
Human annotation process人工标注流程
Automated QC自动质检
Manual sampling人工抽检

Speech-data production tool

Previously, internal speech-data production relied on scripts or one-at-a-time operations on internal pages, which did not fit workflows centered on Lark sheets. External annotation teams lacked a convenient, unified way to audition production-aligned outputs, making it difficult to judge the model’s practical limits.

I independently defined the requirements, designed the product, and delivered it with AI-assisted development. The tool offered two access points: a batch workspace built around Lark sheets for internal users, and a comment bot that gave external annotators access to production-aligned synthesis. Speech recognition, VAD, timestamps, and source-text comparison were used to check generated audio automatically.

The initial rollout covered about 30 internal and external collaborators.

语音数据生产工具

内部语音数据生产过去依赖脚本或内网页面逐条操作，难以融入以飞书表格为中心的工作流。外部标注团队缺少便捷、统一的合成入口，难以试听与实际生产环境一致的结果，也不易判断模型能力边界。

我独立完成需求定义、产品设计和 AI 辅助开发，将分散的生产流程整合为一个内部工具。工具包含两个入口：内部同事通过面向飞书表格的批量工作台生成音频，外部标注团队通过评论机器人使用与生产一致的合成能力。系统结合语音识别、VAD、时间戳和原文对照，对合成结果进行自动质检。

首批覆盖约 30 名内外部协作者。

Before and after the tool数据生产流程前后对比

Before之前

After之后

Internal内部

Scripts or one-at-a-time internal-page operations脚本或内网页面逐条操作
Work detached from Lark sheets工作与飞书表格脱节

Open the floating speech workbench in the sheet workspace在表格工作区内展开浮动语音工作台
Batch configure, preflight, execute, and view progress批量配置、预检、执行并查看进度

External外部

Text-only review makes audio quality hard to judge纯文字审阅难以判断音频质量
Audition model differs from production试听能力与生产环境不一致

Call the comment bot in comments and view parameters在评论区调用评论机器人并查看参数
Audition with production-aligned synthesis使用与生产一致的合成能力完成试听

Shared quality control共享的质量控制

Speech recognition语音识别
VADVAD
Timestamps时间戳
Source-text comparison原文对照