ByteDance

Borui Wan, Gaohong Liu, Zuquan Song, Jun Wang, Yun Zhang, Guangming Sheng, Shuguang Wang, Houmin Wei, Chenyuan Wang, Weiqiang Lou, Xi Yang, Mofan Zhang, Kaihua Jiang, Cheng Ren, Xiaoyun Zhi, Menghan Yu, Zhe Nan, Zhuolin Zheng, B. Zhong, Qinlong Wang, Huan Yu, Jinxin Chi, Wang Zhang, Yuhan Li, Zixian Du, Sida Zhao, Yong-Qiang Zhang, Jing-Zhe Tang, Zherui Liu, Chuan Wu, Yanghua Peng, Haibin Lin, Wencong Xiao, Xin Liu, Liang Xiang

The training scale of large language models (LLMs) has reached tens of thousands of GPUs and is still continuously expanding, enabling faster learning of larger models. Accompanying the expansion of the resource scale is the prevalence of failures (CUDA error, NaN values, job hang, etc.), which poses significant challenges to training stability. Any large-scale LLM training infrastructure should strive for minimal training interruption, efficient fault diagnosis, and effective failure tolerance to enable highly efficient continuous training. This paper presents ByteRobust, a large-scale GPU infrastructure management system tailored for robust and stable training of LLMs. It exploits the uniqueness of LLM training process and gives top priorities to detecting and recovering failures in a routine manner. Leveraging parallelisms and characteristics of LLM training, ByteRobust enables high-capacity fault tolerance, prompt fault demarcation, and localization with an effective data-driven approach, comprehensively ensuring continuous and efficient training of LLM tasks. ByteRobust is deployed on a production GPU platform and advances the state of the art in training robustness by achieving 97% ETTR for a three-month training job on 9,600 GPUs.

Yi-Rui Liu, Lidong Jiang, Deguo Li, Daxiang Kang, Zhaoyang Wei, Yuqi Chai, Bin Niu, Ke Lin, Xiaoning Ding, Jianwen Pi, Hao Luo

A Virtual Private Cloud (VPC) that enables both RDMA and TCP provides advantages for both tenants and cloud providers. It serves the flexible demands of RDMA and TCP of tenant applications while delivering a cost-effective solution compared to the construction of two distinct overlay networks. In this study, we introduce Jakiro, an innovative framework of vNIC design that supports both RDMA and TCP within ByteDance Cloud. Jakiro holds the capability to support fundamental VPC features such as QoS, security groups, etc., for both RDMA and TCP streams while maintaining compatibility with applications and intra-host RDMA optimization techniques. We benchmark Jakiro's performance using basic test cases and real-world high-performance computing applications and distributed machine learning training. The results indicate that the RDMA performance of Jakiro is close to that of the physical RDMA. Concurrently, Jakiro guarantees a weighted fair QoS between RDMA and TCP. Jakiro has been deployed in ByteDance Cloud for one year, we share our critical design and deployment decisions, as well as experiences and lessons from production.

Yitian Liang, Xin-Lei Chen, Shengnan Han, Jinglong Zhang, Yubo Chen

Spillover plays a crucial role in advertising effect on short-form video social platforms.

Jue Wang, Shuxiang Chen, Yu Liu, Yuan Deng, Lei Zhang, Yuanchang Fu, Bo Liu

Despite the advantages of the microservice architecture, there is an urgent need for automated testing microservice APIs, ensuring that each API functions correctly and reliably without exhaustive manual effort. This paper introduces TestGPT-Server, a novel, fully automated testing system for microservice APIs developed and deployed at ByteDance, leveraging the capabilities of large language models (LLMs) to enhance test request generation and bug detection. TestGPT-Server utilizes an LLM-enhanced interface analyzer and a code analyzer to extract syntactic and semantic information of the API under test. Based on the extracted information, it employs a targeted request generator and a guided stochastic fuzzer to produce comprehensive test requests. After sending generated test requests to the API, an LLM-assisted response analyzer identifies potential service crashes, and an assertion miner automatically mines functional assertions to detect functional bugs. Deployed at ByteDance, the evaluation results demonstrate that TestGPT-Server is effective, achieving a 40.52% improvement over existing test requests and identifying 50 crash bugs and 2 functional bugs across 14,366 microservice APIs, with minimal manual effort.

Jiajia Yang, Wowo Zhang

Against the backdrop of the thriving platform economy, the video content industry has seen the parallel development of diverse business models. As two core sectors, short-form video platforms and long-form video platforms have drawn significant attention for their market competition and development models. While they exhibit distinct characteristics and development paths, they also face the industry-wide challenge of how to achieve mutually beneficial interaction. This study takes ByteDance's short video platforms and Tencent Video, iQiyi, and other long video platforms as research objects to explore their differentiated competitive landscape and collaborative development paths. The findings reveal significant differences between the two types of platforms in user acquisition, business logic, and technological application, while also identifying collaborative opportunities in IP development, technological integration, and scenario expansion. Based on these findings, the study proposes a new industry development paradigm of differentiated competition + ecological collaboration. This research provides theoretical references and practical guidance for the sustainable development of the video content industry within the context of the platform economy.