Weiyan Wang

Hi there! My name is Weiyan Wang(王伟俨). Currently, I am working for Tencent, where I fortunately get supports from Project Up(青云计划, 原技术大咖). I focus on the co-design of both system and algorithm for large scale machine learning system. By exploring the large design space combining system and algorithm, I try to find a better way to make the big models trained on big data really happen.

Brief Biography

I am working as a senior R&D engineer in Machine Learning Platform Department(MLPD), Tencent. Before that, I obtained my the PhD. degree in Computer Science and Engineering Department, Hong Kong University of Science and Technology, advised by Prof. Kai Chen. I have received my M.Phil. degree on Computer Software and Theory from Institution of Software, Chinese Academy of Sciences under the supervision of Prof. Yunquan Zhang and Dr. Guoping Long. My B.Eng. degree in Computer Science and Technology is from Huazhong University of Science and Technology.

Research Interests

Co-design in both System and Algorithm especially for:

• Efficient big model training on big data
• Efficient model inference for real-time requirements
• Real big data without perfect supervision
  e.g. semi-supervised, unsupervised, noisy, out-of-distribution and sparse reward

Publications

Efficient Training

• Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding (Technical report, open source project)
  Zhimin Li, Jianwei Zhang, Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Yingfang Zhang, Xingchao Liu, Minbin Huang, Zedong Xiao, Dayou Chen, Jiajun He, Jiahao Li, Wenyue Li, Chen Zhang, Rongwei Quan, Jianxiang Lu, Jiabin Huang, Xiaoyan Yuan, Xiaoxiao Zheng, Yixuan Li, Jihong Zhang, Chao Zhang, Meng Chen, Jie Liu, Zheng Fang, Weiyan Wang, Jinbao Xue, Yangyu Tao, Jianchen Zhu, Kai Liu, Sihuan Lin, Yifu Sun, Yun Li, Dongdong Wang, Mingtao Chen, Zhichao Hu, Xiao Xiao, Yan Chen, Yuhong Liu, Wei Liu, Di Wang, Yong Yang, Jie Jiang, Qinglin Lu
• Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling NeurIPS 2024(CCF-A)
  Shuaipeng Li, Penghao Zhao, Hailin Zhang, Samm Sun, Hao Wu, Dian Jiao, Weiyan Wang, Chengjun Liu, Zheng Fang, Jinbao Xue, Yangyu Tao, Bin CUI, Di Wang
• Addressing Network Bottlenecks with Divide-and-Shuffle Synchronization for Distributed DNN Training, Infocom 2022(CCF-A)
  Weiyan Wang, Cengguang Zhang, Liu Yang, Kai Chen, Kun Tan
• Rethinking transport layer design for distributed machine learning, APNet 2019（CCF-C）
  Jiacheng Xia, Gaoxiong Zeng, Junxue Zhang, Weiyan Wang, Wei Bai, Junchen Jiang, Kai Chen
• Quantifying the Performance of Federated Transfer Learning, IJCAI FL workshop 2019 (Best Student Paper)
  Qinghe Jing, Weiyan Wang, Junxue Zhang, Han Tian, Kai Chen
• Domain-specific Communication Optimization for Distributed DNN Training, preprint 2020
  Hao Wang, Jingrong Chen, Xinchen Wan, Han Tian, Jiacheng Xia, Gaoxiong Zeng, Weiyan Wang, Kai Chen, Wei Bai, Junchen Jiang

Efficient Inference

• Exploiting Student Parallelism for Low-latency GPU Inference of BERT-like Models in Online Services (Technical report)
  Weiyan Wang, Yilun Jin, Yiming Zhang, Victor Junqiu Wei, Han Tian, Li Chen, Kai Chen
• MDP: Model Decomposition and Parallelization of Vision Transformer for Distributed Edge Inference, IEEE MSN(CCF-C)
  Weiyan Wang, Yiming Zhang, Yilun Jin, Han Tian, Li Chen
• Enabling Edge-Cloud Video Analytics for Robotics Applications, Infocom 2021(CCF-A) & full version in Trans on Cloud Computing(CCF-C)
  Yiding Wang, Weiyan Wang, Duowen Liu, Xin Jin, Junchen Jiang, Kai Chen
• Bridging the edge-cloud barrier for real-time advanced vision analytics, Hotcloud 2019(CCF-C)
  Yiding Wang, Weiyan Wang, Junxue Zhang, Junchen Jiang, Kai Chen
• CLSIFT: An Optimization Study of the Scale Invariance Feature Transform on GPUs, HPCC 2013(CCF-C)
  Weiyan Wang, Yunquan Zhang, Guoping Long, Shengen Yan, Haipeng Jia
• Parallelization and performance optimization on face detection algorithm with OpenCL: A case study[J]., Tsinghua Science and Technology (SCI)
  Weiyan Wang,Yunquan Zhang, Shengen Yan, Ying Zhang, Haipeng Jia
• Accelerating Viola-Jones Face Detection Algorithm on GPUs, HPCC 2012(CCF-C)
  Haipeng Jia, Yunquan Zhang, Weiyan Wang, Jianliang Xu

Real Big Data without perfect supervision

• BeamVQ: Aligning Space-Time Forecasting Model via Self-training on Physics-aware Metrics (Technical report) Hao Wu, Xingjian Shi, Ziyue Huang, Penghao Zhao, Wei Xiong, Jinbao Xue, Yangyu Tao, Xiaomeng Huang, Weiyan Wang(Corresponding)
• Prometheus: Out-of-distribution Fluid Dynamics Modeling with Disentangled Graph ODE, ICML 2024（CCF-A)
  Hao Wu, Huiyuan Wang, Kun Wang, Weiyan Wang, ChanganYe, Yangyu Tao, Chong Chen, Xian-Sheng Hua, Xiao Luo
• Multi-task Learning Based Keywords Weighted Siamese Model for Semantic Retrieval, PAKDD 2023(CCF-C)
  Mengmeng Kuang, Zhenhong Chen, Weiyan Wang(Corresponding), Lie Kang, Qiang Yan, Min Tang, Penghui Hao
• Multi-Objective Congestion Control, EuroSys 2022(CCF-A)
  Yiqing Ma, Han Tian, Xudong Liao, Junxue Zhang, Weiyan Wang, Kai Chen, Xin Jin
• Efficient two-stage label noise reduction for retrieval-based tasks, WSDM 2022(CCF-B)
  Mengmeng Kuang, Weiyan Wang, Zhenhong Chen, Lie Kang, Qiang Yan
• Integrating User and Agent Models:A Deep Task-Oriented Dialogue System, preprint 2017
  Weiyan Wang, Yuxiang WU, Yu Zhang, Zhongqi Lu, Kaixiang Mo, Qiang Yang

Projects and Internships

I lead or co-ordinate several HKUST-Enterprise Cooperated Projects:

• Central Software Institute, Huawei: Divide-and-Shuffle Synchronization
• Wechat, Tencent: Learn to label
• PCL and Clustar : Federated Learning Accelerating

I also have the fortune to work with some outstanding colleagues during the part-time internships and the full-time job:

• 2020.6——2022.3      WeChat Search
• 2019.9——2020.1       Pengcheng Lab
• 2014.7——2015.9       headquarter, Bank of China (full time)
• 2014.4——2014.6       IDL, Baidu
• 2013.8——2013.10     Core system， Tabao
• 2013.5——2013.7       AMD China Research

Professional Services and Activities

• 2024             KDD 2025 reviewer
• 2023             Globalcom Workshop AINextGenWN reviewer
• 2022              ICNCIT reviewer
• 2020              Sigcom artifact reviewer
• 2013-2018     CSDN OpenCL forum moderator
• 2011-2013     OpenCV code contributor

Teaching and Knowledge Sharing

• Teaching Assistant in COMP 1021, Spring and Fall 2016, Spring 2017, HKUST
• Teaching Assistant in MSBD 6000B(Deep Learning), Fall 2017, HKUST
• Lecture in Deep Learning Workshop: Diving into CNN, 2019, HKSAIR-HKUST, slides
• Chapter Contributor for the book Transfer Learning

Miscellaneous

• I am a badminton fan and amateur player, especially for the men’s double game. I prefer combining drop shots with smashes to win a game. I usually use a even-balance and long-size Duora racket from Yonex to better control the ball.
• If have time, I also like reading, hiking and watching football to refresh and relax myself.

Links

• Google Scholar
• LinkedIn
• Work Email: wwangbc AT connect.ust.hk
• Personal Email: wangweiyanster AT gmail.com

Last updated on 29th September, 2024
created on 4th Jan, 2022