Industrial Product R&D Experience

2022-Now, Running Startups
Meta, Research Scientist, 2020-2021

Did research and open source libraries with Meta PyTorch team on distributed training for foundation models (BERT, Vision Transformer, etc.). Published two papers and got highlights at the PyTorch.org official website:

[1] PyTorch RPC: Distributed Deep Learning Built on Tensor-Optimized Remote Procedure Calls. with Facebook. MLSys 2023

[2] PipeTransformer: Automated Elastic Pipelining for Distributed Training of Transformers. ICML 2021. with Facebook. https://arxiv.org/abs/2102.03161

https://pytorch.org/blog/pipetransformer-automated-elastic-pipelining

PyTorch Distributed System is now the mainstream backbone of LLM training and serving. I am proud that I have contributed to history.

AWS AI, Research Scientist, 2021-2022

Did research at AWS AI on large-scale distributed training for foundation models (Mixture-of-Experts)

[1] SMILE: Scaling Mixture-of-Experts with Efficient Bi-level Routing. https://arxiv.org/abs/2212.05191

Google, Research Scientist, 2019-2020

Collaborating with Google for Federated Learning Research:
[1] Advances and Open Problems in Federated Learning. JnTML 2021.
https://arxiv.org/abs/1912.04977
[2] A Field Guide to Federated Optimization. https://arxiv.org/abs/2107.06917

Tencent AI Lab (ai.tencent.com), November 2018 – March 2019

Machine Learning Center, Shenzhen, China
Research Areas: Large-scale Distributed Machine Learning, AutoML (Neural Architecture Search), Graph Neural Networks

Tencent (www.tencent.com), November 2014 – July 2018

Tencent Cloud, Social Network Group; Tencent Maps, Mobile Internet Group, Shenzhen, China
Principal Software Engineer (T3.2-T3.3, the same level as T7 in Google); Engineering Manager (L1-2), led a SDE team including 15+ software engineers
Research Areas: Distributed System, Networking System, Cloud Computing, Applied Machine Learning / Deep Learning, and Mobile Computing.

Baidu (www.baidu.com), July 2012 – November 2014

Baidu Maps Location-Based Service, Beijing, China
Senior Software Engineer (T5); Team Leader, led 11 software engineers
Research Areas: Operating System, Cloud Computing, Distributed System, Computer Graphics

Huawei (www.huawei.com), June 2011 – June 2012

Huawei Consumer Business Group, Shenzhen, China
Software Engineer for Huawei Smartphone OS

Production

I was a full-stack software engineer and team manager. I have lots of experience in successful Internet product publication. My experience spans nearly all Internet-related software systems development, including cloud computing, distributed systems, machine learning systems, applied machine learning, operating system, and mobile applications. Business domain: cloud computing, automotive intelligent system, autonomous driving, smartphone, Maps/LBS, games, etc

  • Tencent Cloud. 2017-2018
    Worked on business-oriented cloud computing solution; led a team to develop the 1st generation cloud computing solution for the automotive industry.
  • Tencent Venus Distributed Machine Learning System. 2016-2017
    Worked with Shengping Zhou (now CTO@AlphaCloud), developing the large-scale distributed machine learning system for MIG, Tencent. This platform supports various machine learning algorithms and models, including, LR, SVM, GBDT, DNN, etc.
  • Tencent Location Data Computing Platform. 2016-2017
    worked on streaming data mining systems and algorithms; this platform contains billion-level daily users; I learned the entire system architecture and related real-time data analytics algorithms.
  • Speech Recognition and Natural Language Processing for Tencent Automotive Operating System. 2015-2016
    This project inspired me to explore ML models, algorithms, and systems; deep learning is very fancy that year; we developed a deep learning model compression engine for our product (at that time, TensorFlow Lite is not released yet)
  • Tencent Games: a Pokemon Go-like Mobile Game Engine 2016
    worked on Unity3D and C++ -based game engine; this is a cross-group collaboration, very fun; I learned a lot of engineering and production culture at Tencent Game.
  • Tencent AI in Car (also called Tencent Automotive Service). 2014-2016
    worked on the embedded operating system (Android/QNX) and related back-end services, including more than 10 applications and system core service, such as WeChat MyCar, Tencent Maps, Tencent Music, Tencent Video, etc.
  • As a Team Leader, I Developed the 1st Generation of Smartphone Navigation SDK, Application, and Service in China (Baidu Navigation SDK). 2012-2013
    Baidu Navigation SDK, Application, and Service is a cross-platform (C++/Java/Objective-C) engine that requires computer graphics (OpenGL ES), routing algorithms (A*, CH), offline and online data compiler and networking service (Hadoop; C++/PHP backend), complex navigation UI message state control and dispatch, cross-platform, layered, and modularized complex system design. I am familiar with all of these core modules.
  • As a Software Engineer, I Contributed to the First Generation of Huawei Smartphone. 2011-2012
    The picture below shows Huawei Ascend P6 (Emotion UI)