The Transformer architecture has improved the performance of deep learning models in domains such as Computer Vision and Natural Language Processing. Together with better performance come larger model sizes. This imposes challenges to the memory wall of the current accelerator hardware such as GPU. It is never ideal to train large models such as Vision Transformer, BERT, LLaMA, and GPT on a single GPU or a single machine. There is an urgent demand to train models in a distributed environment. However, distributed training, especially model parallelism, often requires domain expertise in computer systems and architecture. It remains a challenge for AI researchers to implement complex distributed training solutions for their models. To solve this problem, we introduce a unified parallel training system designed to seamlessly integrate different paradigms of parallelization techniques including data parallelism, pipeline parallelism, multiple tensor parallelism, and sequence parallelism. Our system aims to support the AI community to write distributed models in the same way as how they write models normally. This allows them to focus on developing the model architecture and separates the concerns of distributed training from the development process. Our system is able to achieve 2x speedup over state-of-the-art distributed systems for GPT model training.

Mobile and embedded computing devices have become key carriers of deep learning to facilitate the widespread use of machine intelligence. However, there is a widely recognized challenge to achieve real-time DNN inference on edge devices, due to the limited computation/storage resources on such devices. Model compression of DNNs, including weight pruning and weight quantization, has been investigated to overcome this challenge. However, current work on DNN compression suffers from the limitation that accuracy and hardware performance are somewhat conflicting goals that are difficult to satisfy simultaneously. We present our recent work Compression-Compilation Codesign, to overcome this limitation towards the best possible DNN acceleration on edge devices. The neural network model is optimized in a hand-in-hand manner with compiler-level code generation, achieving the best possible hardware performance while maintaining zero accuracy loss, which is beyond the capability of prior work. We are able to achieve real-time on-device execution of a number of DNN tasks, including object detection, pose estimation, activity detection, speech recognition, just using an off-the-shelf mobile device, with up to 180 X speedup compared with prior work. Recently, for the first time, we enable large-scale language and AIGC models such as GPT and Stable Diffusion on mobile devices. Last we will introduce our recent breakthrough of superconducting logic based neural network acceleration that achieves 10^6 times energy efficiency gain compared with state-of-the-art solutions, achieving the quantum limit in computing.

ChatGPT has sparked a new wave of AI, the large models move from the technology breakthrough stage to the industry enablement stage. With the low-cost and available pre-trained large models, a large number of small and medium-sized data centers can fine-tune models and deploy applications for specific industry scenarios. The "tight coupling" of high-performance CPU and xPU is the king to obtain high performance for AI optimized Data center. Ysemi CPU is based on Arm v9 architect, which supports enhanced features on AI, such as SVE2, BF16 MM, INT8 MM etc. Based on Kubernetes scheduling and task management platform, Ysemi optimizes open-source software stack for mainstream AI frameworks, including TensorFlow, Pytorch, oneDNN, and OpenBLAS, on the basis of Arm ACL library. Under the support from heterogeneous computing configurations, CPU+xPU provides cost-effective solution for the AI training and inference.

In this workshop, we will delve into how to fine-tune large models. First, we will provide a brief introduction to the concept of large models, including well-known models like GPT-4 and LLaMA, and explain why there's a need to fine-tune. Subsequently, we will introduce the basic concepts and terminologies of large models, explain the differences between Pre-training and Fine-tuning, and the concept of Transfer Learning. We will discuss in detail the steps and processes of fine-tuning, from data preparation and choosing a pre-trained model to the actual fine-tuning process. Next, we will demonstrate the actual operation of fine-tuning through a real-life case study and analyze the results and outcomes. Lastly, we will discuss some common challenges, such as overfitting, data imbalance, model selection, and resource constraints.

The rise of LLMs has brought new opportunities for the digital transformation of the legal industry. We will delve deep into how large language model technology can be applied in the legal field to address real-world user needs. First, we will provide a brief overview of the legal industry's scenarios and requirements and analyze how LLMs can create value for users. Then, we will explain the mechanisms and design philosophy behind the product, including how it's trained and optimized using a vast amount of legal textual data, and how it conducts deep reasoning on complex judicial logic. Finally, through a real product example, we will showcase and discuss the productization application of LLMs in niche domains