class=”markdown_views prism-atom-one-light”>
The development of mobile Internet has brought great changes to people’s social and entertainment methods. Emerging cultural forms represented by vlogs, short videos, etc. are being favored by more and more people. At the same time, with the application of AI intelligence, beauty retouching and other functions in image and video editing apps, video editing efficiency and video effects have been greatly improved, and video application scenarios have become more abundant.
Current editing products have diverse functions and rich materials, but the development cycle is long and the threshold is high. In order to make the editing software more intelligent, easier to use and improve the efficiency of developers, HMS Core 6 provides developers with a video editing service (Video Editor Kit), providing one-stop video import, editing, rendering, export, media asset management, etc. Video processing capabilities. In addition to supporting complete traditional video editing functions, the video editing service also provides rich AI processing capabilities such as exclusive filters, character tracking, and one-click hair dyeing to assist video creation, bringing users more creative inspiration and creating more intelligent videos. Editing experience.
Figure 1. Display of exclusive filters, character tracking, and one-click hair dyeing effects based on AI capabilities
Diversified intelligent video processing capabilities are realized by neural network models. Since the trained model files are large (the size of a single model is generally more than ten or even tens of megabytes), the ROM and RAM space of mobile phones and other devices Due to its limited size, how to provide developers with richer intelligent video processing capabilities while occupying less space on the terminal device has become a major challenge for mobile application video editors.
To solve the above challenges, HMS Core video editing service chose to use Huawei’s self-developed AI framework MindSpore Lite for neural network model inference. MindSpore Lite is a full-scenario AI inference engine that supports rapid deployment in different environments on the device, edge, and cloud through a unified API interface. It supports HarmonyOS, Android, iOS, Windows and other operating systems, and supports Ascend, GPU, CPU (x86 , arm…) and other hardware execution. In addition to supporting model formats trained by MindSpore, MindSpore Lite also supports conversion and inference of third-party model formats such as TensorFlow, TensorFlow Lite, Caffe, and ONNX.
Figure 2. MindSpore Lite architecture diagram
MindSpore Lite provides a high-performance and ultra-lightweight solution for AI model inference: through efficient kernel algorithms and assembly-level optimization, as well as heterogeneous scheduling of CPU, GPU, and NPU, it can fully utilize the hardware computing power and achieve minimum Reduce inference latency and power consumption; provide model quantization compression technology and use post-training quantization (PTQ) to directly map weight data from floating point to low-bit fixed-point data without a data set, effectively reducing Model size facilitates the deployment and execution of AI models in resource-constrained environments.
Figure 3. Introduction to the principles of quantification technology
Quantization of weight data supports two forms: fixed bit quantization and mixed bit quantization. Fixed bit quantization adopts the Bit-Packing method, supporting weight quantization of any bits from 1 to 16 to meet user requirements in different compression scenarios. At the same time, according to the data distribution after model quantization, the appropriate encoding strategy is automatically selected for compression encoding. To achieve the best compression effect.
Figure 4. Fixed bit quantization compression
Hybrid bit quantization adopts the mean square error as the optimization target based on the different sensitivities of different layers of the neural network to quantization losses, and automatically searches for the bits most suitable for the current layer, achieving greater compression while ensuring accuracy. Rate. At the same time, for the quantized model, Finite State Entropy (FSE) is used to entropy encode the quantized weight data to further compress it, achieving efficient compression of the model, improving the model transmission rate and reducing the model storage space.
Figure 5. Mixed bit quantization compression
In addition, Bias Correction will be used during quantization to minimize the quantization error. Bias Correction will calibrate the weight data during inverse quantization based on the inherent statistical characteristics, so that the weight value has the same expectation and variance before and after quantification, which can greatly improve the accuracy of the model.
The AI model in the video editing service uses the mixed bit quantization method provided by MindSpore Lite, which ultimately achieves an average model compression effect of 5x+ while ensuring accuracy. For example, the one-click hair dye model is compressed from the original 20.86M to 3.76M. , effectively solving the deployment difficulties caused by too many models and too large files.
Figure 6. Quantitative effect of video editing model (derived from MindSpore Lite measured data)
Through quantitative compression of the AI model, it is guaranteed to be cut while keeping the ROM space occupied.��The product can deploy more AI models and give full play to AI capabilities to provide more special application scenarios, making the editing function more powerful and smarter. After Huawei’s official editing software Petal Clip has access to video editing service capabilities, users can make video editing more convenient and richer by using AI video editing functions such as exclusive filters and person tracking (some features will be released with the upgrade of the Petal Clip app). Fun.
MindSpore Lite is committed to building a high-performance, ultra-lightweight, full-scenario AI engine. In addition to high-performance kernel algorithms and hardware heterogeneous scheduling and quantitative compression, it also provides one-stop training and inference capabilities for device-cloud collaboration. . HMS Core video editing service is based on MindSpore Lite, helping developers create more easy-to-use and intelligent editing tools.
For more information, please visit the official website
Huawei Developer Alliance HMS Core official website
MindSpore Open Source Community
微信扫一扫打赏
支付宝扫一扫打赏
