Meituan

Solution

Meituan reworked its vision AI inference environment into a microservices-based algorithm process that separates CPU services from other accelerator services and uses a middle-layer scheduling service to place workloads on the most suitable hardware. For low-traffic, long-tail inference workloads, Meituan runs models on 4th Gen Intel Xeon Scalable processors to reuse existing CPU resources and reduce dependence on higher-cost discrete GPUs. It then applies Intel Advanced Matrix Extensions (Intel AMX), Intel Extension for PyTorch (Intel IPEX), and Intel Integrated Performance Primitives (Intel IPP) to optimize deep-learning execution, including converting multiple vision models from FP32 to BF16 to raise throughput with limited precision loss. Meituan also separates latency-sensitive online traffic from offline batch jobs and uses dynamic scaling to reallocate off-peak capacity, improving utilization and serving a range of vision AI applications such as image processing, certificate identification, QR code bike locks, package scanning, and identity verification.

Data flow

Visual data from Meituan business scenarios enters a microservices-based vision AI process where a middle-layer scheduling service routes jobs to CPU services or other accelerator services. Low-traffic long-tail models are executed on Intel Xeon CPU infrastructure, optimized with Intel AMX, Intel IPEX, BF16 conversion, and dynamic scaling. Results are then used by downstream vision applications such as image processing, certificate recognition, QR code bike locks, package scanning, and identity verification.