UC Berkeley needed to make generative LLM inference more efficient through low-precision quantization, but its SqueezeLLM approach depended on custom CUDA kernels that took weeks to implement and optimize and would be costly to port manually to Intel GPU hardware.
Reported impact
The migration gave UC Berkeley cross-platform support for its quantized LLM inference kernels with minimal additional engineering work and delivered immediate performance gains on Intel GPUs without manual tuning.
University of California Berkeley used SqueezeLLM to run generative LLM inference with 4-bit quantization, then ported the custom CUDA kernels to SYCL so the same low-precision inference approach could run on Intel hardware. The team used the Intel oneAPI Base Toolkit to run the SYCLomatic tool, which automatically converted CUDA kernel code into SYCL-compatible implementations with the required kernel definition and invocation changes. To preserve the existing inference flow, the Python bindings were adapted from a CUDAExtension build to the PyTorch XPU CPP extension (DPCPPExtension), letting the migrated kernels be installed with setup.py and called from the existing SqueezeLLM Python code with only minor changes. The converted kernels were then benchmarked on Intel Data Center GPU Max Series systems accessed through Intel Tiber Developer Cloud, where quantized matrix-vector operations for the Llama 7B model delivered immediate performance gains without manual tuning.
Existing SqueezeLLM Python code sends quantized inference work through DPCPPExtension bindings into SYCL-converted custom kernels built with Intel oneAPI Base Toolkit and migrated by SYCLomatic. Those kernels run matrix-vector operations on Intel Data Center GPU Max Series systems in Intel Tiber Developer Cloud and return results to the PyTorch-based inference flow for Llama 7B token generation.
"After installing the kernel bindings, the converted SYCL kernels could be called from PyTorch code, thereby enabling running end-to-end inference using the converted kernels."
"Researchers at UC Berkeley have developed SqueezeLLM as a solution to enable efficient and accurate low-precision quantization."
"The SqueezeLLM team used Intel Data Center GPUs made available through the Intel® Tiber™ Developer Cloud to test and benchmark the ported kernel implementations."
"To call the kernel from the Python code, the bindings were adapted to use the PyTorch* XPU CPP extension (DPCPPExtension)."
"Using the CUDA-to-SYCL code migration with SYCLomatic, the efficient kernel methods can be decoupled from the target deployment platform, thereby enabling inference on different target architectures with minimal additional engineering effort."
"We quickly attained a 2.0x speedup on Intel® Data Center GPUs with 4-bit quantization without any manual tuning."
"We, therefore, wanted a solution to be able to quickly and easily port our custom CUDA kernels to SYCL in order to target Intel Data Center GPUs."
Review how this reference was scored and trace each normalized claim back to its documented source.
Scoring methodology
Customer Reference Value measures usefulness as a customer reference. Evidence Confidence measures the quality and traceability of available evidence, not independent verification.
Traceability
Field coverage shows which facts each source supports. Dates are shown only when documented by the source.
Related
Intel Sample Dec 18, 2024 UC Davis UC Davis researchers needed a practical way to create convincing materials for scientific datasets under multiple lighting conditions, while also overcoming the long runtimes and memory constraints involved in training… 6 times faster training and inference speeds compared to using CPU only ControlNet · Intel Data Center GPU Max Series · Intel Extension for PyTorch 
Microsoft Sample Oct 15, 2026 Bar-Ilan University Students who miss lectures struggle to reconnect new lessons with prior material when course information is spread across notes, recordings, slides, and readings, and lecturers need a reliable way to answer routine… 71 courses Azure AI Services · Microsoft Azure · Microsoft Copilot Studio 
Google Cloud Sample Modified · Oct 9, 2026 Mindvalley Mindvalley needed to scale personalized AI learning companions for a large active-user base while keeping latency, uptime, and compute costs under control across customer-facing conversational workloads. 1.3 million annual agent requests BigQuery · Cloud Run · Cloud SQL 