Quark for Ryzen AI NPU

Here you will find references on how you can leverage Quark to seamlessly run quantized models on the Ryzen AI NPU. Ryzen AI leverages ONNX models to represent models and execute them through ONNX Runtime.

To help you get started, we also have examples at the ONNX Examples page!

Resources

• Quick Start for Ryzen AI
• Best Practice for Ryzen AI in AMD Quark ONNX
• Auto-Search for Ryzen AI ONNX Model Quantization
• Quantizing LLMs for ONNX Runtime GenAI
• FP32/FP16 to BF16 Model Conversion
• Power-of-Two Scales (Xint8) Quantization
• Float Scales (A8W8 and A16W8) Quantization

Quark also delivers a plethora of post-processing tools that might be of use for Ryzen AI. refer to the ONNX Tools to learn more!