Accessing ONNX Examples

Users can get the example code after downloading and unzipping amd_quark.zip (referring to Installation Guide). The example folder is in amd_quark.zip.

Directory Structure of the ZIP File:

+ amd_quark.zip
   + amd_quark.whl
   + examples    # HERE IS THE EXAMPLES
      + torch
         + language_modeling
         + diffusers
         + ...
      + onnx # HERE ARE THE ONNX EXAMPLES
         + image_classification
         + language_models
         + ...
   + ...

ONNX Examples in Quark for This Release

Improving Model Accuracy

• Block Floating Point (BFP)
• MX Formats
• Fast Finetune AdaRound
• Fast Finetune AdaQuant
• Cross-Layer Equalization (CLE)
• GPTQ
• Mixed Precision
• Smooth Quant

Dynamic Quantization

• Quantizing an Llama-2-7b Model
• Quantizing an OPT-125M Model

Image Classification

• Quantizing a ResNet50-v1-12 Model

Language Models

• Quantizing an OPT-125M Model

Weights-Only Quantization

• Quantizing an Llama-2-7b Model Using the ONNX MatMulNBits
• Quantizating Llama-2-7b model using MatMulNBits