Installation Guide

Installation Guide#

Prerequisites#

  1. Python 3.9, 3.10, 3.11 or 3.12 is required. Python 3.13. is currently not supported by Quark’s dependencies.

  2. Install PyTorch for your compute platform (such as CUDA, ROCm, and CPU).

    • Only select PyTorch version 2.2.0 or later.

  3. Quark has the following dependencies, which are installed automatically with pip and wheel installs of Quark.
  4. Install a C++ compiler.

    • On Ubuntu the g++ compiler is installed with the build-essential package. Note that whilst many versions of C++ compiler may work, we currently only confirm support for g++ version 13.3, which installs with the build-essential package on Ubuntu 24.04.

    • When installing natively on Microsoft Windows, Visual Studio is necessary, with Visual Studio 2022 being the minimum required version. During the compilation process, you can choose one of the following methods:

      1. Use the Developer Command Prompt for Visual Studio. When installing Visual Studio, ensure that you include the Developer Command Prompt. You can access it through Visual Studio by clicking Tools > Command Line > Developer Command Line. You can execute the programs in the CMD window of the Developer Command Prompt for Visual Studio.

      2. Manually add paths to the environment variables. The tools cl.exe, MSBuild.exe, and link.exe from Visual Studio are used. Ensure that their paths are included in the PATH environment variable. These programs can be found in the Visual Studio installation directory. In the Edit Environment Variables window, click New, and then paste the path to the folder containing the cl.exe, link.exe, and MSBuild.exe files. Click OK on all of the windows to apply the changes.

Installation#

Note

The AMD Quark package distribution name has been renamed to amd-quark. Please use the new package name for releases newer than 0.6.0.

We recommend new users install Quark from PyPI with pip. It’s also possible to install from a ZIP download.

Install from PyPI with Pip#

Releases of AMD Quark are available on PyPI https://pypi.org/project/amd-quark/, and can be installed with pip:

pip install amd-quark

Nightly builds are not yet available on PyPI.

Install from ZIP Download#

Step 1: Download and unzip 📥*amd_quark-*.zip* which has a wheel package in it. You can download wheel package 📥*amd_quark-*.whl* directly.

📥amd_quark.zip release_version (recommend)

📥amd_quark.whl release_version

Directory Structure of the zip file:

+ amd_quark.zip
   + amd_quark.whl
   + examples    # Examples code of Quark
   + docs        # Off-line documentation of Quark.
   + README.md

We strongly recommend downloading the zip file, as it includes examples compatible with the wheel package version.

Step 2: Install the quark wheel package by running the following command:

pip install amd_quark*.whl

Installation Verification#

  1. (Optional) Verify the installation by running python -c "import quark". If no error is reported, the installation is successful.

  2. (Optional) Compile the fast quantization kernels. When using Quark’s quantization APIs for the first time, it compiles the fast quantization kernels using your installed Torch and CUDA, if available. This process might take a few minutes, but the subsequent quantization calls are much faster. To invoke this compilation now and check if it is successful, run the following command:

    python -c "import quark.torch.kernel"

    Note

    For Quark with GPU on windows, if the kernel cannot be loaded successfully, please follow the steps below to troubleshoot

    • Ensure c++ compiler is install.

    • Ensure that torch.cuda.is_available() returns True.

    • Verify that nvcc or hipcc can be invoked from the command line.

    • If the compilation is successful but Python fails to load the DLL, locate the Quark build directory (the build path will be printed in the log) and check the dependencies using:

      dumpbin /DEPENDENTS kernel_ext.pyd

    • A common issue is that the required DLL is not included in the system path or the version of the dependent DLL is incorrect.

  3. (Optional) Compile the custom operators library. When using Quark-ONNX’s custom operators for the first time, it compiles the custom operators library using your local environment. To invoke this compilation now and check if it is successful, run the following command:

    python -c "import quark.onnx.operators.custom_ops"

Older Versions#

Note: The following links are for older versions of AMD Quark, before the package distribution name was renamed to amd-quark.

Contents