quark.torch.export.gguf_export.api

Module Contents

Functions

quark.torch.export.gguf_export.api.convert_exported_model_to_gguf(model_name: str, json_path: str | pathlib.Path, safetensor_path: str | pathlib.Path, tokenizer_dir: str | pathlib.Path, output_file_path: str | pathlib.Path) → None

This function is used to convert quark exported model to gguf model.

Args:

model_name (str): name of this model which will be written to gguf field general.name json_path (Union[str, Path]): Quark exported model consists of a .json file and a .safetensors file.

This arguments indicates the path of .json file

safetensor_path (Union[str, Path]): Path of .safetensors file. tokenizer_dir (Union[str, Path]): Tokenizer needs to be encoded into gguf model.

This argument specifies the directory path of tokenizer which contains tokenizer.json, tokenizer_config.json and/or tokenizer.model

output_file_path (str): The path of generated gguf model.