quark.torch.export.gguf_export.api#

Module Contents#

Functions#

quark.torch.export.gguf_export.api.convert_exported_model_to_gguf(model_name: str, json_path: Union[str, pathlib.Path], safetensor_path: Union[str, pathlib.Path], tokenizer_dir: Union[str, pathlib.Path], output_file_path: Union[str, pathlib.Path]) → None#

This function is used to convert quark exported model to gguf model.

Parameters:

  • model_name (str) – name of this model which will be written to gguf field general.name

  • json_path (Union[str, Path]) – Quark exported model consists of a .json file and a .safetensors file. This arguments indicates the path of .json file

  • safetensor_path (Union[str, Path]) – Path of .safetensors file.

  • tokenizer_dir (Union[str, Path]) – Tokenizer needs to be encoded into gguf model. This argument specifies the directory path of tokenizer which contains tokenizer.json, tokenizer_config.json and/or tokenizer.model

  • output_file_path (str) – The path of generated gguf model.