I downloaded the llamada2 to my local disk, but i still cannot run your code, The error message is: "TypeError: Object of type LlamaTokenizerFast is not JSON serializable."

[zob109]

There is my issue:
/home/fountain/miniconda3/bin/conda run -n Linear_Alignment --no-capture-output python /home/fountain/pycharmProjects/Linear_Alignment/demo.py
Loading checkpoint shards: 100%|██████████████████| 2/2 [00:17<00:00, 8.77s/it]
Traceback (most recent call last):
File "/home/fountain/pycharmProjects/Linear_Alignment/demo.py", line 134, in
model = ConstractiveDecodingModel.from_pretrained(
File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/site-packages/transformers/modeling_utils.py", line 3510, in from_pretrained
model.generation_config = GenerationConfig.from_pretrained(
File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/site-packages/transformers/generation/configuration_utils.py", line 753, in from_pretrained
config = cls.from_dict(config_dict, **kwargs)
File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/site-packages/transformers/generation/configuration_utils.py", line 791, in from_dict
logger.info(f"Generate config {config}")
File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/site-packages/transformers/generation/configuration_utils.py", line 350, in repr
return f"{self.class.name} {self.to_json_string(ignore_metadata=True)}"
File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/site-packages/transformers/generation/configuration_utils.py", line 876, in to_json_string
return json.dumps(config_dict, indent=2, sort_keys=True) + "\n"
File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/json/init.py", line 234, in dumps
return cls(
File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/json/encoder.py", line 201, in encode
chunks = list(chunks)
File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/json/encoder.py", line 431, in _iterencode
yield from _iterencode_dict(o, _current_indent_level)
File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/json/encoder.py", line 405, in _iterencode_dict
yield from chunks
File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/json/encoder.py", line 438, in _iterencode
o = _default(o)
File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.class.name} '
TypeError: Object of type LlamaTokenizerFast is not JSON serializable
ERROR conda.cli.main_run:execute(125): conda run python /home/fountain/pycharmProjects/Linear_Alignment/demo.py failed. (See above for error)
How to solve this question?

[zob109]

my enviroment are follows:
Ubuntu 22.04.3, Python 3.9.19

[sod1010]

Have you solved this problem already? I encountered the same problem.

[Linzy-Liu]

I've solved the problem. The problem is caused by the extra parameter tokenzier in model initialization. The tokenizer will be passed to the kwargs param in from_pretained method and finally reach the json.dump method, where the error occurs.

The solution to the problem is to remove the tokenizer param in initialization ,and add a custom from_pretrained method to seperate the intialization of Llama and tokenizer. Here is my solution:

class ConstractiveDecodingModel(LlamaForCausalLM):
    _tied_weights_keys = ["lm_head.weight"]

    def __init__(self, config):
        super().__init__(config)
        self.tokenizer = None  # Initialize tokenizer separately

    @classmethod
    def from_pretrained(cls, pretrained_model_name_or_path, tokenizer=None, **kwargs):
        # Load the model without leaving the tokenizer in kwargs
        model = super().from_pretrained(pretrained_model_name_or_path, **kwargs)

        if tokenizer is not None:  # Attach tokenizer separately
            model.tokenizer = tokenizer
        return model

[sod1010]

Thank you so much~ I have solved this BUG through your method!