Setting pad_token_id to eos_token_id:151645 for open-end generation.

[yuecao0119]

Hello, I have a question:
When executing generate.sh, it will prompt:
Setting pad_token_id to eos_token_id:151645 for open-end generation.
Will this affect the model performance?

[zwcolin]

For which model do you see this error? For most models we adapt the code from their official demo snippet to implement the response generation part. If it happens due to a model-specific setting, you should reach out to the authors that developed the model.

[yuecao0119]

OK, thanks.

…

---- Replied Message ---- | From | Zirui ***@***.***> | | Date | 11/25/2024 14:56 | | To | ***@***.***> | | Cc | Yue ***@***.***>***@***.***> | | Subject | Re: [princeton-nlp/CharXiv] Setting `pad_token_id` to `eos_token_id`:151645 for open-end generation. (Issue #11) | For which model do you see this error? For most models we adapt the code from their official demo snippet to implement the response generation part. If it happens due to a model-specific setting, you should reach out to the authors that developed the model. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>