Skip to content

why pre-processed 4 bit model on huggingface is larger than normal 4 bit model ? and what about Qwen2.5,only saw Qwen2 #30

@chuangzhidan

Description

@chuangzhidan

couple of things i am wondering:
1.it is generally 4G larger in terms of disk usage for a 72b-sized model,even without considering pissa init folder size. nou sure why

2.i can just funtune this model directly on a pre-processed 4 bit model and saved chekpoint will also be a 4 bit model ,yes?

3.last thing ,do u have pre-processed Qwen2.5 series models ? only saw Qwen2 on huggingface,not sure how much GPU i need to process a large 72b sized model

thanks ,for u attention on this matter

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions