question about format and running in comfy
so i noticed you were playing around with different formats and i decided to try my hand at it as well. i was able to quant a w3a8 but am unable to test it due to some sort of errors preventing offloading and the fact the models arent recognized. i was wondering if this update to comfy kitchen was strictly for w4a8 and what is required to actually run it as the diffusion model loader doesnt house the models (either one) and the custom node i created for it just OOMs or fails to errors regarding actually reading the file itself as if the format isnt recognized.
am i missing something simple here and was overthinking all night last night?
https://huggingface.co/realrebelai/MiniMax-H3_Experimental/tree/main
It would need kernel implementations, and did you check what kind of relative weight error you can get with 3bit? Because 4bit is already near the edge of usability especially for video models at least.
It would need kernel implementations, and did you check what kind of relative weight error you can get with 3bit? Because 4bit is already near the edge of usability especially for video models at least.
that was kind of what i was attempting to test with comfy as im not versed on this matter in any way other than what ive quanted previously with claude so my knowledge lacks heavily when it comes to reading what im looking at. pitfalls of not going to school i guess. but my hope was that the configuration of the file with the nodes pushing the format would help figure out if it was viable