POPA ReconFor cost-effectiveness sections of less than KRW 2 million, about 30b quantizes 5 to 4 bits with MoE model 16gb or more vram
For cost-effectiveness sections of less than KRW 2 million, about 30b quantizes 5 to 4 bits with MoE model 16gb or more vram There is also a method of offloading some layers and using no censorship, but the performance is about 4.5 on SONET? I think it comes out (77 token speed based on rtx 5070ti)
The excerpt is quoted from a public Telegram channel, which owns its text.