Comment Re:It's a webpage frontend (Score 1) 64
I have both the 8 billion token and the 70 billion token models running on a multi Xeon machine with "only" 128 GB RAM and an M.2 SSD, the 8B version runs acceptably good and uses about 20 GB in memory. The 70B version uses about 75 GB RAM and bogs horribly (A word every few seconds) but does work. For the full version, yeah I'd need a ridiculous amount of RAM that I don't have.