I can run the 30B models in system RAM using llama.cpp/ooba, but I do need to compile my own llama.cpp with the right settings. Also – you would want to take into account that llama.cpp can fit parts of a model into the GPU depending on how much VRAM you have. For me that’s what really gets the fastest speeds, even on my 5700XT. Great start though!
Day: March 14, 2024
Categories
Memes Feel. Soyjak Template
” 🇺🇸🇬🇧🇪🇺⚡️🇷🇺 O director da CIA, William Burns, disse que a assistência dos EUA à Ucrânia (sujeita a apropriação pelo Congresso) deve garantir que Kiev tome a iniciativa no campo de batalha no início de 2025 e avance para uma nova contra-ofensiva.
🧵🧵🧵 pic.twitter.com/42xzqAozB2
— Abu Obeida 🇵🇸 (@rfn_otal) March 14, 2024
Driving Julia to school today, I put New Order on Spotify (again) and she said: “Daddy, of course.” I swear, she will outdo Tony Wilson before she’s ten. Love this kid.
— Michael Weiss (@michaeldweiss) March 14, 2024