• 0 Posts
  • 76 Comments
Joined 1 year ago
cake
Cake day: June 12th, 2023

help-circle


















  • I can run 7B models on my laptop with its embedded GPU. Running on a phone or a Pi is possible with smaller models, but very slow. Expect good speed with a desktop Nvidea GPU. Later this year, there should be new computers with an NPU integrated to the CPU which should speed up computers that don’t have a dedicated GPU. (But a GPU will still outperform them by a lot.)

    70B models will run very slowly on even the best consumer hardware due to memory limitations.


  • I do a lot of code. That means I often deal with three or four programs at the same time, and perhaps 10 loaded throughout the day and I want to see them all. So I have two monitors that are each 27" and 4k.

    This means I can see a web browser sized to a full 1080 size, next to a database query, and still see the code that I’m working on, and keep an eye on any new emails or text chats. Without needing to Alt-tab to switch windows. It’s like spreading your work over a dining room table, instead of those little desks you got in high school.

    Most apps don’t need to be larger than 1080. But some can be taller to see more code (maybe 160 lines, for example) without scrolling too much. And I hardly ever deal with just one window at a time.