Loading…
28
I finally got a small AI model running on my 2016 laptop last week and it actually works
Last Tuesday I installed a tiny language model on my old ThinkPad in my apartment in Portland, mostly as a joke since the thing has 8GB of RAM. I expected it to choke, but it loaded in about 40 seconds and answered simple prompts at maybe 5 words per second. The trick was picking a model small enough to fit in a few gigabytes and using a tool that runs it on the CPU. It's slow and not that smart, but the fact that my beat up laptop from 2016 can run this at all feels wild to me. Has anyone tried a tiny model on old hardware and found a sweet spot for size?
0 comments
Log in to join the discussion
Log In0 Comments
No comments yet
Be the first to share your thoughts on this discussion.