Local LLMs are about to change big time (probably for the better)
Wow, the new Mac Studios look really impressive. I can’t find the post that I saw last night about it, so I can’t share that author’s thoughts (I thought I commented or reacted at least, apparently not…), but what was interesting is the M3 Ultra goes up to 512GB of unified memory. 512GB… Of unified memory… If you’ve been playing with LLMs locally, and you have Apple Silicon, you’ll know that Apple’s unified memory is pretty performant, and it’s great for running local LLMs because the GPU uses the same system memory as the CPU, which means it can access most if not all of the RAM. ...