Bolt Graphics GPU will have 2 DDR5 laptop DIMM slots
They have a few working prototypes, & are aiming for pre-production examples made by end of this year, & full production by Christmas 2027. Interesting specs:…
Latest
Anyone else end up building a web access layer for local AI agents?
I've been running local models for most of my experiments, and I kept running into the same issue. The model lives locally, but everything it needs…
Mellum2 local deployments
Hey local community, I work at JetBrains with the team that trained Mellum2 models — 12B-2.5A LLMs. Those models are trained completely from scratch, targeting fast…
NASA testing local LLM inference for future space missions
Red Hat published a blog post last week about an initiative I supported with NASA researchers at Johnson Space Center building a medical AI assistant. It's…
Setup an H200 NVL on consumer(ish) hardware
https://preview.redd.it/v1lqvebe18ah1.jpg?width=1280&format=pjpg&auto=webp&s=65d8b41e9ad106d94506ef45a44c0c71ed2543fd Setup : MB ASUS WRX90E - SAGE SE 64 Core Threadripper processor H200NVL It required a bunch of BIOS config, especially PCIe BUS config CSM…
Kimi and GLM on frontier code
submitted by /u/Charuru [link] [comments]
CPU-only GLM 5.2: Epyc and 512GB RAM
This is just a preview of some content I'm putting together to share with you all. I have a server I've put together and I'm testing…
Any good uses for a 192 GB DDR3 Server in the LLM world?
I've been gifted this old IBM System X V4 with a dual Xeon E5-2640 [6c12t @ 2.7 GHz] and a whooping 192 GB of DDR3 1666…
Is it ever possible to have a malicious LLM with a backdoor
I was just brainstorming of possibilities that the LLMs behave differently than normal if trained to recognize a specific secret sentence, and then unlocks a backdoor…
How can I get better response time by caching my system prompt?
Hi, I've spent some time trying to find a solution to make my local AI cache the system prompt (unless it is already caching and hitting…