A Reddit user reports running the OpenAI 120B model locally on a gaming PC, achieving 17.83 tokens/sec but consuming nearly all available system RAM. The experiment was motivated by concerns over cloud AI dependency during potential internet or power outages caused by major disruptions like war. The user highlights the significant local hardware requirements, particularly memory capacity, for running large language models offline.
Read original
reddit/r/LocalLLM