Janas: Run LLMs larger than RAM by streaming experts from SSD
github.comkhalidkhair
Janas is an open-source project that enables running large language models exceeding available RAM by streaming model experts from SSD storage. This approach allows users to work with larger models than their system memory would normally permit.
Why it mattersReaders can run more capable LLMs on consumer hardware without expensive RAM upgrades, reducing barriers to experimentation with larger models.
Read at github.com