This is a heavily interactive web application, and JavaScript is required. Simple HTML interfaces are possible, but that is not what this is.
Post
Tommi Somersuo
tommis.fi
did:plc:jl5s7l7e4b4ycdkwig3cas75
LLM in a flash: Efficient Large Language Model Inference with Limited Memory - By optimizing data transfer and storage with flash memory, the method runs models twice the size of available DRAM, achieving 4-5x faster inference on CPUs and 20-25x on GPUs
https://arxiv.org/abs/2312.11514
2023-12-20T07:19:40.573Z