Key points
- The article says ExLLaMa has arrived and brings a significant improvement in the speed of Open Chat, which is now ten times faster.
- It notes a remaining context limitation of 4k tokens, making the setup unsuitable for handling excessively lengthy queries.
ExLLaMa has arrived, bringing with it a significant improvement in the speed of Open Chat, which is now ten times faster. However, it is important to note that there is still a context limitation of 4k tokens, making it unsuitable for handling excessively lengthy queries.
Leave a Reply