Screenshot by Editor

It’s been an interesting 12 months. A lot has happened with large language models (LLMs) being at the forefront of everything tech-related. You have LLMs such as ChatGPT, Gemini, and more. 

These LLMs are currently run in the cloud, meaning they run somewhere else on someone else's computer. For something to be run elsewhere, you can imagine how expensive it is. Because if it was so cheap, why not run it locally on your own computer?

But that’s all changed now. You can now run different LLMs with LM Studio.

What is LM Studio?

LM studio is a tool that you can use to experiment with local and open-source LLMs. You can run these LLMs on your laptop, entirely off. There are two ways that you can discover, download and run these LLMs locally:

• Through the in-app Chat UI
• OpenAI compatible local server

All you have to do is download any model file that is compatible from the HuggingFace repository, and boom done! 

So how do I get started?

LM Studio Requirements

Before you can get kickstarted and start delving into discovering all the LLMs locally, you will need these minimum hardware/software requirements:

• M1/M2/M3 Mac
• Windows PC with a processor that supports AVX2. (Linux is available in beta)
• 16GB+ of RAM is recommended
• For PCs, 6GB+ of VRAM is recommended
• NVIDIA/AMD GPUs supported

If you have these, you’re ready to go!

So what are the steps?

How to Use LM Studio

Your first step is to download LM Studio for Mac, Windows, or Linux, which you can do here. The download is roughly 400MB, therefore depending on your internet connection, it may take a whole. 

Your next step is to choose a model to download. Once LM Studio has been launched, click on the magnifying glass to skim through the options of models available. Again, take into consideration that these models with be large, therefore it may take a while to download. 

Once the model has been downloaded, click the Speech Bubble on the left and select your model for it to load. 

Ready to chit-chat!

There you have it, that quick and simple to set up an LLM locally. If you would like to speed up the response time, you can do so by enabling the GPU acceleration on the right-hand side. 

Wrapping it up

Do you see how quick that was? Fast right.

If you are worried about the collection of data, it is good to know that the main reason for being able to use an LLM locally is privacy. Therefore, LM Studio has been designed exactly for that!

Have a go and let us know what you think in the comments!
 
 

Nisha Arya is a data scientist, freelance technical writer, and an editor and community manager for KDnuggets. She is particularly interested in providing data science career advice or tutorials and theory-based knowledge around data science. Nisha covers a wide range of topics and wishes to explore the different ways artificial intelligence can benefit the longevity of human life. A keen learner, Nisha seeks to broaden her tech knowledge and writing skills, while helping guide others.