Just looking for some opinions, on which models and why, from people who have used FreeGPT w/ Start9.
I have not fired it up yet, so I’m open to all reasons.
Thnx
I think the language models differ mainly in size and perhaps in emphasis. Which one you choose should be dependent on how beefy your hardware is. The instructions say "Download a Large Language Model (LLM)
In the FreeGPT interface click on “DOWNLOAD MODELS”
Choose a language model from the list provided. Models are automatically filtered based on the available RAM of your server. Larger models should provide better responses, but they will be slower.
I am running a Server Puer with plenty of resources show I chose “Nous-Hermes-Llama2-13B”. When you look at the list provided on your Start9 Server it will show the amount of GB required
1 Like
Not a StartOS-specific question but I was wondering if the LLM can pointed to personal files. Can I give it access to certain file folders and then prompt it with questions about the contents of those files / can I upload PDFs and ask it questions about them? Or does it only operate on data from the internet?
Trying to figure out if I’ll actually use it before allocating the resources. Thanks.
1 Like
Welcome to our community, daddywarbits!
LLMs on FreeGPT-2 don’t access personal files or PDFs directly from services like Nextcloud or File Browser. If the LLM supports file uploads, you can upload documents directly in the conversation bar, and the LLM will parse them.
3 Likes
Thanks for the warm welcome, h0mer. And thanks for the explanation.
3 Likes
This is rad. Sounds like the available models are based on hardware specs. Are Llama 3.1 70B or 8B available? If so, how is their performance?
mistral:7b model is recommended. The link for that is here library
You can browse around and try different models. Your performance will depend on your specs. A small computer will not perform like ChatGPT’s giant servers but it is useful for some tasks, fun, and most of all private.
1 Like