Hugging Face account with a read-only API token. You will use this to fetch the models that vLLM will run. You may also need to accept the terms and conditions or usage license agreements associated with the models you intend to use. In some cases, you may need to request access to the model from the model owners on Hugging Face. For this guide, make sure you have accepted any terms required for the... - Source: dev.to / 4 days ago

This template ships with Google Gemini models/gemini-1.0-pro-001 as the default. However, thanks to the Vercel AI SDK, you can switch LLM providers to OpenAI, Anthropic, Cohere, Hugging Face, or using LangChain with just a few lines of code. - Source: dev.to / 6 days ago

I wanted to get a project for running my own pipeline with somewhat interchangeable parts. Models can be swapped around so that you can make the most of the latest models either available on Hugginface, OpenAI or wherever. - Source: dev.to / about 2 months ago

Log in to your Huggingface account at https://huggingface.co. Click Access Token in the menu to generate a new token. - Source: dev.to / 18 days ago

While looking into how to create text embeddings quickly and directly, we discovered a few helpful tools that allowed us to achieve our goal. Consequently, we created an easy-to-use PHP extension that can generate text embeddings. This extension lets you pick any model from Sentence Transformers on HuggingFace. It is built on the CandleML framework, which is written in Rust and is a part of the well-known... - Source: dev.to / 24 days ago

Similar to https://faraday.dev/ that also runs locally. I wish I can install on desktop like Faraday to try it. - Source: Hacker News / 3 months ago

Sadly I can't try this because I'm on Windows or Linux. Was testing apps like this if anyone is interested: Best / Easy to use: - https://lmstudio.ai - https://msty.app - https://jan.ai More complex / Unpolished UI: - https://gpt4all.io - https://pinokio.computer - https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generative-ai/ - https://github.com/LostRuins/koboldcpp (Ai Characters) No UI / Command line (not... - Source: Hacker News / 4 months ago

Aside from LM Studio there's also Faraday https://faraday.dev/. - Source: Hacker News / 7 months ago

I'm having a lot of fun chatting with characters using Faraday and koboldcpp. Faraday has a great UI that lets you adjust character profiles, generate alternative model responses, undo, or edit dialogue, and experiment with how models react to your input. There's also SillyTavern that I have yet to try out. - https://faraday.dev/. - Source: Hacker News / 7 months ago

This is the easiest to setup: https://faraday.dev/ I think Wizard is the “meta” for technical questions now. - Source: Hacker News / 8 months ago