Or if you just ignore federal courts, which seems to be the current fashion.

Rclone can do file mounts as well as sync.

A lot of the answers here are short or quippy. So, here’s a more detailed take. LLMs don’t “know” how good a source is. They are word association machines. They are very good at that. When you use something like Perplexity, an external API feeds information from the search queries into the LLM, and then it summarizes that text in (hopefully) a coherent way. There are ways to reduce hallucination rate and check factualness of sources, e.g. by comparing the generated text against authoritative information. But how much of that is employed by Perplexity et al I have no idea.

For stuff like that, it’s best to have an auto formatter like checkstyle or something.

Had a team lead that kept requesting nitpicky changes, going in a FULL CIRCLE about what we should change or not, to the point that changes would take weeks to get merged. Then he had the gall to say that changes were taking too long to be merged and that we couldn’t just leave code lying around in PRs.

Jesus fucking Christ.

There’s a reason that team imploded…

https://agnos.is/posts/tech-recruitment-is-out-of-control.html

This was my experience at the beginning of 2024. It was bad enough that I had to write a blog post about it.

How much speed are you actually getting on Mixtral (I assume that’s the 8x7b). I have 64 GB of RAM and an AMD RX 6800 XT with 16 GB of VRAM. I get like 4 tokens per second with Q5_K_M quant.

The only problem I really have, is context size. It’s harder to get larger than 8k context size and maintain decent generation speed with 16 GB of VRAM and 16 GB of RAM. Gonna get more RAM at some point though, and hope ollama/llamacpp gets better at memory management. Hopefully the distributed running from llamaccp ends up in ollama.

I do have a local setup. Not powerful enough to run Mixtral 8x22b, but can run 8x7b (albeit quite slowly). Use it a lot.

No trying to get around anything. No funny instructions like my grandma singing a lullaby about illegal activities. Just using instructions to tell a story. Even things like having a superhero in a fight is enough to trigger this. Also doesn’t explain why regen makes it continue.

A vector search converts your query into magic numbers, and then searches the database for other magic numbers that are “similar” (closet to it in the vector space, which is basically an N-dimensional graph of points and directions). These results are then returned as snippets to the LLM and it does stuff from that point.

The effectiveness of the vector search depends on how Open WebUI breaks up the documents into smaller sections, and how good the embeddings are.

I’m not exactly sure what you want to achieve, but you might have success in using an LLM to summarize the documents beforehand, using a specific prompt to extract the info you want, then feed that into the vector DB. This would require some scripting, of course.

The easiest thing to do is try it. See if Open WebUI’s vector search is able to handle it. Make sure to use a good embedding model like nomic-embed-text (can be found on ollama.com). You can change the vector search settings in the documents settings from the workspace on OpenWebUI.

Edit: https://ollama.com/library/nomic-embed-text

Open WebUI’s document management loads everything into a vector database. When you use the hashtag, it will trigger a search against the vector database based on your prompt. These results are run feed into the LLM. Open WebUI should generate a hashtag that can reference all the documents. But the quality of the results will be influenced by the embeddings and the LLM that responds to you.

Install ollama. It has ROCm support (on Linux at least). Then hook it up to your favorite client. It has its own API and an openai compatible one.

KoboldCPP has ban tokens that prevent those tokens from being output. Otherwise just put it in the prompt and it should probably work.

https://sh.itjust.works/c/localllama

Even the smell of Olives causes me to gag. I absolutely cannot eat them. Olive oil is fine. But actual olives, no. Doesn’t matter if they’re old, new, canned, fresh. They’re absolutely disgusting. One of the few foods I outright cannot and will not eat.

Doesn’t gnome already have this?

Cygwin ftw

Lol. Git itself can act as a server over the git protocol. Might have been easier 🤪

There’s plenty of git forges that aren’t GitHub. Git itself has nothing to do with central servers and can theoretically be used in a completely decentralized manner.