Welcome to the FOSAI Nexus!

(v0.0.1 - Summer 2023 Edition)

The goal of this knowledge nexus is to act as a link hub for software, applications, tools, and projects that are all FOSS (free open-source software) designed for AI (FOSAI).

If you haven't already, I recommend bookmarking this page. It is designed to be periodically updated in new versions I release throughout the year. This is due to the rapid rate in which this field is advancing. Breakthroughs are happening weekly. I will try to keep up through the seasons while including links to each sequential nexus post - but it's best to bookmark this since it will be the start of the content series, giving you access to all future nexus posts as I release them.

If you see something here missing that should be added, let me know. I don't have visibility over everything. I would love your help making this nexus better. Like I said in my welcome message, I am no expert in this field, but I teach myself what I can to distill it in ways I find interesting to share with others.

I hope this helps you unblock your workflow or project and empowers you to explore the wonders of emerging artificial intelligence.

Consider subscribing to /c/FOSAI if you found any of this interesting. I do my best to make sure you stay in the know with the most important updates to all things free open-source AI.

Find Us On Lemmy!

[email protected]

---

Fediverse Resources

Lemmy

• Your Lemmy Crash Course to Free Open-Source AI

• [email protected]

---

Large Language Model Hub

Download Models

oobabooga

text-generation-webui - a big community favorite gradio web UI by oobabooga designed for running almost any free open-source and large language models downloaded off of HuggingFace which can be (but not limited to) models like LLaMA, llama.cpp, GPT-J, Pythia, OPT, and many others. Its goal is to become the AUTOMATIC1111/stable-diffusion-webui of text generation. It is highly compatible with many formats.

Exllama

A standalone Python/C++/CUDA implementation of Llama for use with 4-bit GPTQ weights, designed to be fast and memory-efficient on modern GPUs.

gpt4all

Open-source assistant-style large language models that run locally on your CPU. GPT4All is an ecosystem to train and deploy powerful and customized large language models that run locally on consumer-grade processors.

TavernAI

The original branch of software SillyTavern was forked from. This chat interface offers very similar functionalities but has less cross-client compatibilities with other chat and API interfaces (compared to SillyTavern).

SillyTavern

Developer-friendly, Multi-API (KoboldAI/CPP, Horde, NovelAI, Ooba, OpenAI+proxies, Poe, WindowAI(Claude!)), Horde SD, System TTS, WorldInfo (lorebooks), customizable UI, auto-translate, and more prompt options than you'd ever want or need. Optional Extras server for more SD/TTS options + ChromaDB/Summarize. Based on a fork of TavernAI 1.2.8

Koboldcpp

A self-contained distributable from Concedo that exposes llama.cpp function bindings, allowing it to be used via a simulated Kobold API endpoint. What does it mean? You get llama.cpp with a fancy UI, persistent stories, editing tools, save formats, memory, world info, author's note, characters, scenarios, and everything Kobold and Kobold Lite have to offer. In a tiny package around 20 MB in size, excluding model weights.

KoboldAI-Client

This is a browser-based front-end for AI-assisted writing with multiple local & remote AI models. It offers the standard array of tools, including Memory, Author's Note, World Info, Save & Load, adjustable AI settings, formatting options, and the ability to import existing AI Dungeon adventures. You can also turn on Adventure mode and play the game like AI Dungeon Unleashed.

h2oGPT

h2oGPT is a large language model (LLM) fine-tuning framework and chatbot UI with document(s) question-answer capabilities. Documents help to ground LLMs against hallucinations by providing them context relevant to the instruction. h2oGPT is fully permissive Apache V2 open-source project for 100% private and secure use of LLMs and document embeddings for document question-answer.

---

Image Diffusion Hub

Download Models

StableDiffusion

Stable Diffusion is a text-to-image diffusion model capable of generating photo-realistic and stylized images. This is the free alternative to MidJourney. It is rumored that MidJourney originates from a version of Stable Diffusion that is highly modified, tuned, then made proprietary.

SDXL (Stable Diffusion XL)

With Stable Diffusion XL, you can create descriptive images with shorter prompts and generate words within images. The model is a significant advancement in image generation capabilities, offering enhanced image composition and face generation that results in stunning visuals and realistic aesthetics.

ComfyUI

A powerful and modular stable diffusion GUI and backend. This new and powerful UI will let you design and execute advanced stable diffusion pipelines using a graph/nodes/flowchart-based interface.

ControlNet

ControlNet is a neural network structure to control diffusion models by adding extra conditions. This is a very popular and powerful extension to add to AUTOMATIC111's stable-diffusion-webui.

TemporalKit

An all-in-one solution for adding Temporal Stability to a Stable Diffusion Render via an automatic1111 extension. You must install FFMPEG to path before running this.

EbSynth

Bring your paintings to animated life. This software can be used in conjunction with StableDiffusion + ControlNet + TemporalKit workflows.

WarpFusion

A TemporalKit alternative to produce video effects and animation styling.

---

Training & Education

LLMs

• Oobabooga's text-generation-webui

• Axolotl

• OpenAI Cookbook

• AemonAlgiz

Diffusers

• enigmatic_e

• Albert Bozesan

• ControlNet Tutorial

• Stable Diffusion + ControlNet + TemporalKit + EbSynth Workflow

• Stable Diffusion + Warp Fusion Workflow

---

Bonus Recommendations

AI Business Startup Kit

LLM Learning Material from the Developer of SuperHOT (kaiokendev):

Here are some resources to help with learning LLMs:

Andrej Karpathy’s GPT from scratch

Huggingface’s NLP Course

And for training specifically:

Alpaca LoRA

Vicuna

Community training guide

Of course for papers, I recommend reading anything on arXiv’s CS - Computation & Language that looks interesting to you: https://arxiv.org/list/cs.CL/recent.

---

Support Developers!

Please consider donating, subscribing to, or buying a coffee for any of the major community developers advancing Free Open-Source Artificial Intelligence.

If you're a developer in this space and would like to have your information added here (or changed), please don't hesitate to message me!

TheBloke

• https://www.patreon.com/TheBlokeAI

Oobabooga

• https://ko-fi.com/oobabooga

Eric Hartford

• https://erichartford.com/

kaiokendev

• https://kaiokendev.github.io/

---

Major FOSAI News & Breakthroughs

• (June 2023) MPT-30B: Raising the bar for open-source foundation models

• (May 2023) Google "We Have No Moat, And Neither Does OpenAI"

• (May 2023) Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs

• (March 2023) OpenAI Releases Chat-GPT 4

• (November 2022) OpenAI Releases Chat-GPT 3

• (December 2017) Attention Is All You Need

---

Looking for other open-source projects based on these technologies? Consider checking out this GitHub Repo List I made based on stars I have collected throughout the last year or so.

Huge news for AMD fans and those who are hoping to see a real* open alternative to CUDA that isn't OpenCL!

*: Intel doesn't count, they still have to get their shit together in rendering things correctly with their GPUs.

We plan to expand ROCm support from the currently supported AMD RDNA 2 workstation GPUs: the Radeon Pro v620 and w6800 to select AMD RDNA 3 workstation and consumer GPUs. Formal support for RDNA 3-based GPUs on Linux is planned to begin rolling out this fall, starting with the 48GB Radeon PRO W7900 and the 24GB Radeon RX 7900 XTX, with additional cards and expanded capabilities to be released over time.

cross-posted from: https://lemmy.world/post/809672

A very exciting update comes to koboldcpp - an inference software that allows you to run LLMs on your PC locally using your GPU and/or CPU.

Koboldcpp is one of my personal favorites. Shoutout to LostRuins for developing this application. Keep the release memes coming!

koboldcpp-1.33 Ultimate Edition Release Notes

A.K.A The "We CUDA had it all edition"

The KoboldCpp Ultimate edition is an All-In-One release with previously missing CUDA features added in, with options to support both CL and CUDA properly in a single distributable. You can now select CUDA mode with --usecublas, and optionally low VRAM using --usecublas lowvram. This release also contains support for OpenBLAS, CLBlast (via --useclblast), and CPU-only (No BLAS) inference.

Back ported CUDA support for all prior versions of GGML file formats for CUDA. CUDA mode now correctly supports every single earlier version of GGML files, (earlier quants from GGML, GGMF, GGJT v1, v2 and v3, with respective feature sets at the time they were released, should load and work correctly.)

Ported over the memory optimizations I added for OpenCL to CUDA, now CUDA will use less VRAM, and you may be able to use even more layers than upstream in llama.cpp (testing needed).

Ported over CUDA GPU acceleration via layer offloading for MPT, GPT-2, GPT-J and GPT-NeoX in CUDA.

Updated Lite, pulled updates from upstream, various minor bugfixes. Also, instruct mode now allows any number of newlines in the start and end tag, configurable by user.

Added long context support using Scaled RoPE for LLAMA, which you can use by setting --contextsize greater than 2048. It is based off the PR here ggerganov#2019 and should work reasonably well up to over 3k context, possibly higher.

To use, download and run the koboldcpp.exe, which is a one-file pyinstaller. Alternatively, drag and drop a compatible ggml model on top of the .exe, or run it and manually select the model in the popup dialog.

...once loaded, you can connect like this (or use the full koboldai client): http://localhost:5001

For more information, be sure to run the program with the --help flag.

If you found this post interesting, please consider subscribing to the /c/FOSAI community at [email protected] where I do my best to keep you in the know with the most important updates in free open-source artificial intelligence.

Interested, but not sure where to begin? Try starting with Your Lemmy Crash Course to Free Open-Source AI

Although watching TV shows from the 1970s suggests otherwise, the era wasn't completely devoid of all things resembling modern communication systems. Sure, the 50Kbps modems that the ARPANET ran on were the size of refrigerators, and the widely used Bell 103 modems only transferred 300 bits per second. But long-distance digital communication was common enough, relative to the number of computers deployed. Terminals could also be hooked up to mainframe and minicomputers over relatively short distances with simple serial lines or with more complex multidrop systems. This was all well known; what was new in the '70s was the local area network (LAN). But how to connect all these machines?

Virgin Galactic will be launching their first commercial, sub-orbital space flight today. Link is to the Live Stream for the event.

cross-posted from: https://lemmy.world/post/800062

Eric Hartford (a.k.a. faldore) has announced OpenOrca, an open-source dataset and series of instruct-tuned language models he plans to release alongside Microsoft's new open-source challenger, Orca.

You can support Eric and all of the hard work he has done for the open-source community by following his newsletter on his site here.

Eric, if you're reading this and would like to share a donation link - I would be more than happy to include it on this post and any future regarding your work. Shoot me a message anytime.

Eric Hartford's Announcement

Today I'm announcing OpenOrca.

https://erichartford.com/openorca

https://twitter.com/erhartford/status/1674214496301383680

The dataset is completed. ~1mil of GPT4 augmented flanv2 instructions and ~3.5mil of GPT3.5 augmented flanv2 instructions.

We are currently training on LLaMA-13b. We expect completion in about 2 weeks.

When training is complete, we will release the dataset and the model at the same time.

We are seeking GPU compute sponsors for various targets, please consult the blog post and reach out if interested.

Thank you to our sponsors!

https://chirper.ai

https://preemo.io

https://latitude.sh

A few more highlights from the full article, which you should read here when you have a chance.

We expect to release OpenOrca-LLaMA-13b in mid-July 2023. At that time we will publish our evaluation findings and the dataset.

We are currently seeking GPU compute sponsors for training OpenOrca on the following platforms:

Falcon 7b, 40b

LLaMA 7b, 13b, 33b, 65b

MPT-7b, 30b

Any other targets that get a sponsor. (RWKV, OpenLLaMA)

Dataset consists of:

• ~1 million of FLANv2 augmented with GPT-4 completions

• ~3.5 million of FLANv2 augmented with GPT-3.5 completions

If you found this post interesting, please consider subscribing to the /c/FOSAI community at [email protected] where I do my best to keep you in the know with the most important updates in free open-source artificial intelligence.

cross-posted from: https://lemmy.world/post/708817

Visit TheBloke's HuggingFace page to see all of the new models in their SuperHOT glory.

SuperHOT models are LLMs who's LoRAs have been adapted to support a context length of 8,000 tokens!

For reference, this is x4 times the default amount of many LLMs (i.e. 2048 tokens). Even some of the newer ones can only reach a context length of 4096 tokens, half the amount of these SuperHOT models!

Here are a few that were released if you couldn't view his HuggingFace:

New GPTQ Models from TheBloke

• airoboros (13B)
• CAMEL (13B)
• Chronos (13B)
• Guanaco (13B & 33B)
• Manticore (13B)
• Minotaur (13B)
• Nous Hermes (13B)
• Pygmalion (13B)
• Samantha (13B & 33B)
• Snoozy (13B)
• Tulu (13B & 33B)
• Vicuna (13B & 33B)
• WizardLM (13B)

We owe a tremendous thank you to TheBloke, who has enabled many of us in the community to interact with versions of Manticore, Nous Hermes, WizardLM and others running the remarkable 8k context length from SuperHOT.

Many of these are 13B models, which should be compatible with consumer grade GPUs. Try using Exllama or Oobabooga for testing out these new formats.

Shoutout to Kaikendev for the creation of SuperHOT. You can learn more about their work here. or in Meta's new research paper covering this method.

If you enjoyed reading this, please consider subscribing to /c/FOSAI where I do my best to keep you in the know with the latest and greatest advancements regarding free open-source artificial intelligence.

Paper available here: https://www.computer.org/csdl/proceedings-article/sp/2023/933600a951/1NrbYmtLXB6

Transcription below

A BILL TO BE ENTITLED

AN ACT TO STUDY THE HOLDING OF BULLION AND VIRTUAL CURRENCY AND THEIR POTENTIAL BENEFITS AND WHETHER TO ESTABLISH A NORTH CAROLINA BULLION DEPOSITORY FOR SUCH ASSETS.

The General Assembly of North Carolina enacts: SECTION 1. The Department of State Treasurer shall conduct a study that examines (i) the process of acquiring, securely storing, insuring, and liquidating any investment metal bullion as defined in G.S. 105-164.13(69), such as gold, and virtual currency as defined in G.S. 53-208.42(20), such as Bitcoin, that may be held on behalf of the State, (ii) the expected impact of allocating a portion of the General Fund to investment metal bullion and virtual currency to hedge against inflation and systemic credit risks, reduce overall portfolio volatility, and increase portfolio returns over time, and (iii) the costs, benefits, and security of utilizing a privately managed depository or another state's depository or creating a State-administered depository in North Carolina to serve as the custodian, guardian, and administrator of certain investment metal bullion and virtual currency that may be transferred to or otherwise acquired by this State or an agency, a political subdivision, or another instrumentality of this State and to provide a repository for investors to use for such assets. The Department of State Treasurer shall 18 report on the results of the study, along with any legislative or other recommendations, to the 19 Joint Legislative Commission on Governmental Operations by January 1, 2024.

SECTION 2. There is appropriated from the General Fund to the Department of State Treasurer the nonrecurring sum of fifty thousand dollars ($50,000) for the 2023-2024 fiscal year to conduct the study required by this act.

SECTION 3. Section 2 of this act becomes effective July 1, 2023. The remainder of 24 this act is effective when it becomes law.

Rapid changes, fueled by AI, are impacting the large pockets of the internet, argues a new column. An excerpt:

In recent months, the signs and portents have been accumulating with increasing speed. Google is trying to kill the 10 blue links. Twitter is being abandoned to bots and blue ticks. There's the junkification of Amazon and the enshittification of TikTok. Layoffs are gutting online media. A job posting looking for an "AI editor" expects "output of 200 to 250 articles per week." ChatGPT is being used to generate whole spam sites. Etsy is flooded with "AI-generated junk."

Chatbots cite one another in a misinformation ouroboros. LinkedIn is using AI to stimulate tired users. Snapchat and Instagram hope bots will talk to you when your friends don't. Redditors are staging blackouts. Stack Overflow mods are on strike. The Internet Archive is fighting off data scrapers, and "AI is tearing Wikipedia apart." The old web is dying, and the new web struggles to be born. The web is always dying, of course; it's been dying for years, killed by apps that divert traffic from websites or algorithms that reward supposedly shortening attention spans. But in 2023, it's dying again -- and, as the litany above suggests, there's a new catalyst at play: AI.

Guess where? Unironically r/Save3rdPartyApps

The Reddit search for Lemmy also gives these privacy copy-pasta as top results when searching for Lemmy. I'm still betting that Reddit employees are involved in boosting these posts.

cross-posted from: https://lemmy.world/post/708817

Visit TheBloke's HuggingFace page to see all of the new models in their SuperHOT glory.

SuperHOT models are LLMs who's LoRAs have been adapted to support a context length of 8,000 tokens!

For reference, this is x4 times the default amount of many LLMs (i.e. 2048 tokens). Even some of the newer ones can only reach a context length of 4096 tokens, half the amount of these SuperHOT models!

Here are a few that were released if you couldn't view his HuggingFace:

New GPTQ Models from TheBloke

• airoboros (13B)
• CAMEL (13B)
• Chronos (13B)
• Guanaco (13B & 33B)
• Manticore (13B)
• Minotaur (13B)
• Nous Hermes (13B)
• Pygmalion (13B)
• Samantha (13B & 33B)
• Snoozy (13B)
• Tulu (13B & 33B)
• Vicuna (13B & 33B)
• WizardLM (13B)

We owe a tremendous thank you to TheBloke, who has enabled many of us in the community to interact with versions of Manticore, Nous Hermes, WizardLM and others running the remarkable 8k context length from SuperHOT.

Many of these are 13B models, which should be compatible with consumer grade GPUs. Try using Exllama or Oobabooga for testing out these new formats.

Shoutout to Kaikendev for the creation of SuperHOT. You can learn more about their work here.

If you enjoyed reading this, please consider subscribing to /c/FOSAI where I do my best to keep you in the know with the latest and greatest advancements regarding free open-source artificial intelligence.

https://web.archive.org/web/20230627195308/https://www.redhat.com/en/blog/red-hats-commitment-open-source-response-gitcentosorg-changes