Technology

34728 readers

123 users here now

This is the official technology community of Lemmy.ml for all news related to creation and use of technology, and to facilitate civil, meaningful discussion around it.

Ask in DM before posting product reviews or ads. All such posts otherwise are subject to removal.

Rules:

1: All Lemmy rules apply

2: Do not post low effort posts

3: NEVER post naziped*gore stuff

4: Always post article URLs or their archived version URLs as sources, NOT screenshots. Help the blind users.

5: personal rants of Big Tech CEOs like Elon Musk are unwelcome (does not include posts about their companies affecting wide range of people)

6: no advertisement posts unless verified as legitimate and non-exploitative/non-consumerist

7: crypto related posts, unless essential, are disallowed

founded 5 years ago

MODERATORS

[email protected]

TheBloke Releases "SuperHOT" Versions of Various GPTQ Models - Empowering LLM Users w/ a Context Length of 8,000 Tokens! (lemmy.world)

submitted 1 year ago by [email protected] to c/[email protected]

0 comments fedilink hide all child comments

cross-posted from: https://lemmy.world/post/708817

Visit TheBloke's HuggingFace page to see all of the new models in their SuperHOT glory.

SuperHOT models are LLMs who's LoRAs have been adapted to support a context length of 8,000 tokens!

For reference, this is x4 times the default amount of many LLMs (i.e. 2048 tokens). Even some of the newer ones can only reach a context length of 4096 tokens, half the amount of these SuperHOT models!

Here are a few that were released if you couldn't view his HuggingFace:

New GPTQ Models from TheBloke

airoboros (13B)

CAMEL (13B)

Chronos (13B)

Guanaco (13B & 33B)

Manticore (13B)

Minotaur (13B)

Nous Hermes (13B)

Pygmalion (13B)

Samantha (13B & 33B)

Snoozy (13B)

Tulu (13B & 33B)

Vicuna (13B & 33B)

WizardLM (13B)

We owe a tremendous thank you to TheBloke, who has enabled many of us in the community to interact with versions of Manticore, Nous Hermes, WizardLM and others running the remarkable 8k context length from SuperHOT.

Many of these are 13B models, which should be compatible with consumer grade GPUs. Try using Exllama or Oobabooga for testing out these new formats.

Shoutout to Kaikendev for the creation of SuperHOT. You can learn more about their work here. or in Meta's new research paper covering this method.

If you enjoyed reading this, please consider subscribing to /c/FOSAI where I do my best to keep you in the know with the latest and greatest advancements regarding free open-source artificial intelligence.

no comments (yet)

sorted by: hot top controversial new old

there doesn't seem to be anything here