AI

4155 readers

2 users here now

Artificial intelligence (AI) is intelligence demonstrated by machines, unlike the natural intelligence displayed by humans and animals, which involves consciousness and emotionality. The distinction between the former and the latter categories is often revealed by the acronym chosen.

founded 3 years ago

1-bit LLMs Could Solve AI’s Energy Demands (spectrum.ieee.org)

submitted 5 months ago by [email protected] to c/[email protected]

7 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 8 points 5 months ago (1 children)

In February, Wei’s team announced BitNet 1.58b, in which parameters can equal -1, 0, or 1, which means they take up roughly 1.58 bits of memory per parameter. A BitNet model with 3 billion parameters performed just as well on various language tasks as a full-precision LLaMA model with the same number of parameters and amount of training, but it was 2.71 times as fast, used 72 percent less GPU memory, and used 94 percent less GPU energy. Wei called this an “aha moment.” Further, the researchers found that as they trained larger models, efficiency advantages improved.

That's pretty impressive.

[–] [email protected] 1 points 5 months ago (1 children)

What exactly is a half bit? Is a transistor "half open" or something?

[–] [email protected] 4 points 5 months ago (1 children)

I'm no expert but I think it's just a statistical measure of information, not something that can be physically realized in isolation. If two possible states are 1 bit and four possible states are 2 bits, then 3 possible states lies somewhere in between.

I did a bit of a search and found this: How can there be a fraction of a bit?

[–] [email protected] 1 points 5 months ago

Oh cool. I was wondering if it was just a math thing or it was related to hardware used in ternary computers.