mediocreatbest

joined 1 year ago
MODERATOR OF
mediocreatbest
sorted by: new top controversial old
pointers are very eleganto in c/[email protected]
[–] [email protected] 5 points 1 year ago

If you've never seen this before, I think it's transformative to how you read C/C++ declarations and clearer up a lot of confusion for me when I was learning.

https://cseweb.ucsd.edu/~gbournou/CSE131/rt_lt.rule.html

1
[StackOverflow] If a PCI device is completely non-responsive, it's possible to completely remove the device and then re-scan it, hopefully re-initializing the device so it works again. (unix.stackexchange.com)
 

echo 1 | sudo tee /sys/bus/pci/<pci-id-of-device>/remove and then echo 1 | sudo tee /sys/bus/pci/rescan

1
[GitHub] bduggan/raku-jupyter-kernel allows you to run Raku (né Perl 6) within a Jupyter Notebook environment. In terms of onboarding, this seems to be one of the easiest ways to start using Raku. (github.com)
1
[Paper] Optimizing Deep Learning Models For Raspberry Pi. Custom CNN (on MNIST data) performance from 114ms to 3.75ms. ResNet50 (on "flowers" data): from 1.1s to 1.0s (lowest) or 1.6s (highest). (arxiv.org)
 

I'm a little unsure on if I interpreted the results correctly. It seems like some things that TF Lite natively supports (apparently, their custom CNN model trained on MNIST) get really fast, and other things are a little hit-or-miss.

1
TinyNeuralNetwork is a library to compress machine learning models through pruning, quantization, and more. Can also convert PyTorch models to TF Lite models. (github.com)
1
Overview of machine learning frameworks that are supported on Raspberry Pi: OpenCV, TF Lite, Tencent ncnn, Tencent TNN, Alibaba MNN, Paddle Lite, ARMnn, MXNet + Gluon, PyTorch, and Caffe. (qengineering.eu)
1
Arm NN is an optimized library of tensor operators for machine learning models to use. Support for TF Lite / ONNX models and Raspberry Pi 4 / armv7. (github.com)
1
TextSynth is a hosted service for generating text completions using language models. Free and paid tiers. Could be useful to play with LLMs without a strong computer (Pricing discussion in body text). (textsynth.com)
 

I have linked the pricing page because I think that's the most important aspect to a service like this.

The price isn't too expensive, but it also isn't particular cheap either.

Compared to OpenAI's ChatGPT model and generating 1 million tokens (i.e. the King James Bible), you're looking at:

  • OpenAI's gpt-3.5-turbo ("ChatGPT-3.5") is $2 / 1m tokens
  • TextSynth's M2M100 1.2B (cheapest) is $3 / 1m tokens
  • OpenAI's gpt-4 ("ChatGPT-4") is $4 / 1m tokens
  • TextSynth's GPT-Neox 20B (most expensive) is $35 / 1m tokens
1
LaMini-LM is a collection of small language models that are accessible to run on local hardware without lots of resources. Models range from 250MB to 6.3GB. (github.com)
1
jncraton/languagemodels is a simple Python library for running LLMs locally. Supports instruction and embedding use cases. Chooses models according to available RAM. (github.com)
 

More information on the LocalLLaMA subreddit from the author

Megathread for Reddit Blackouts and News - Day 3 in c/[email protected]
[–] [email protected] 1 points 1 year ago

I don't know what kind of comments and posts you've made on Reddit, but if any of them are technical how-to's or something that may come up when people search for specific problems, then it might be good to leave those comments, or else just prefix each comment with your "purged" message instead of overwriting them entirely. I mean if it's fun memes or discussions, then you do you 😅 I'm just thinking of the tale of DenverCoder9. Plus, it probably costs more for Reddit to store a longer comment than a shorter one! Pennies or less, but still!

1
Altoids tin for watercolor using sculpey modeling clay to create a custom tray for the paints (www.instructables.com)
1
Taming AI Bots: Prevent LLMs from entering "bad" states using continuous guidance from the LLM ("is this good? bad?") to avoid bad states. (arxiv.org)
1
"Prompt Gisting:" Train two models such that given inputs "Translate French" and "G2>The cat," then G1 and G2 represent the entire instruction. (arxiv.org)
 

Abstract: "Prompting is now the primary way to utilize the multitask capabilities of language models (LMs), but prompts occupy valuable space in the input context window, and re-encoding the same prompt is computationally inefficient. Finetuning and distillation methods allow for specialization of LMs without prompting, but require retraining the model for each task. To avoid this trade-off entirely, we present gisting, which trains an LM to compress prompts into smaller sets of "gist" tokens which can be reused for compute efficiency. Gist models can be easily trained as part of instruction finetuning via a restricted attention mask that encourages prompt compression. On decoder (LLaMA-7B) and encoder-decoder (FLAN-T5-XXL) LMs, gisting enables up to 26x compression of prompts, resulting in up to 40% FLOPs reductions, 4.2% wall time speedups, storage savings, and minimal loss in output quality. "

view more: next ›