Smorty

I'm trying to learn blender for making tandril studio-like animations (the UI animations usually published by Microsoft when announcing something new like here with Copilot).

I know that's a big goal, but I still wanna do it...

marty@Marty-PC:~/git/exllama$ pip install numpy
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    
    See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

I get this error every time I try install any kind of python package. So far, I always just used the --break-system-packages flag, but that seems, well, rather unsafe and breaking.

To this day, I see newly written guides, specifically for Linux, which don't point out this behaviour. They just say [...] And then install this python package with 'pip install numpy'

Is this something specific to my system, or is this a global thing?

I posted this question on the godot forum, but that has been overrun by bots like crazy, it's all random characters, so probably some AI.

I want to edit the text of the current script editor without modifying the internal undo-redo of the editor.

So essentially I want this to be possible:

1. User writes some code
2. Program modifies it, maybe imroving the formatting and such...
3. Program then reverts to previously written code
4. User is able to undo their last code addition, without the programmatically inserted stuff to show up as the last undo-step.

I posted this question on the godot forum, but that has been overrun by bots like crazy, it's all random characters, so probably some AI.

I want to edit the text of the current script editor without modifying the internal undo-redo of the editor.

So essentially I want this to be possible:

1. User writes some code
2. Program modifies it, maybe imroving the formatting and such...
3. Program then reverts to previously written code
4. User is able to undo their last code addition, without the programmatically inserted stuff to show up as the last undo-step.

I am using a code-completion model for (will be open sourced very soon).

Qwen2.5-coder 1.5b though tends to repeat what has already been written, or change it slightly. (See the video)

Is this intentional? I am passing the prefix and suffix correctly to ollama, so it knows where it currently is. I'm also trimming the amount of lines it can see, so the time-to-first-token isn't too long.

Do you have a recommendation for a better code model, better suited for this?

I am using a code-completion model for (will be open sourced very soon).

Qwen2.5-coder 1.5b though tends to repeat what has already been written, or change it slightly. (See the video)

Is this intentional? I am passing the prefix and suffix correctly to ollama, so it knows where it currently is. I'm also trimming the amount of lines it can see, so the time-to-first-token isn't too long.

Do you have a recommendation for a better code model, better suited for this?

This is something I have been stuck on for a while.

I want to use Wayland for that variable refresh rate and some better handeling of screen recordings.

I have tried time and time again to get a wayland session running with the proprietary nvidia driver, but have not gotten there yet.

Only the X11 options are listed on the login screen. When using the fallback FOSS nvidia driver however, all the correct X11 and Wayland options show up (Including Gnome and KDE, both in X11 and Wayland).

Wasn't this fixed, like, about a year ago? I have the "latest" proprietary nvidia driver, but the current debain one is still pretty old (535.183.06).

Sun Oct 27 03:21:06 2024       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.183.06             Driver Version: 535.183.06   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce GTX 1060 6GB    Off | 00000000:01:00.0  On |                  N/A |
| 25%   43C    P0              25W / 120W |    476MiB /  6144MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      6923      G   /usr/lib/xorg/Xorg                          143MiB |
|    0   N/A  N/A      7045    C+G   ...libexec/gnome-remote-desktop-daemon       63MiB |
|    0   N/A  N/A      7096      G   /usr/bin/gnome-shell                         81MiB |
|    0   N/A  N/A      7798      G   firefox-esr                                 167MiB |
|    0   N/A  N/A      7850      G   /usr/lib/huiontablet/huiontablet             13MiB |
+---------------------------------------------------------------------------------------+

I want to integrate the online documentation a bit nicer into the editor itself. Is it somehow possible to query that page and get the contents of the searched-for entries?

Best case would be, if we can get the queried site content as JSON, that'd be nice, but very unlikely I think.

For about half a year I stuck with using 7B models and got a strong 4 bit quantisation on them, because I had very bad experiences with an old qwen 0.5B model.

But recently I tried running a ~smaller~ ~model~ like llama3.2 3B with 8bit quant and qwen2.5-1.5B-coder on full 16bit floating point quants, and those performed super good aswell on my 6GB VRAM gpu (gtx1060).

So now I am wondering: Should I pull strong quants of big models, or low quants/raw 16bit fp versions of smaller models?

What are your experiences with strong quants? I saw a video by that technovangelist guy on youtube and he said that sometimes even 2bit quants can be perfectly fine.

UPDATE: Woah I just tried llama3.1 8B Q4 on ollama again, and what a WORLD of difference to a llama3.2 3B 16fp!

The difference is super massive. The 3B and 1B llama3.2 models seem to be mostly good at summarizing text and maybe generating some JSON based on previous input. But the bigger 3.1 8B model can actually be used in a chat environment! It has a good response length (about 3 lines per message) and it doesn't stretch out its answer. It seems like a really good model and I will now use it for more complex tasks.

Some image I found when searching for deep pressure on duckduckgo. Never heard of the term before, buuuuuut seems interesting...

Also, from the same image, there is a hotdog

I used to do this as a child, alone..

And now I ask YOU, have you heard of this deep pressure term before? What are your experiences? (Now u be the entertainer! I'm done with pretending!)

Local friend I found! He goes Quark 🏳️‍🌈🏳️‍⚧️ <3

I'm really curious about which option is more popular. I have found, that format JSON works great even for super small models (e.g. Llama 3.2-1B-Q4 and Qwen-2.5-0.5B-Q4) which is great news for mobile devices!

But the strictly defined layout of function calling can be very alluring as well, especially since we could have an LLM write the layout given the full function text (as in, the actual code of the function).

I have also tried to ditch the formatting bit completely. Currently I am working on a translation-tablecreator for Godot, which requests a translation individually for every row in the CSV file. Works mostly great!

I will try to use format JSON for my project, since not everyone has the VRAM for 7B models, and it works just fine on small models. But it does also mean longer generation times... And more one-shot prompting, so longer first-token-lag.

Format JSON is too useful to give up for speed.