accessibility is honestly the first good use of ai. i hope they can find a way to make them better than youtube's automatic captions though.
this post was submitted on 09 Jan 2025
339 points (98.6% liked)
Opensource
1592 readers
631 users here now
A community for discussion about open source software! Ask questions, share knowledge, share news, or post interesting stuff related to it!
⠀
founded 1 year ago
MODERATORS
I know Jeff Geerling on Youtube uses OpenAIs Whisper to generate captions for his videos instead of relying on Youtube's. Apparently they are much better than Youtube's being nearly flawless. I would have a guess that Google wants to minimize the compute that they use when processing videos to save money.
I am still waiting for seek previews
I've been waiting for ~~this~~ break-free playback for a long time. Just play Dark Side of the Moon without breaks in between tracks. Surely a single thread could look ahead and see the next track doesn't need any different codecs launched, it's technically identical to the current track, there's no need to have a break. /rant
Perhaps we could also get a built-in AI tool for automatic subtitle synchronization?
I was part of that!
Thank you for your service
This is great timing considering the recent Open Subtitles fiasco.
Huh?
Open Subtitles now only allows 5 downloads per 24 hours per IP. You have to pay for more.
I don't mind the idea, but I would be curious where the training data comes from. You can't just train them off of the user's (unsubtitled) videos, because you need subtitles to know if the output is right or wrong. I checked their twitter post, but it didn't seem to help.
subtitles aren't a unique dataset it's just audio to text
They may have to give it some special training to be able to understand audio mixed by the Chris Nolan school of wtf are they saying.
No, if you have a center track you can just use that. Volume isn't a problem for a computer listening to it since they don't use the physical speakers.
I hope they're using Open Subtitles, or one of the many academic Speech To Text datasets that exist.
I know people are gonna freak out about the AI part in this.
But as a person with hearing difficulties this would be revolutionary. So much shit I usually just can’t watch because open subtitles doesn’t have any subtitles for it.
The most important part is that it’s a local ~~LLM~~ model running on your machine. The problem with AI is less about LLMs themselves, and more about their control and application by unethical companies and governments in a world driven by profit and power. And it’s none of those things, it’s just some open source code running on your device. So that’s cool and good.
Also the incessant ammounts of power/energy that they consume.
Curious how resource intensive AI subtitle generation will be. Probably fine on some setups.
Trying to use madVR (tweaker's video postprocessing) in the summer in my small office with an RTX 3090 was turning my office into a sauna. Next time I buy a video card it'll be a lower tier deliberately to avoid the higher power draw lol.
Running an llm llocally takes less power than playing a video game.
The training of the models themselves also takes a lot of power usage.
Source?
I don't have a source for that, but the most that any locally-run program can cost in terms of power is basically the sum of a few things: maxed-out gpu usage, maxed-out cpu usage, maxed-out disk access. GPU is by far the most power-consuming of these things, and modern video games make essentially the most possible use of the GPU that they can get away with.
Running an LLM locally can at most max out usage of the GPU, putting it in the same ballpark as a video game. Typical usage of an LLM is to run it for a few seconds and then submit another query, so it's not running 100% of the time during typical usage, unlike a video game (where it remains open and active the whole time, GPU usage dips only when you're in a menu for instance.)
Data centers drain lots of power by running a very large number of machines at the same time.
Yeah, transcription is one of the only good uses for LLMs imo. Of course they can still produce nonsense, but bad subtitles are better none at all.
load more comments
(1 replies)
And yet they still can't seek backwards
Iirc this is because of how they've optimized the file reading process; it genuinely might be more work to add efficient frame-by-frame backwards seeking than this AI subtitle feature.
That said, jfc please just add backwards seeking. It is so painful to use VLC for reviewing footage. I don't care how "inefficient" it is, my computer can handle any operation on a 100mb file.
If you have time to read the issue thread about it, it's infuriating. There are multiple viable suggestions that are dismissed because they don't work in certain edge cases where it would be impossible for any method at all to work, and which they could simply fail gracefully for.
That kind of attitude in development drives me absolutely insane. See also: support for DHCPv6 in Android. There's a thread that has been raging for I think over a decade now
I now know more about Android IPv6 than ever before
Same for simply allowing to pause on click... Luckily extension exists but it's sad that you need one.
You can easily write a video reader using openCV that would be able to read backward using cache
My experience with generated subtitles is that they're awful. Hopefully these are better, but I wish human beings with brains would make them.
subtitling by hand takes sooooo fucking long :( people who do it really are heroes. i did community subs on youtube when that was a thing and subtitling + timing a 20 minute video took me six or seven hours, even with tools that suggested text and helped align it to sound. your brain instantly notices something is off if the subs are unaligned.
Oh shit, I knew it was tedious but it sounds like I seriously underestimated how long it takes. Good to know, and thanks for all you've done.
Sounds to me like big YouTubers should pay subtitlers, but that's still a small fraction of audio/video content in existence. So yeah, I guess a better wish would be for the tech to improve. Hopefully it's on the right track.
load more comments
(1 replies)
load more comments
(1 replies)
All hail the peak humanity levels of VLC devs.
FOSS FTW
Et tu, Brute?
VLC automatic subtitles generation and translation based on local and open source AI models running on your machine working offline, and supporting numerous languages!
Oh, so it's basically like YouTube's auto-generatedd subtitles. Never mind.
Hopefully better than YouTube's, those are often pretty bad, especially for non-English videos.
Youtube's removal of community captions was the first time I really started to hate youtube's management, they removed an accessibility feature for no good reason, making my experience with it significantly worse. I still haven't found a replacement for it (at least, one that actually works)
Same here. It kick-started my hatred of YouTube, and they continued to make poor decision after poor decision.
and if you are forced to use the auto-generated ones remember no [__] swearing either! as we all know disabled people are small children who need to be coddled!
load more comments
(13 replies)
load more comments
(7 replies)
view more: next ›