Kanał Linux Obiektywnie

This article will describe how to download an image from a (docker) container registry.

Manual Download of Container Images with wget and curl

Intro

Remember the good `'ol days when you could just download software by visiting a website and click "download"?

Even apt and yum repositories were just simple HTTP servers that you could just curl (or wget) from. Using the package manager was, of course, more secure and convenient -- but you could always just download packages manually, if you wanted.

But have you ever tried to curl an image from a container registry, such as docker? Well friends, I have tried. And I have the scars to prove it.

It was a remarkably complex process that took me weeks to figure-out. Lucky you, this article will break it down.

Examples

Specifically, we'll look at how to download files from two OCI registries.

1. Docker Hub
2. GitHub Packages

Terms

First, here's some terminology used by OCI

1. OCI - Open Container Initiative
2. blob - A "blob" in the OCI spec just means a file
3. manifest - A "manifest" in the OCI spec means a list of files

Prerequisites

This guide was written in 2024, and it uses the following software and versions:

1. debian 12 (bookworm)
2. curl 7.88.1
3. OCI Distribution Spec v1.1.0 (which, unintuitively, uses the '/v2/' endpoint)

Of course, you'll need 'curl' installed. And, to parse json, 'jq' too.

sudo apt-get install curl jq

What is OCI?

OCI stands for Open Container Initiative.

OCI was originally formed in June 2015 for Docker and CoreOS. Today it's a wider, general-purpose (and annoyingly complex) way that many projects host files (that are extremely non-trivial to download).

One does not simply download a file from an OCI-complianet container registry. You must:

1. Generate an authentication token for the API
2. Make an API call to the registry, requesting to download a JSON "Manifest"
3. Parse the JSON Manifest to figure out the hash of the file that you want
4. Determine the download URL from the hash
5. Download the file (which might actually be many distinct file "layers")

One does not simply download from a container registry

In order to figure out how to make an API call to the registry, you must first read (and understand) the OCI specs here.

• https://opencontainers.org/release-notices/overview/

OCI APIs

OCI maintains three distinct specifications:

1. image spec
2. runtime spec
3. distribution spec

OCI "Distribution Spec" API

To figure out how to download a file from a container registry, we're interested in the "distribution spec". At the time of writing, the latest "distribution spec" can be downloaded here:

• https://github.com/opencontainers/distribution-spec/releases/tag/v1.1.0
• https://github.com/opencontainers/distribution-spec/releases/download/v1.1.0/oci-distribution-spec-v1.1.0.pdf

The above PDF file defines a set of API endpoints that we can use to query, parse, and then figure out how to download a file from a container registry. The table from the above PDF is copied below:

In OCI, files are (cryptically) called "blobs". In order to figure out the file that we want to download, we must first reference the list of files (called a "manifest").

The above table shows us how we can download a list of files (manifest) and then download the actual file (blob).

Examples

Let's look at how to download files from a couple different OCI registries:

1. Docker Hub
2. GitHub Packages

Docker Hub

To see the full example of downloading images from docker hub, click here

GitHub Packages

To see the full example of downloading files from GitHub Packages, click here.

Why?

I wrote this article because many, many folks have inquired about how to manually download files from OCI registries on the Internet, but their simple queries are usually returned with a barrage of useless counter-questions: why the heck would you want to do that!?!

The answer is varied.

Some people need to get files onto a restricted environment. Either their org doesn't grant them permission to install software on the machine, or the system has firewall-restricted internet access -- or doesn't have internet access at all.

3TOFU

Personally, the reason that I wanted to be able to download files from an OCI registry was for 3TOFU.

Verifying Unsigned Releases with 3TOFU

Unfortunaetly, most apps using OCI registries are extremely insecure. Docker, for example, will happily download malicious images. By default, it doesn't do any authenticity verifications on the payloads it downloaded. Even if you manually enable DCT, there's loads of pending issues with it.

Likewise, the macOS package manager brew has this same problem: it will happily download and install malicious code, because it doesn't use cryptography to verify the authenticity of anything that it downloads. This introduces watering hole vulnerabilities when developers use brew to install dependencies in their CI pipelines.

My solution to this? 3TOFU. And that requires me to be able to download the file (for verification) on three distinct linux VMs using curl or wget.

⚠ NOTE: 3TOFU is an approach to harm reduction.

It is not wise to download and run binaries or code whose authenticity you cannot verify using a cryptographic signature from a key stored offline. However, sometimes we cannot avoid it. If you're going to proceed with running untrusted code, then following a 3TOFU procedure may reduce your risk, but it's better to avoid running unauthenticated code if at all possible.

Registry (ab)use

Container registries were created in 2013 to provide a clever & complex solution to a problem: how to package and serve multiple versions of simplified sources to various consumers spanning multiple operating systems and architectures -- while also packaging them into small, discrete "layers".

However, if your project is just serving simple files, then the only thing gained by uploading them to a complex system like a container registry is headaches. Why do developers do this?

In the case of brew, their free hosing provider (JFrog's Bintray) shutdown in 2021. Brew was already hosting their code on GitHub, so I guess someone looked at "GitHub Packages" and figured it was a good (read: free) replacement.

Many developers using Container Registries don't need the complexity, but -- well -- they're just using it as a free place for their FOSS project to store some files, man.

Niedawno spadł mi komputer, ale zdawało się, ze wszystko jest Ok. Kilka dni później robiłem aktualizacje - bardzo długą i wielokrotnie trzeba było potwierdzać, by aktualizowano „domyślnie”.

Od tego czasu kompa nie włączałem - dziś wyskakuje mi takie okno jak na zdjęciu.

Wtf?

Pod linkiem bardzo ciekawa dyskusja o tym jak uczyć dzieciaki Linuxa, trochę przykładów i sporo inspiracji.

Hello, I would like to make sure that this Laptop supports Fedora. I found out that it only had RHEL and Ubuntu Certification (https://ubuntu.com/certified/202309-32038), but not Fedora. So my question is will Fedora 100% work on it without any problems? https://psref.lenovo.com/Product/ThinkPad/ThinkPad_P16v_Gen_1_AMD I plan to use Fedora Onyx or Fedora Kinoite. Product id: 21FE000VCK

Jestem laikiem. Postanowiłem napisać ten post, bo po raz 147 wkurzam się na ten system.

Jestem słaby w komputery i sprawy informatyczne i równie słaby z języków obcych. Wielokrotnie miałem różne problemy z linuxem - od braku painta (xD) do braku umiejętności ściągania napisów do filmu.

Ktoś mądrzejszy ode mnie powiedział mi kiedyś, że nie rozumie, po co tacy ludzie jak ja (którzy robią na komputerze bardzo podstawowe rzeczy) mają linux. Wcześniej powiedziano mi, że hasło dostępu do linux jest złamać znacznie trudniej, niż do windows.

Gdzie jest prawda? Czy mogę w prosty sposób zabezpieczyć komputer mając windows, by był zabezpieczony tak, jak hasłem na linux (mam jedno hasło).

Piszę to teraz, bo jak nie napiszę teraz, to mi przejdzie i nigdy tego wątku nie ruszę, a chodzi mi po głowie od dawna.

Jedynym plusem, jaki znam, w kwestii używania linuxa jest to, że nie ściąga (chyba) mi wirusów i nie muszę co jakiś czas robić formata.

Ktoś wie jak zmusić sk21 od skyloong do współpracy z Linux(linuksem/linuxem?). Zakupiłem tego numpada jeszcze przed migracją z Windows na Linuxa. Jest to wersja z możliwością podłączenia bluetooth, niestety nawet po kablu nie system nie reaguje na wciśnięcia klawiszy. Co ciekawe system wykrywa, że jakaś dodatkowa klawiatura jest podłączona. Zna ktoś remedium na ten problem czy może raczej mam się pogodzić z porażką?

W sobotę wyszła wersja Release Candidate GNOME 45. W tym momencie nie powinny się już pojawić żadne zmiany w UI, można więc sprawdzić jak będzie wyglądać nowa odsłona mojego ulubionego biurka.

Każda dystrybycja jest niezmienna, gdy nazwiesz root directory 'home'.

change my mind

Red Hat (teraz IBM) znany z przejmowania i ubijania projektów open source, chyba poczuł się na tyle bezkarnie aby odważyć się zamknąć kod źródłowy RHEL oraz zastrzec że nie może być dystrybuowany co jest pogwałceniem licencji GPL Linuksa.

Hej, wiem że brave ma wiele problemów, ale dzisiaj na przykład zdałem sobie sprawę że wprowadzili naprawdę przyzwoite webappy.

Kolejny raz widzę, że projekty OpenSource szybciej integrują się z nowymi trendami w technologii. Ciekawe jak podobne projekty będą wyglądały za kilka lat.

Już teraz widzialem pluginy do Bender-a, które za pomocą sztucznej inteligencji wykonują powtarzalne operacje, które użytkownik może definiować za pomocą złożonych zdań w języku angielskim.

What a time to be alive!

Dzisiaj na blogasku wykorzystuję przykład Tuby, mojego ulubionego klienta fediwersum, by zachęcić was do zaangażowania się w rozwój wolnego oprogramowania.