Package management, repositories, metadata

I found the following post from Pascal Bleser talking about repositories, metadata, packages, and so on. All this started due to the problems with package management in SUSE 10.1:

The generic and most appropriate term is “repository”.
Well, the repository is on the server-side, for example:

  1. http://ftp.gwdg.de/pub/linux/misc/suser-guru/rpm/10.1

    Is a yast2 repository

  2. http://ftp.gwdg.de/pub/linux/misc/suser-guru/rpm/10.1/RPMS

    Is an RPM-MD (/repomd/yum, see below) repository

  3. http://ftp.gwdg.de/pub/linux/suse/apt/SuSE/10.1-i386

    Is an APT-RPM repository

The following terms are used by different package managers and are all references to repositories.

Such references usually contain information such as:

  • the base URL, where to retrieve the metadata and the packages
  • a name, but also additional information, depending on what the package manager is capable of handling.
  • YaST2 calls them “installation sources”
  • ZMD/rug calls them “catalogs”
  • smart calls them “channels” (AFAICR, Red Carpet calls them “channels” as well)
  • apt-rpm calls them “sources”
  • yum calls them “repos” (repositories)

but they’re all the same thing: references to package repositories.

Package repositories are made of “metadata”.
That metadata contains various information about the RPM packages that
are available in a repository, such as:

  • package name, version, release, target architecture, distribution
  • summary, description, license, project website
  • what it requires
  • what it provides, and the list of files contained in the package

That metadata information is needed by package managers like libzypp,
ZMD/rug, apt-rpm, smart, yum, … to be able to compute how to perform
package installations, upgrades, removals (the requires/provides
information is particularly important here).

There are different metadata formats:

  • RPM-MD (RPM-MetaData), also called “repomd” (Repository MetaData), also called “yum” (because its the native format for the yum package manager)
  • yast2
  • apt-rpm (for RPM) and apt-deb (for DEB)
  • Red Carpet (also called Open Carpet)
  • RPM-HDL (RPM Header List)
  • URPMI (for Mandriva’s package manager)
  • slack, for Slackware’s “package manager” and maybe a few others.

So, unfortunately, there’s more or less a metadata format for every single package manager. It seems the NiH (*) syndrom has spread a lot amongst package manager developers.
(*) “Not invented Here”

Nowadays, most package managers seem to head for RPM-MD support, which is more or less becoming a standard:

  • YaST2 supports it in SUSE Linux 10.0
  • libzypp supports it in SUSE Linux 10.1
  • yum supports it, obviously, it’s the yum-specific format 😉
  • apt-rpm supports it too, since one or two releases (not the version shipped with SUSE Linux 10.1 though)
  • smart supports it, of course 😉 (more information about that below)

RPM-MD is represented in XML – actually, the repository metadata is stored in gzipped XML files, in a “repodata” subdirectory on the server (*), with the following files:

  • repomd.xml: main repository file, very small, contains references to the others, as well as checksums and timestamps
  • primary.xml.gz: contains the most important information: list of packages (with version, release, architecture), what it requires, size of the package, summary, description, etc….
  • filelists.xml.gz: contains the list of files that are included in the packages
  • other.xml.gz: not used by all package managers, it contains the changelog information of every package

(*) for an example:
http://ftp.gwdg.de/pub/linux/misc/suser-guru/rpm/10.1/RPMS/repodata/

Read here for more information on RPM-MD:
http://linux.duke.edu/projects/metadata/

smart is a notable exception, because it supports *all* of the formats
above (and it has no smart-specific format).

When you add a smart “channel” (repository information), here are the
types you can use (excerpt from “smart channel –help”):
apt-deb – APT-DEB Repository
apt-rpm – APT-RPM Repository
deb-dir – DEB Directory
red-carpet – Red Carpet Channel
rpm-dir – RPM Directory
rpm-hdl – RPM Header List
rpm-md – RPM MetaData
slack-site – Slackware Repository
up2date-mirrors – Mirror Information (up2date format)
urpmi – URPMI Repository
yast2 – YaST2 Repository

(note, I removed the *-sys channels, they represent the packages
currently installed in the RPM database (or DEB database on Debian, etc…))

And yes, you can even mix them. In my list of channels for smart, I mix
yast2, rpm-md and apt-rpm repositories.

smart is even independent of the package subsystem, as it works for RPM (SUSE/Redhat/Fedora/Mandriva), Deb (Debian/Ubuntu) and Slackware. So you could install a .deb Debian/Ubuntu package on SUSE Linux with smart, but it wouldn’t make any sense because it would be stored in a DEB package database… that knows nothing about the RPM database, etc… (so that usage is pretty pointless, and not the goal of smart anyway). Actually it’s interesting because you can use smart (unmodified) on RPM, Deb and Slackware based distributions.

Advertisements

4 thoughts on “Package management, repositories, metadata

  1. Aún a riesgos de parecer un anuncio de detergentes, mis problemas con los repositorios de Suse y las dependencias de paquetes han desaparecido con Smart… precisamente hace un par de días escribí un post cantando mis loas y alabanzas a esta herramienta. ¿Sabes si funciona igual de bien con otras distribuciones?

  2. Gracias por el comentario, José María 🙂

    La verdad es que aún me queda mucho que aprender sobre el manejo de paquetes y repositorios en SUSE.

    He sido siempre un incondicional de Red Hat y Fedora, pero desde la aparición de Xgl, cambié a SUSE 10.1. Sin embargo, mis experiencias con el manejo de paquetes son bastante lamentables: ZMD es horriblemente lento y funciona esporádicamente, YaST2 tarda una eternidad en refrescar los repositorios, y no consigo que YUM descargue los metadatos (suele fallar con errores que indican que las cabeceras están incompletas o corrompidas).

    No he tenido el placer de usar Smart aún, pero espero hacerlo en breve. Cuando eso ocurra, intentaré escribir mis experiencias con él.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s