Debian packaging - just the very basics

johnraff · 2024-06-11 08:23:40

There is a mountain of documentation about Debian packaging, but even the "tutorials" and "introductions" seem to assume people already have an idea what's going on. A recent discussion brought this out - @hhh found a ream of documents. Especially, the Debian Wiki often has obsolescent or contradictory advice, and the whole thing can be pretty overwhelming. Here I'm going to try to describe the basics of the basics, in particular how it applies to BunsenLabs.

What's a Debian package?

All those something.deb files that can be installed on your computer. Usually apt takes care of them, downloading from a repository (which you've set in /etc/apt/sources...) installing, upgrading and removing as needed, but you can also download a .deb file and install it directly with gdebi, apt or dpkg. (NB don't do that unless you know what you're doing.)

Deb files that apt has downloaded are kept for a while in /var/cache/apt/archives so you can copy one from there to play with. Right-click > Open With > Archive Manager and you can see what's inside without trying to install it again.
eg bunsen-docs will show control.tar.gz and data.tar.gz.

Open data.tar.gz to see all the files that would be installed, nicely arranged in a directory tree, starting with usr/share/ in this case. Along with the package specific stuff, all packages will install /usr/share/doc/<packagename>/changelog.Debian.gz copyright and usually README

Open control.tar.gz to see the files that keep things organized, in the case of bunsen-docs it's just control and md5sums, both readable in any text editor. Any dependencies would be listed in control but bunsen-docs has none.

Debian package source

Such .deb files are ready to be distributed in a repository, and if their content is only text files then it can easily be checked by anyone, but if a package contains binary files then the source code has to be available somewhere. Along with that, a Debian package's source also contains other information that isn't needed in the "binary" .deb file. Have a look at bunsen-docs' source code on GitHub: https://github.com/BunsenLabs/bunsen-docs
The bunsen-docs.deb file is built from that source code using one of a whole bunch of possible Debian utilities. That's what "building" a package means - making a .deb file from the package source.

There's no particular rule about how the files in the source repository have to be organized - it's the job of the files in the debian/ subfolder to put them correctly in the .deb file. In fact there's no rule that Debian source has to be on GitHub (many would prefer somewhere not owned by Microsoft like GitLab, though that's a different discussion...), or even in a git repository at all.

The traditional Debian way to provide source code is in two tarballs *.orig.tar.gz and *debian.tar.gz orig is the original "upstream" code and debian is made by the Debian package maintainer. In the case of BL, both upstream and maintainer are from the same organization and often the same person, but the general idea is that Debian maintainers don't touch the original upstream code in orig but do all their tweaking in debian (ie inside the debian/ subfolder).

Here's what BL have for bunsen-docs: https://pkg.bunsenlabs.org/debian/pool/ … nsen-docs/ You can see the .deb files, the source files and some .dsc files which have metadata.

If you have a look inside the debian/ subdirectory on GitHub you'll see a lot more than what was in the final bunsen-docs.deb file. All that stuff is to get what's "outside" in the main repo properly installed on a user's computer when they install the .deb file. They're all regular text files so you can easily look at what's there.

Compared with many Debian packages bunsen-docs is really simple - in fact none of the BunsenLabs packages are that complicated, so I'm just going to pass over the more difficult stuff in this thread. Next post will be about those files in debian/

johnraff · 2024-06-13 06:41:28

First a few definitions:

upstream means the people who developed a piece of software. Debian itself makes only a few packages (eg dpkg, apt) and everything else in the repositories comes from some "upstream".

A Debian maintainer is the person who puts some software in a Debian package so it can go in the repositories. There are other people in the Debian organization: developers, groups, FTP masters, etc etc, but the first people users like us will encounter (eg when reporting bugs) are likely to be the maintainers.

binary in the Debian context means a *.deb file that is ready to host on a repository and install on a user's computer with apt and dpkg. This goes in contrast with the package source.

building in Debian means turning a package's source code into a binary .deb package. As an Open Source organization Debian are obliged to make the original source available somewhere so that anyone can build it themselves and get the same .deb package out of it.

Debianizing is the process of turning a piece of software into something that can be built into a Debian *.deb package, fit to be added to the repository. That "fitness" depends on a lot of things, mostly defined in the Debian Policy.

A tarball is the compressed archive (often *.tar.gz but also *.tar.xz etc) which holds either the original source code of a package from upstream, or the added debian/ directory from the maintainer. During the Debianization process a tarball from upstream - which might have any name - will be renamed ending orig.tar.gz (eg bunsen-docs_12.0.1.orig.tar.gz) but the content should be the same as the original tarball from upstream.

johnraff · 2024-06-13 08:22:52

So how does the debian/ directory turn the source code into a Debian package? It can be very, very complicated, but for BunsenLabs packages it's usually pretty simple.

For a start, our packages will not be uploaded into the official Debian repositories so in principle there's no need to follow Debian Policy at all, but in practice we at BL see the value of that policy and try to follow it anyway. There's a utility called lintian which checks for policy violations and issues warnings about fringe issues. Some of our packages get lintian warnings, and I think one or two even get errors, but we at least know the reasons in each case, and could fix them some time in the future. (Of course we're pretty sure there's no implication for users' safety in any of these.)

Another difference in BL's situation is that we're simultaneously upstream and maintainer. That means the original source code is totally under our control, making debianization easier. Maintainers sometimes find debian/ by itself is not enough to get a package's content into shape and have to edit the content of the orig tarball itself - that's done by patching, using patch files again inside debian/. We at BL don't have to worry about patching packages.

files inside debian/

There can be many, but the most important files, that must be present, are:

control which defines the package Depends and other relationships, so apt can install it correctly, along with other important metadata and information to show the user.

copyright this defines who owns the copyright and is essential for the open source licence to be valid.

changelog defines the current package version - important for handling upgrades - along with some info about what actually changed in the latest version. Apart from the original package source online somwehere, users can check out their local copy which will have been installed as /usr/share/doc/<packagename>/changelog.gz

rules This is a makefile. (Please don't ask me about makefiles, but remind me to find a nice tutorial and read it some day.) For now, we can be happy that there's a modern app called debhelper that automates all the tasks of building a package from source, so all a rules file needs to have is:

#!/usr/bin/make -f

%:
< tab >dh $@

(NOTE: that tab cannot be replaced by spaces. This is a makefile thing.)
That tells debhelper to do everything. (Those things can be controlled by installing other files in /debian)

Here's a more complicated rules file: https://salsa.debian.org/multimedia-tea … type=heads
Here's an exhaustive guide to those required files in debian/: https://www.debian.org/doc/manuals/main … eq.en.html

Some other files in debian/

There are many possible ones, but here are some of the more common:
(These filenames will be prefixed with <packagename>. if the source code builds more than one binary, which is often the case.)

install Just a list of files to install from the source code, and where to put them. For non-compiled packages with text files this is the way to do it.

links Likewise for symlinks - if you need to add some at build-time.

dirs Similarly for directories. There's no need to list directories here to hold files - they're made automatically - just empty directories, so you won't often see this. Bunsen-configs has one. NOTE git ignores empty directories, so a common trick to get it to see them is to put a .gitignore file inside. Debhelper in turn has to be told not to put .gitignore files in the package...

docs Files you want to be installed in /usr/share/doc/<packagename>/ along with changelog, often README if there is one.

There are others, but let's stop there.
Here's a detailed list: https://www.debian.org/doc/manuals/main … er.en.html

I hope that's enough to at least get an idea what's happening in Debian packages.

More reading

Debian New Maintainers' Guide The canonical reference. I always come back here when I get stuck.

"Simple Example" by Osamu Aoki - maybe not so simple but a good introduction.

These three on the Debian Wiki:
Packaging Intro
BuildingTutorial
Packaging Learn

And this more BL-oriented developers' discussion here on the forum.

Feel free to post questions about packaging for BL in the above discussion, and about general Debian packaging here on this thread.
We'll at least try to find some helpful web link.

Last edited by johnraff (2024-12-06 02:34:59)

sniper-kun_2 · 2024-09-15 21:02:45

Hi. My question is more about Debian than BL, hope I can ask here.
When I install testing/trixie I use the weekly iso from here: https:// cdimage . debian . org/cdimage/weekly-builds/amd64/iso-dvd/
When I look into /etc/apt/sources.list I see both backports and updates lines are enabled as per the trixie branch BUT! if you look here https:// wiki . debian . org/DebianTesting under How to upgrade to Debian (next-stable) Testing it says:

3. Remove, disable or comment out any other stable-specific apt sources, like *-backports or *-updates.

I was wondering what's the best way to handle this, leave it as is as per what the iso installs or really disable the backports and updates lines? What do you recommend?

hhh · 2024-09-16 05:46:25

https://gist.github.com/hakerdefo/1599c … 48d9c6c71d

https://www.debian.org/doc/manuals/secu … rt-testing

If your on sid/unstable, you only need the sid repo.

From my days with aptosid (now Siduction), I recommend using the sid repo. If a package that breaks the system is broken in sid, it usually gets fixed within a day or two, maybe a week at most. If that package made it to sid, it could take much longer to get the fix in...

https://www.debian.org/doc/manuals/debi … tml#s3.1.7

Last edited by hhh (2024-09-16 05:52:41)

johnraff · 2024-09-16 05:52:38

sniper-kun_2 wrote:

Hi. My question is more about Debian than BL, hope I can ask here.

This thread is about how software is packaged for Debian, so your question is somewhat off-topic, but anyway I'll try to give a short answer. If you want to extend the discussion please start a new thread - this " BL & General Linux Discussion" section is fine.

When I install testing/trixie I use the weekly iso from here: https:// cdimage . debian . org/cdimage/weekly-builds/amd64/iso-dvd/
When I look into /etc/apt/sources.list I see both backports and updates lines are enabled...

You're talking about what you find in the system after it's installed, right? Not what you find when you extract the iso and look what's inside it?

BUT! if you look here https:// wiki . debian . org/DebianTesting under How to upgrade to Debian (next-stable) Testing it says:
3. Remove, disable or comment out any other stable-specific apt sources, like *-backports or *-updates.
I was wondering what's the best way to handle this, leave it as is as per what the iso installs or really disable the backports and updates lines? What do you recommend?

Those instructions are for how to upgrade to Testing (currently Trixie) from Stable (currently Bookworm), and are good advice.
But you're already on Testing, so that doesn't apply.

That said, I think it's unusual for backports to be enabled out of the box, and in a Debian stable release they usually aren't. Remember, the weekly isos are for testing purposes by developers.

EDIT: what @hhh says is correct.

johnraff · 2024-09-16 05:54:19

hhh wrote:

If that package made it to sid, it could take much longer to get the fix in...

You meant "If that package made it to Testing..." maybe?

hhh · 2024-09-16 05:55:44

^ I edited my post, apparently there are update and backport repos for testing...

https://ftp.debian.org/debian/dists/testing-updates/

https://ftp.debian.org/debian/dists/testing-backports/

There's even a proposed-updates repo...

https://ftp.debian.org/debian/dists/tes … d-updates/

johnraff · 2024-09-16 06:37:56

^You said that a reason to use Sid over Testing was that it got fixes faster right? I think that's correct, so surely your sentence should have read:

If a package that breaks the system is broken in sid, it usually gets fixed within a day or two, maybe a week at most. If that package made it to Testing, it could take much longer to get the fix in...

No?

johnraff · 2024-09-16 06:39:46

But anyway, we're moving away from both sniper-kun_2's question and the Original Topic of this thread.

hhh · 2024-09-16 07:43:54

johnraff wrote:

^You said that a reason to use Sid over Testing was that it got fixes faster right? I think that's correct, so surely your sentence should have read:
If a package that breaks the system is broken in sid, it usually gets fixed within a day or two, maybe a week at most. If that package made it to Testing, it could take much longer to get the fix in...
No?

Yes. Oops.

Use sid, because breakage will get fixed faster than testing. Also you can monitor the Siduction forum to check the status of breakages...

https://forum.siduction.org/index.php?board=22.0

#1 2024-06-11 08:23:40