Getting into Debian packages

Bearded_Blunder · 2024-06-08 04:00:18

MOD EDIT:

johnraff wrote:

Topic split off from Wayland and BunsenLabs so it wouldn't derail the Wayland discussion.

johnraff wrote:

Fact is, it would be quite nice to have somebody else working on BL Debian packaging... smile

Don't look at me, I can hack at some packages, but the whole doing it properly thing I haven'y (at least yet) learned,

Last edited by johnraff (2024-06-11 02:36:10)

hhh · 2024-06-09 19:39:18

johnraff wrote:

Fact is, it would be quite nice to have somebody else working on BL Debian packaging...

What kind of help would you like?

johnraff · 2024-06-10 01:01:55

^First, being able to do basic stuff on git would be needed, but editing an existing Debian package is much easier than making a new one from scratch. Eg, if enough improvements to some package have built up in the git repo from various contributors, then it becomes time for a new version to be released. The basic thing there is to edit debian/changelog which is done via the utility dch, perhaps also needing an update of the debian/copyright year, possibly checking over the other files in debian/... So if someone got up to speed on that stuff, and got a package ready to be built and uploaded that would be a help.

Actually building and uploading packages once the source code is ready isn't hard for me now because I've got it automated with scripts. A second uploader would need to get some GPG signing keys set up, and become familiar with the build tools (I use pbuilder) and repo management (I use reprepro), so that stuff might be further down the road. It's not all that difficult, but there's quite a lot of it. I had @nobody to help me get on top of it, and would be happy to pass on the little I know to someone who was interested.

johnraff · 2024-06-10 03:56:10

...so, I would suggest to someone who wanted to get into BL's Debian packaging that they choose one of our packages and become its maintainer. I'll be happy to help at first. Most of our packages are quite simply put together, so just pick one that they're interested in, and already know something about what's in there.

johnraff · 2024-06-10 05:10:49

hhh wrote:

^ Right, I've already done some backports, so that seems like something I could handle. Start a new thread for this?

Great! Perhaps you'd like to take on one of the graphics packages like bunsen-themes? Maybe a thread in the dev section?
https://forums.bunsenlabs.org/viewforum.php?id=28

ie this thread.

Last edited by johnraff (2024-06-11 02:44:02)

Bearded_Blunder · 2024-06-10 05:53:15

johnraff wrote:

^First, being able to do basic stuff on git would be needed

Currently utterly clueless about git.

but editing an existing Debian package is much easier than making a new one from scratch. Eg, if enough improvements to some package have built up in the git repo from various contributors, then it becomes time for a new version to be released. The basic thing there is to edit debian/changelog which is done via the utility dch, perhaps also needing an update of the debian/copyright year, possibly checking over the other files in debian/...

Closest I've got is backporting a package (maybe two) locally, using an ancient guide found on the Debian wiki, I think, so I've used dch in it's most basic way about twice.

So if someone got up to speed on that stuff, and got a package ready to be built and uploaded that would be a help.

Hmmm, not exactly sure how up to speed I could get rapidly, nor can I guarantee circumstances mightn't cause me to vanish again, Mother & Sister both in fragile health.

Actually building and uploading packages once the source code is ready isn't hard for me now because I've got it automated with scripts. A second uploader would need to get some GPG signing keys set up, and become familiar with the build tools (I use pbuilder) and repo management (I use reprepro), so that stuff might be further down the road. It's not all that difficult, but there's quite a lot of it. I had @nobody to help me get on top of it, and would be happy to pass on the little I know to someone who was interested.

I did have a quick look at the documentation I could find regarding using pbuilder before the last time life intervened, & setting up a local repo for my own use, can't even remember what I looked into for potentially managing that, It will have been a quick documentation check on the choices to see which looked simplest.

That only got as far as looking at the documentation, it looked a bit daunting, it also looked a tad resource heavy given the newest machine I have was purchased about the month before Windows 7 released to the public, it's still wearing it's COA sticker for Windows Vista.

johnraff · 2024-06-10 06:07:52

^@B_B thanks for your interest anyway!

As for the build/upload part with pbuilder & reprepro I'm fine to continue doing that for the moment. Now everything's set up it's not really time-consuming. (Eventually Bus Factor is a thing to consider though.)

hhh · 2024-06-10 15:43:47

@johnraff, if you could point me to or outline a decent workflow for package build, that would help. For Debian there is this...

https://wiki.debian.org/BuildingTutorial

... but I'm wondering how up to date it is. There's this...

https://www.debian.org/doc/manuals/maint-guide/
https://www.debian.org/doc/manuals/main … ld.en.html

... and I'm willing to take the time to grok it but any shortcuts would help.

There's this...

https://wiki.debian.org/Packaging

...which immediately points to this...

https://wiki.debian.org/Packaging/Intro … nPackaging

... and this...

https://wiki.debian.org/Packaging/Learn

... but there are also this and this...

https://wiki.debian.org/SimplePackagingTutorial

https://www.debian.org/doc/manuals/pack … ial.en.pdf

... and I could have sworn last night that one of those links said that there was a more modern, efficient way of packaging using a different program but I can't find that reference at the moment.

Ah, here it is...

https://wiki.debian.org/HowToPackageForDebian

... which suggests this and has a ToDo about writing a new wiki page for it...

https://packages.debian.org/search?keyw … ildpackage

So, which to start with that will get me close to your workflow?

As to Hyprland, I need to do a new install to double check my steps and create a working tutorial, will do (but in no hurry so as to get it right.)

johnraff · 2024-06-11 02:31:56

johnraff wrote:

Fact is, it would be quite nice to have somebody else working on BL Debian packaging...

Topic split off from Wayland and BunsenLabs so it wouldn't derail the Wayland discussion.

johnraff · 2024-06-11 04:28:47

hhh wrote:

@johnraff, if you could point me to or outline a decent workflow for package build, that would help. For Debian there is this...
(lots of stuff)
So, which to start with that will get me close to your workflow?

First off, I'll fork off yet another thread in BL & General Linux Discussion with generic info on packaging that I've scraped up so far, FWIW. Others are free to contribute of course.

Then I'll come back here with some specific suggestions for our case here in the BL team.

EDIT I've written the first post in that thread here Debian packaging - just the very basics but maybe the next one - about the files in debian/ will have to wait till tomorrow.

---
Everyone has their own approach to learning new stuff, but to date mine has been to take something that's already working and tweak it till something breaks. Then look up in the docs about the particular bit I was hacking on, find out how to fix it, move on to breaking something else... Others prefer to start at page 1 and learn the basics first.

Last edited by johnraff (2024-06-11 08:27:47)

johnraff · 2024-06-11 05:01:17

Anyway, I'd like to emphasize the difference between:
1) writing/maintaining the source code of a package, and
2) building that source into a "binary" .deb file that can be installed.

(2) I can handle OK for BL atm because I've written a couple of scripts that make sure I don't forget anything (usually ) and all the gpg signing keys etc etc are set up nicely. For a long time @nobody was doing this for us, and I was just providing the package code written at stage (1).

But (1), ie maintaining our code, is where a bit of help wouldn't hurt.

Of course (2) can be interesting too, eg if you wanted to package up some original scripts or backports.

Also, if you're tweaking the source code of a package at stage (1) then maybe before pushing it to GitHub for someone (me atm) to build and upload to the repo, you might want to try building it yourself to make sure it's OK. So a bit of knowlege of simple building methods wouldn't hurt.

Last edited by johnraff (2024-06-11 05:59:16)

johnraff · 2024-06-11 08:59:40

Anyway, here's my take on the docs you found:

hhh wrote:

@johnraff, if you could point me to or outline a decent workflow for package build, that would help. For Debian there is this...
https://wiki.debian.org/BuildingTutorial

Not bad in fact, but I'd regard it as background reading rather than a recipe. I think it's a bit distant from a BL maintainer's workflow. (And I've never used dget.)

There's this...
https://www.debian.org/doc/manuals/maint-guide/
https://www.debian.org/doc/manuals/main … ld.en.html
... and I'm willing to take the time to grok it but any shortcuts would help.

Yes that's huge but it's what I come back to when I want to check out something. Seems closest to the canonical reference to me. I would skip that build.en.html page for now though. I would skim the introductory pages, then maybe see what you make of the stuff in debian/
https://www.debian.org/doc/manuals/main … eq.en.html
https://www.debian.org/doc/manuals/main … er.en.html

There's this...
https://wiki.debian.org/Packaging

Put away for future reference...

...which immediately points to this...
https://wiki.debian.org/Packaging/Intro

which is quite good I thought.

... and this...
https://wiki.debian.org/Packaging/Learn

I'd skip that for now.

... but there are also this and this...
https://wiki.debian.org/SimplePackagingTutorial

OK I guess.

https://www.debian.org/doc/manuals/pack … ial.en.pdf

Maa, skip that one too.

... and I could have sworn last night that one of those links said that there was a more modern, efficient way of packaging using a different program but I can't find that reference at the moment.
Ah, here it is...
https://wiki.debian.org/HowToPackageForDebian

Not bad for background reading.

... which suggests this and has a ToDo about writing a new wiki page for it...
https://packages.debian.org/search?keyw … ildpackage

No need to worry about git-buildpackage for BL purposes IMO.

So, which to start with that will get me close to your workflow?

I'll come back with a breakdown of my current workflow when I've finished that tutorial thread. I kind of got hyped-up writing it and it might take another couple of days...

...and I'll definitely put in some of the links you found.

Last edited by johnraff (2024-06-11 09:00:56)

micko01 · 2024-06-12 10:56:42

This tutorial by Osamu Aoki <osamu@debian.org> is both simple and mildly entertaining (sense of humour dependent ). It's the first one I read and I still refer to it from time to time.

An example:

$ vim debhello-0.0/debian/control
 ... hack, hack, hack, ...
 $ cat debhello-0.0/debian/control
Source: debhello
Section: devel
Priority: optional
Maintainer: Osamu Aoki <osamu@debian.org>
Build-Depends: debhelper-compat (= 13)
Standards-Version: 4.5.1
Homepage: https://salsa.debian.org/debian/debmake-doc
Rules-Requires-Root: no

Package: debhello
Architecture: any
Multi-Arch: foreign
Depends: ${misc:Depends}, ${shlibs:Depends}
Description: Simple packaging example for debmake
 This Debian binary package is an example package.
 (This is an example only)

hhh · 2024-06-12 21:52:11

^, ^^, OK, thanks for the suggestions gents!

johnraff · 2024-06-13 03:16:51

micko01 wrote:

This tutorial by Osamu Aoki <osamu@debian.org> is both simple and mildly entertaining (sense of humour dependent ).

That's excellent! I glanced at that debmake-doc manual the other day but missed this nice "simple" example on page 4.
I'll add a link to the tutorial I'm half-way through writing, although for BL purposes it could be quite a bit simpler still. The vast majority of our stuff is in text files - or images - and needs no C compiling or the like. But it's nice to read more about the modern package tools - debmake didn't exist when I started, I used dh_make but soon moved to:

Osamu Aoki wrote:

Many packages are packaged using only a text editor while imitating how other similar packages are packaged and consulting how the Debian policy requires us to do. This seems to me the most popular method for the real-life packaging activity.

I note he didn't mention pbuilder along with sbuild, debuild and other build tools. sbuild is the "official" Debian builder but pbuilder is also used by a lot of people (including me). I understand sbuild to be a bit harder to configure but I haven't yet tried it at all.

Anyway, thanks for finding this! Aoki-san really seems to cover all the important issues.

johnraff · 2024-06-13 08:25:26

That general background tutorial is done, as far as I think it can be. If you see anything that needs correcting or changing, please say.
https://forums.bunsenlabs.org/viewtopic.php?id=9013

johnraff · 2024-06-14 03:52:02

Now down to specifics of package maintainance for BunsenLabs.

This is the overall flow I'm following now:

1) Try out stuff at home, outside of git. This isn't necessary for small tweaks of course.

2) Edit files in the git repo, and do 'git commit' as appropriate.

3) When there are enough changes in the repo to justify a package upgrade, update the changelog with dch.

4) Build the needed binary etc files for the upload to the server, with pbuilder.

5) Add the built package files to a local Debian package repository, managed by reprepro.

6) Synchronize the updated repository with what's on the server, with rsync.

Originally 4) 5) & 6) were done on the server after uploading the source files, which is the Debian way (using different tools), but maintaining a local repository and synchronizing it with rsync is easier to manage when all three steps are being done by the same person, as is the case now.

I'll expand a bit below.

johnraff · 2024-06-14 04:09:56

git

Git is great for working as a team on a single body of code - it makes it easy for everyone to keep in sync. But it's also good for a single developer to keep track of a complicated job. If you commit often and keep individual commits focussed on a single issue, then it's easy to go back and undo something you did a while ago without bringing the whole thing crashing down, and to borrow commits from one branch to re-use in another.

In git, branches are cheap. Creating a new branch is instantaneous, and takes up no extra disk space until the branches diverge. If you want to try something tricky you can 'git checkout -b tempbranch' and do your work on tempbranch (any name is OK) without damaging the main branch. If it goes pear-shaped go back to main and'git branch -d tempbranch' will delete it. OTOH if all goes well, go back to main and 'git merge tempbranch' will merge it into main. Any time you're not sure, just make a new branch and work there.

I'm not particularly recommending these, but I have some git shortcuts in my ~/.bash_aliases to save typing:

gs() {
    git status
}

gc() {
    git commit -am"$*"
}

gd() {
    git diff "$@"
}

gl() {
    git log "$@"
}

gp() {
    git push || { echo "push failed" >&2; return 1;}
}

# run this after updating changelog with dch, to get a clean commit
gch() {
    version=$(dpkg-parsechangelog --show-field Version)
    case "$(git status --porcelain)" in
    ' M debian/changelog')
        echo "Changelog has been modified.";;
    '')
        echo "No changes to commit."
        return 1;;
    *)
        echo "Some other files have been changed. Commit them first."
        return 1;;
    esac
    git commit debian/changelog -m"Update changelog: ver. $version" || { echo "commit failed" >&2; return 1;}
}

And it might not hurt to add your name and email to ~/.gitconfig to save time with commits:

[user]
	email = *****@bunsenlabs.org
	name = John Crawley

Last edited by johnraff (2024-06-14 06:03:00)

johnraff · 2024-06-14 04:55:56

ssh

You can access GitHub via https if you want, but ssh is much easier once you've set up an ssh keypair. Instead of password every time you upload, just once per session enter the ssh passphrase and ssh-daemon is ready to log you in, no questions asked.

1) Make an ssh key pair. This guide is as good as any: https://www.digitalocean.com/community/ … nux-server

2) Edit your ~/.ssh/config, adding this line:

AddKeysToAgent yes

So the first time that session you use your key to log in somewhere, ssh-agent will remember it for you so you only need to type the passphrase once.

3) Add your ssh key for authentication to your GitHub account: https://docs.github.com/en/authenticati … ub-account

4) Any existing local GitHub repos can be switched to use ssh for login by editing .git/config in the repo directory. Change any url entries that start with https like 'url= https://github.com/BunsenLabs/bunsen-docs.git' to 'url = git@github.com:BunsenLabs/bunsen-docs.git'. When cloning a repo from scratch GitHub offers the SSH option "Use a password-protected SSH key" from the little green '<> Code' box.

johnraff · 2024-06-14 05:06:23

workflow 2 - local git repo

I'm leaving out step 1) which is what everyone's already doing anyway...

If you haven't already got a local copy of the package's git repo, then go to where you keep your BL packages (I use ~/git/bunsen/) and

git clone git@github.com:BunsenLabs/<package>.git

Now you can edit the code inside as you like. If you're going to make a lot of changes, or are afraid of breaking something, make a temporary branch and do it there. The usual advice is to "commit often", and keep each commit on a single topic, even if it isn't a single file.

If you do a 'git push' then your changes will go on the git server where other devs can access them. Don't wait too long, especially if other people might be working on the same package. Of course if your changes are still on a temporary branch they won't affect anybody else. If you want feedback you might even consider making a Pull Request from your forked branch instead of just merging it straight in to the main branch.

#1 2024-06-08 04:00:18

Re: Getting into Debian packages

#2 2024-06-09 19:39:18

Re: Getting into Debian packages

#3 2024-06-10 01:01:55

Re: Getting into Debian packages

#4 2024-06-10 03:56:10

Re: Getting into Debian packages

#5 2024-06-10 05:10:49

Re: Getting into Debian packages

#6 2024-06-10 05:53:15

Re: Getting into Debian packages

#7 2024-06-10 06:07:52

Re: Getting into Debian packages

#8 2024-06-10 15:43:47

Re: Getting into Debian packages

#9 2024-06-11 02:31:56