Tutorial image archives - best filename and directory structure?

johnraff · 2024-10-09 07:30:07

It's time to be serious about making an archive of the images used in our tutorials and documentation. Imgur has started to be a bit unreliable, especially with anonymous uploads that were made a long time ago.

https://forums.bunsenlabs.org/viewtopic.php?id=9146
https://forums.bunsenlabs.org/viewtopic.php?id=9184
https://forums.bunsenlabs.org/viewtopic … 07#p138807

Really, GitHub isn't the ideal place for an archive of images, but for us there would be the advantage of keeping all our assets in the same place. I think the repo size would be unlikely to hit even 1GB, or file sizes to be more that 1MB at most.

But first we need to decide exactly how the archive is to be laid out, so as to make it as easy as possible to retrieve the images we need if Imgur becomes unusable.

r.chaffee53 wrote:

Suggest to keep the same image name as imgur uses.

This might work if we used the same filename referenced in the original post.
The full-sized image only is enough so we can ignore any final t m or l in the filename.

Questions:

*) Whether it would be better to store images individually in directories related to the topic, or in tarballs holding all the images for a specific post.

*) Directories - one per post with individual images, or one per topic with tarballs for each post?

*) Post or topic references - by name, like Index » Tutorials » jgmenu - Creating an Applications Menu or by url address, like https://forums.bunsenlabs.org/viewtopic.php?id=6540 In fact that could be simplified down to the topic number 6540 and it would still be easy to find. Individual posts - like the first one in that topic - have urls like https://forums.bunsenlabs.org/viewtopic … 299#p98299 so the post number is 98299.

Those numbers could be used as directory names or tarball names...

But let's get this right! Changing stuff like this afterwards can be a real PITA.

Any other ideas/comments/suggestions?

r.chaffee53 · 2024-10-09 14:11:43

Archive names I've used so far:

The first one, like the rest, follow forum naming; adding the author's name in case of similar title names.

Index » Getting Started » Introduction to the Bunsenlabs Boron Desktop

get_strt-intro_boron_desktop-bob

Hyphen (-) between sections and underline (_) for multi word names.

Submitted for discussion.

Bob

Last edited by r.chaffee53 (2024-10-09 14:13:15)

brontosaurusrex · 2024-10-09 15:09:18

Just a note, size of a lot of this images can be halved by reducing colors (https://i.imgur.com/oz1VvFJ.png):

pngquant --nofs oz1VvFJ.png

goes from ~10k to ~4k.

This will not look perfect for images with a lot of gradients (removing --nofs might help with some), so perhaps some can be cropped..., and the future posters of this tutorials can be encouraged to post images with singular bg color. p.s. Pngquant is in debian repos.

edit2: removed avif talk, it doesn't actually halve the images.

r.chaffee53 · 2024-10-09 15:14:03

^ Thank you. Just what I needed. Keirei.

Bob .

Last edited by r.chaffee53 (2024-10-09 15:27:55)

johnraff · 2024-10-10 02:43:19

johnraff wrote:

Whether it would be better to store images individually in directories related to the topic, or in tarballs holding all the images for a specific post.

One consideration is that if the images are stored as-is, uncompresed, then they can be viewed online. eg here:
https://github.com/BunsenLabs/bunsen-im … rs/default
That might be useful if images in a tutorial become completely unavailable, so the dev replacing them can see what they're supposed to look like without having to download files first.

johnraff · 2024-10-10 02:53:16

Directories - two options (without a preference at this point) would be a tree, like:

Tutorials
jgmenu - Creating an Applications Menu
file1 file2...

or a "flat" layout with all the directories at the top level, identified by their topic ID or post ID (possibly with a human-readable name too):

6540-jgmenu-Creating an Applications Menu
file1 file2...

We should consider the most likely workflow of a dev who needs to restore a bunch of missing images...

johnraff · 2024-10-10 03:10:17

brontosaurusrex wrote:

pngquant --nofs oz1VvFJ.png
goes from ~10k to ~4k.
Pngquant is in debian repos.

Thanks! From that, also discovered pngcrush and optipng.
Size of png files is a long-standing annoyance.

r.chaffee53 · 2024-10-10 20:01:21

johnraff wrote:

We should consider the most likely workflow of a dev who needs to restore a bunch of missing images...

This is a reprint from the image backup thread.

get_strt-intro_boron_desktop-bob has been updated.

Included in the archive is a saved copy of the web site. This is the total package for the Dev who might need to rebuild from scratch or just use as a reference. The saved web page only includes the thumbs of the images and links to imgur for the full image; which are also included with the archive. The new file name includes a 'w'.

get_strt-intro_boron_desktop-bob-w.tar.gz

I̶s̶ ̶t̶h̶e̶r̶e̶ ̶a̶ ̶n̶o̶t̶e̶/̶n̶o̶t̶i̶c̶e̶ ̶o̶f̶ ̶s̶o̶m̶e̶ ̶k̶i̶n̶d̶ ̶n̶o̶t̶i̶c̶e̶ ̶t̶h̶a̶t̶ ̶c̶o̶u̶l̶d̶ ̶b̶e̶ ̶i̶n̶c̶l̶u̶d̶e̶d̶ ̶w̶i̶t̶h̶ ̶t̶h̶e̶ ̶a̶r̶c̶h̶i̶v̶e̶?̶
I included the start of one, see if you like it.

Archive found here for the intrepid reader.

Submitted for your consideration.

Bob

Edit: I'm not married to the naming theme I've started, go along to get along. Hell, I'm okay with whatever; just pick the best and leave the rest. When the Elders have time of course, no rush.)

Last edited by r.chaffee53 (2024-10-11 01:51:38)

johnraff · 2024-10-11 06:14:31

johnraff wrote:

We should consider the most likely workflow of a dev who needs to restore a bunch of missing images...

I guess it's most likely that if the archive is on GitHub then a dev will have cloned it to their local machine.
Bunsen-images which has a lot of much bigger images is only ~300MB, so the tutorial images archive will likely be smaller than that, so no big deal.

Just replacing one or two images can easily be done by hand, but if the image host we were using went totally offline and we needed to do all of them, then a bit of automation would probably be called for.

Something to bear in mind...

johnraff · 2024-10-11 06:26:38

r.chaffee53 wrote:

get_strt-intro_boron_desktop-bob has been updated.
Included in the archive is a saved copy of the web site page. This is the total package for the Dev who might need to rebuild from scratch or just use as a reference. The saved web page only includes the thumbs of the images and links to imgur for the full image; which are also included with the archive. The new file name includes a 'w'.

Thanks for your work Bob.

I should mention here that repairing any image links cannot be done by editing an html page. Those are generated on the fly by the forum software. Editing would have to go via the forum "edit post" interface and be done with BB Code. An unwelcome complication, yes.

Complete automation of the restore process will likely be impossible, but maybe something that:
1) identifies a place in a post than needs fixing
2) identifies the needed image
3) uploads the replacement image from our archive (probably on the dev's computer) to wherever we have chosen as our image host at that time
4) puts the correct BBCode on the clipboard for pasting in over the defunct image link (whether the t m l filename endings will still work or not depends on whether that image host is still imgur)

...repeat...

r.chaffee53 · 2024-10-11 10:26:36

@johnraff please don't edit my quotes. I did not say what is claimed I said. Thanks for your understanding in this matter.

Bob

johnraff · 2024-10-15 01:11:45

^Sorry about that. I maybe should have included something like "FIFY Johnraff".

The tarball archives no longer seem to be available, but when I downloaded one last week I found it contained a single html file. That corresponds to a "web page" not a "web site". In fact to provide the whole web site in a tarball archive would be quite difficult. I once downloaded the whole CrunchBang forum site with wget and it took 4~5 days and occupied a (very compressed) 350MB squashfs file.

So I just assumed "web site" was a simple slip of the kind we all make now and then. Editing the quote directly seemed at the time like the easiest way to keep the discussion moving, but if I find such things in future I will add a note instead of editing the quote.

#1 2024-10-09 07:30:07

Tutorial image archives - best filename and directory structure?

#2 2024-10-09 14:11:43

Re: Tutorial image archives - best filename and directory structure?

#3 2024-10-09 15:09:18

Re: Tutorial image archives - best filename and directory structure?

#4 2024-10-09 15:14:03

Re: Tutorial image archives - best filename and directory structure?

#5 2024-10-10 02:43:19

Re: Tutorial image archives - best filename and directory structure?

#6 2024-10-10 02:53:16

Re: Tutorial image archives - best filename and directory structure?

#7 2024-10-10 03:10:17

Re: Tutorial image archives - best filename and directory structure?

#8 2024-10-10 20:01:21

Re: Tutorial image archives - best filename and directory structure?

#9 2024-10-11 06:14:31

Re: Tutorial image archives - best filename and directory structure?

#10 2024-10-11 06:26:38

Re: Tutorial image archives - best filename and directory structure?

#11 2024-10-11 10:26:36

Re: Tutorial image archives - best filename and directory structure?

#12 2024-10-15 01:11:45

Re: Tutorial image archives - best filename and directory structure?

Board footer