You are not logged in.

#1 2016-06-13 07:58:26

earlybird
ほやほや
Registered: 2015-12-16
Posts: 738
Website

[Helium] Adopt saltstack (or alternative) for config management tasks?

Yes; reworking the welcome script for the next release would be a nice thing.

Actually, personally, I'd suggest to replace the BL system management with Salt or similar. It's a config management system where you have so-called state definitions. For example, a package that'd install google chrome might have the following state file:

google-chrome-stable:
  pkgrepo.managed:
    - dist: stable
    - file: /etc/apt/sources.list.d/google-chrome.list
    - humanname: Official Google Chrome Stable repository managed by Google
    - key_url: https://dl.google.com/linux/linux_signing_key.pub
    - name: deb http://dl.google.com/linux/chrome/deb/ stable main
  pkg.latest: []

then we could apply this configuration, for example when a user selects the action from a pipemenu, just by executing the command:

salt-call state.apply bunsen.pkg.google-chrome-stable # sets up the repository, adds the signing key, and installs the package in one go

That'd allow us to delete lots of custom log in scripts and concentrate on enhancing user-facing things like bl-welcome, like refactoring it into a extremely neat dialog-based GUI that just applies salt-states to the system instead of packing all the logic itself. I think that channelling future development in that way would do away with about 90% of the criticism raised in the DW review!

Last edited by earlybird (2016-06-13 07:59:54)

Offline

#2 2016-06-13 10:17:43

iMBeCil
WAAAT?
From: Edrychwch o'ch cwmpas
Registered: 2015-09-29
Posts: 759

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

madoromi wrote:

<cut>
Actually, personally, I'd suggest to replace the BL system management with Salt or similar.
<cut>

God forbid! yikes yikes yikes

As much as bl-welcome might annoy some people, it is very nice way of giving options to install additional stuff for (semi)advanced people, and at the same time being polite and unobtrusive.

Not to mention that the time spent on understanding debian+friends is much more useful than spending time to understand new  software, with unknown (future) life span ...

As for the DW review: I would suggest that somewhere on www.bunsenlabs.org web page following would be visible:
"The BunsenLabs is meant to be for (semi)experienced users, but newbies shouldn't be afraid of using it with the help of extremely friendly and knowledgeable forum."

(I know it is awfully put, but you see what I mean ... sorry, Engrish is not my native language.)

The point being: BunsenLabs is for bloated-Linux-distro-refugees (like Ubuntu), and as such requires a little bit of knowledge/experience. But, newbies will be able to use it, too, with the help of the excellent community.

Last edited by iMBeCil (2016-06-13 10:21:46)


Postpone all your duties; if you die, you won't have to do them ..

Offline

#3 2016-06-13 10:57:04

xaos52
The Good Doctor
From: Planet of the @pes
Registered: 2015-09-30
Posts: 695

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

Maybe not replace the way it is done currently, but offer alternatives (to bunsen-welcome, the black and grey look, more...)?

Offline

#4 2016-06-13 12:38:23

hhh
Meep!
Registered: 2015-09-17
Posts: 9,280
Website

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

@twoion, we should open another thread to discuss Salt and alternatives.

I left a thank you comment and counterpoint at DW (#16 and #17). I could have razzed him for opening autostart to disable conky, but meh. smile

Offline

#5 2016-06-13 12:50:09

twoion
ほやほや
Registered: 2015-08-10
Posts: 2,675

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

iMBeCil wrote:
madoromi wrote:

<cut>
Actually, personally, I'd suggest to replace the BL system management with Salt or similar.
<cut>

God forbid! yikes yikes yikes

As much as bl-welcome might annoy some people, it is very nice way of giving options to install additional stuff for (semi)advanced people, and at the same time being polite and unobtrusive.

You don't know Salt? It would make BL much more KISS and allow for the deletion of lots of code.

For example, I'm porting some stuff right now as a PoC, and this wall of text for just adding and pinning the deb-multimedia repository to the system becomes just:

repo-deb-multimedia:
  pkgrepo.managed:
    - name: deb http://www.deb-multimedia.org jessie main non-free
    - dist: jessie
    - file: /etc/apt/sources.list.d/deb-multimedia.list
    - humanname: deb-multimedia repository

deb-multimedia-keyring:
  pkg.latest:
    - refresh: true

/etc/apt/preferences.d/deb-multimedia.conf:
  file.managed:
    - contents: |
        Package: *
        Pin: origin *.deb-multimedia.org
        Pin-Priority: $dmo_priority
    - owner: root
    - group: root
    - mode: 644

and actually adding the repository is this single command – which can be executed by bl-welcome or anything else:

salt-call state.apply repo.deb-multimedia # Add repo, install keyring, add pinning, and apt-get update!

Or to add Bunsen backports:

salt-call state.apply repo.bunsen.jessie-backports # Add repo, install key (if needed) and apt-get update!

or as previously shown:

salt-call state.apply pkg.google-chrome.stable # install Chrome stale
salt-call state.apply pkg.google-chrome.unstable # install Chrome unstable

While .deb packages have access to sophisticated frameworks to move or override configuration files, actually changing or even templating configuration files sucks big time. Saltstack is a tool that tries to fill the void left between the limitations of system package managers and totally manually writing every config file while minimizing boilerplate volume and errorprone code duplication (see bl-welcome). Note how different other Bunsen stuff is from bunsen-welcome, i.e. bunsen-pipemenus contains real programs, while bunsen-welcome messes around with calling sed on system configuration files and adding logic and conditionals – IMO such a change would not really be user-visible, not intransparent (because the state definitions will still be installed and available on the system!), and allow us to concentrate on making the real programs and user interfaces better instead of trying to get boilerplate 'to work'.


Wahllos schlägt das Schicksal zu / heute ich und morgen du.

Online

#6 2016-06-13 13:09:05

xaos52
The Good Doctor
From: Planet of the @pes
Registered: 2015-09-30
Posts: 695

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

@twoion,

You have my attention allright.

Installed packages from saltstack repo.
I have master and minion running.
Reading docs now.

Looks promising indeed.
Makes configuring and maintaining a system == managing some simple text files.

I did not know about salt before you pointed it out. Thanks.

Offline

#7 2016-06-13 13:10:17

iMBeCil
WAAAT?
From: Edrychwch o'ch cwmpas
Registered: 2015-09-29
Posts: 759

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

twoion wrote:
iMBeCil wrote:
madoromi wrote:

<cut>
Actually, personally, I'd suggest to replace the BL system management with Salt or similar.
<cut>

God forbid! yikes yikes yikes

As much as bl-welcome might annoy some people, it is very nice way of giving options to install additional stuff for (semi)advanced people, and at the same time being polite and unobtrusive.

You don't know Salt? It would make BL much more KISS and allow for the deletion of lots of code.

No, I don't. (Apparently, I might be ignorant because of this.) From what I found from their web site, it seems to be sort of  abstractions of the packages management. KISS? Perhaps for the devs, but not for the users ... I don't know what would I - an ordinary user - gain with it. What's wrong with apt-get (and metapackages, for that matter)?

I don't want to force anything on BunsenLabs' devs - feel free to include it ... but IMHO, this goes against 'BL = only debian stable with OB and tint'. Personally, I have seen a few distros with 'custom packaging systems', and that stopped me from even trying them. "Where is 'apt-get/aptitude' in my debian?"  big_smile

Or did I get it completely wrong?


Postpone all your duties; if you die, you won't have to do them ..

Offline

#8 2016-06-13 13:13:52

twoion
ほやほや
Registered: 2015-08-10
Posts: 2,675

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

xaos52 wrote:

@twoion,

You have my attention allright.

Installed packages from saltstack repo.
I have master and minion running.
Reading docs now.

Looks promising indeed.
Makes configuring and maintaining a system == managing some simple text files.

I did not know about salt before you pointed it out. Thanks.

Note that what I am working with here and which would probably fit our purpose better is running as a masterless minion. There is no need to even have a daemon running: one can just run salt-call to invoke the engine and (fire and) forget.

Salt is actually conceptually more fit to manage large deployments of dozens, hundreds, thousands of systems in an enterprise environment – that's when you would have one or multiple masters and have the managed systems running as minions.

One interesting thing is that it would be easy to revert the system to a vanilla state. With a proper top.sls,

sudo salt-call state.highstate

could reset the system to a vanilla Debian + BL install.


Wahllos schlägt das Schicksal zu / heute ich und morgen du.

Online

#9 2016-06-13 13:40:07

twoion
ほやほや
Registered: 2015-08-10
Posts: 2,675

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

iMBeCil wrote:

Or did I get it completely wrong?

Hm, as far as I see it saltstack != package management. Packages and metapackages will be around! Salt provides a way of declaring the state a system should be in and automating the steps to get there as much as possible. It can do this by using things like dpkg and apt in the first place.

For example, the tasks of adding a new repository to apt consists of a) setting up the .list file and b) making sure that the key is added and c) refresh the package database so that the user can install from the repo. You could to that in a maintainer package script by whipping up some lines of code that put the lines there – but what about e.g. the choice between enabling non-free packages or not? Right, we have to parametrize the installation process and template the apt .list file:

deb http://somerepo somedistro ${user_choice}

Debian has something for that: debconf. It's just that it's a nightmare to use and one still needs custom package maintaner scripts to call out to it (and thereby custom logic).

Conversely, Salt is an abstraction layer that allows me to say

This is how I want things to be: "Package A is installed from repository B".

So, say I want to go get the system from state A to state Z, but in order to get to Z there are distinct actions that need to be taken inbeetween – possibly even in an exact, specific order because e.g. you can't install a package when the repository that provides it is not  yet known to the system. Salt provides a way for me to say A->Z but the abstraction layer performs the steps B..Y in their correct order, and I only need to let the system know A, Z, and variables the system cannot know on its own  like the repository URL.

That's why I think using it can help us get rid of lots of boilerplate code, most notably bl-welcome.


Wahllos schlägt das Schicksal zu / heute ich und morgen du.

Online

#10 2016-06-13 14:17:54

iMBeCil
WAAAT?
From: Edrychwch o'ch cwmpas
Registered: 2015-09-29
Posts: 759

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

^Hm, when you put it that way, it makes more sense. So, I declare myself possibly wrong on the 'salt' topic  big_smile

However, would the end user need to have 'salt' package(s) installed in BunsenLabs? This is what bothered me. This, and the question whether apt-get/aptitude will be still usable. That's what I meant by 'package management abstraction'.

As I said, it might work good, but I think yet another package management 'layer' above apt-get might be turn-off-er for user(s).

Nevertheless, it would be interesting to see attempts with 'salt' ...

Anyway, thanks for very nice explanation, and - basically - RTFM-ing instead of me  ops

Last edited by iMBeCil (2016-06-13 19:54:14)


Postpone all your duties; if you die, you won't have to do them ..

Offline

#11 2016-06-13 17:21:37

Head_on_a_Stick
Member
From: London
Registered: 2015-09-29
Posts: 8,759
Website

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

madoromi wrote:

I think that channelling future development in that way would do away with about 90% of the criticism raised in the DW review!

I don't think that such a radical change should be predicated on the opinion of a single person.

In your opinion, what is the specific problem(s) for which salt offers a solution?

Don't get me wrong, it looks like a lovely tool but surely a static set of scripts is both more reliable and easier to debug?


“Et ignotas animum dimittit in artes.” — Ovid, Metamorphoses, VIII., 18.

Forum Rules   •   How to report a problem   •   Software that rocks

Offline

#12 2016-06-14 17:35:34

Horizon_Brave
Operating System: Linux-Nettrix
Registered: 2015-10-18
Posts: 1,473

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

I like the idea. As much as I love the welcome script it's a fantastic idea, I think (and I've expressed this concern here: https://forums.bunsenlabs.org/viewtopic.php?id=1762  )
That the script often seems to be a go to 'dumping ground' for problems that we encounter.  "Hmm how to add support for feature X,Y,Z? ..... Let's put it in the Bunsen-Welcome script!"   It just seems like the slim and sleekness of it would be lost and it'd turn into a huge phone book of scripts, and paths that we (JohnRaff) would have to maintain.


"I have not failed, I have found 10,000 ways that will not work" -Edison

Offline

#13 2016-06-15 10:37:11

xaos52
The Good Doctor
From: Planet of the @pes
Registered: 2015-09-30
Posts: 695

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

HoaS wrote:

Don't get me wrong, it looks like a lovely tool but surely a static set of scripts is both more reliable and easier to debug?

May I conclude from this that you prefer sysv-init over systemd?  tongue

@twoion,
Now I know where you got the Bunsenlabs release name schema from smile

Offline

#14 2016-06-15 17:55:50

Head_on_a_Stick
Member
From: London
Registered: 2015-09-29
Posts: 8,759
Website

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

xaos52 wrote:

May I conclude from this that you prefer sysv-init over systemd?

Actually, my favourite init system is rc(8) smile


“Et ignotas animum dimittit in artes.” — Ovid, Metamorphoses, VIII., 18.

Forum Rules   •   How to report a problem   •   Software that rocks

Offline

#15 2016-06-19 11:23:05

twoion
ほやほや
Registered: 2015-08-10
Posts: 2,675

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

We could also start small and provide dedicated repository-setup packages, that is packages which own /etc/apt/sources.list.d/bunsen.list and configure the set up enabled repositories accordingly.

Or, provide one repository management package and read a configuration from /etc/default/bunsen-repository, which can be a shell script fragment:

BL_MIRROR="pkg.bunsenlabs.org"
BL_BACKPORTS="yes" # or no
DEBIAN_MIRROR="http://ftp.de.bunsenlabs.org/debian/"
DEBIAN_RELEASE="jessie" # change to testing for upgrading to testing, etc
DEBIAN_SUITES="main contrib non-free"
DEBIAN_BACKPORTS="no"
DEBIAN_MOZILLA="no"

unlike the way it is done currently, this file can be read fully by a shell script just by source'ing it. After changing that file, the user then would run

dpkg-reconfigure bunsen-repositories

The welcome script then could just show a dialog using debconf when installing the repository package for the first time the script is run.


Wahllos schlägt das Schicksal zu / heute ich und morgen du.

Online

#16 2016-06-27 08:03:55

johnraff
nullglob
From: Nagoya, Japan
Registered: 2015-09-09
Posts: 6,538
Website

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

Hi twoion, I've been away for this discussion, and just returned from Cambodia.

To throw in a few first impressions, in random order (I hope to have something more coherent after some more thought):

  • I agree that a major rewrite of bunsen-welcome will be needed sooner or later. If nothing else, the repetitions of 'apt-get update' after every repo modification could be avoided if all user choices were collected first, and executed en-masse afterwards.

  • There has already been some attempt at modularity, in the bl-welcome "pages" and in the functions in bunsen-common, which can be used by other scripts like the pipemenus. A more complete abstraction (like Salt?) sounds interesting, at least.

  • I don't think bl-welcome is as intrinsically different from the pipemenus as you suggest: "...bunsen-pipemenus contains real programs, while bunsen-welcome messes around with calling sed..." A lot of the code in both those zones was written by the same person (me). smile In fact, there are three places where bl-welcome uses sed to edit a file: 1) dev-install-lamp-stack, to allow the default apache to read .htaccess files, 2) system-tweaks, to comment out fdpowermon from autostart and 3) check-repos, to workaround the installer bug where contrib and non-free sometimes needed to be added to sources.list. 1) and 3) could be done by something like Salt, but it hardly corresponds to the kind of mess that you invoke IMO.

  • "Logic and conditions" have been added to try and avoid damaging settings that a user might have deliberately put in place. For that reason, regexes have been made wider than would have been needed if we could guarantee everything had been done by us. I have tried to make all bunsen packages that I've worked on as modular as possible, so that users can cherrypick, while minimizing breakage. I'm sure there is more to be done in that department.

  • I remain to be convinced that your example with deb-multimedia (now soon to be dropped from bl-welcome btw) would cover all the corner cases that the script does.

  • I would like to repeat Head_on_a_Stick's request for a specific example of a problem which salt would make easier to handle - in particular some of the "90% of the issues raised in the DW review". EDIT: In fact, let's list up all the problems that were reported there, and see what the issues actually were.

I hope this doesn't sound too defensive. I definitely agree that bl-welcome needs cleaning up, and I'm all in favour of more abstraction in principle, as long as we don't end up replicating Gnome or something...

I'll no-doubt have some more questions about Salt when I've actually looked at it. What packages should I install to do the kind of things you are suggesting here? (We would be able to use standard Debian packages, right?)

Last edited by johnraff (2016-06-27 09:19:38)


...elevator in the Brain Hotel, broken down but just as well...
( a boring Japan blog (currently paused), idle Twitterings and GitStuff )

Introduction to the Bunsenlabs Lithium Desktop

Offline

#17 2016-07-03 04:52:29

johnraff
nullglob
From: Nagoya, Japan
Registered: 2015-09-09
Posts: 6,538
Website

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

Raphael Herzog has been using Salt too (bottom of page): https://raphaelhertzog.com/2016/07/01/m … june-2016/


...elevator in the Brain Hotel, broken down but just as well...
( a boring Japan blog (currently paused), idle Twitterings and GitStuff )

Introduction to the Bunsenlabs Lithium Desktop

Offline

#18 2016-07-03 13:25:26

hhh
Meep!
Registered: 2015-09-17
Posts: 9,280
Website

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

I'm trying to get up to speed here, it's over my head.

iMBeCil wrote:

However, would the end user need to have 'salt' package(s) installed in BunsenLabs?

Yes (right guys?), but it won't affect dpkg/apt. The jessie packages are way outdated, but backports has them.
https://packages.debian.org/search?sear … alt-master

@twoion, so more config options with less work? Or just more logically laid out? You see the worry... "I'm an end user, will this help me or confuse me?" But your saying choices like not adding contrib/nonfree, enabling backports, installing packages from the right-click menu "Install" entries with or without recommends, installing GIMP with or without the plugin registry, these could all be handled by salt with similar syntax (pardon my layman's summary)?

Offline

#19 2016-07-03 13:29:57

hhh
Meep!
Registered: 2015-09-17
Posts: 9,280
Website

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

BTW, the saltstack beginner guide...
https://docs.saltstack.com/en/getstarted/

johnraff wrote:

In fact, let's list up all the problems that were reported there, and see what the issues actually were.

I'll create a new topic for this or it will derail this thread.

Offline

#20 2016-07-04 03:05:06

johnraff
nullglob
From: Nagoya, Japan
Registered: 2015-09-09
Posts: 6,538
Website

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

The Jessie packages may be old (I think salt-minion is the one we'd need to install), but if they do the job we need done, then OK, right?

To be honest, the only topic I see in the DW complaints list that might be relevant is #3, that bl-welcome goes step-by-step, not as a batch. I guess @twoion is suggesting that using a system like Salt would make a bl-welcome rewrite easier?
I do think this:

twoion wrote:

channelling future development in that way would do away with about 90% of the criticism raised in the DW review

might be a tad over-optimistic though...


...elevator in the Brain Hotel, broken down but just as well...
( a boring Japan blog (currently paused), idle Twitterings and GitStuff )

Introduction to the Bunsenlabs Lithium Desktop

Offline

#21 2016-07-04 20:27:41

pvsage
Internal Affairs
Registered: 2015-09-29
Posts: 1,433

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

Why are we talking about the Jessie packages for saltstack if we're looking to use it in Helium?  Shouldn't we be talking about Stretch instead?

twoion wrote:

channelling future development in that way would do away with about 90% of the criticism raised in the DW review

Not true; over half the criticism was "Oh God, it's so gray.  Look at how dray and depressing it is!  It's just so gray.  Also, I hate Conky, and I can't keep my mouse in a window so scrollwheel desktop switching should be disabled."  I agree that it would address a majority of the legitimate criticism though.


Be excellent to each other, and...party on, dudes!
BunsenLabs Forum Rules
Tending and defending the Flame since 2009

Offline

#22 2016-07-05 06:41:31

johnraff
nullglob
From: Nagoya, Japan
Registered: 2015-09-09
Posts: 6,538
Website

Re: [Helium] Adopt saltstack (or alternative) for config management tasks?

pvsage wrote:

Why are we talking about the Jessie packages for saltstack if we're looking to use it in Helium?  Shouldn't we be talking about Stretch instead?

Good point!


...elevator in the Brain Hotel, broken down but just as well...
( a boring Japan blog (currently paused), idle Twitterings and GitStuff )

Introduction to the Bunsenlabs Lithium Desktop

Offline

Board footer

Powered by FluxBB