You are not logged in.

#1 2017-11-04 14:37:13

Head_on_a_Stick
Member
From: London
Registered: 2015-09-29
Posts: 8,759
Website

What the shell? Bash, KornShell & zsh

Quick demonstration of the inherent speed advantage that the original KornShell (ksh93) has over the latest version of bash:

$ time assoc-arr-test-bash
[...]
    0m19.01s real     0m18.99s user     0m00.01s system
$ time assoc-arr-test-ksh93
[...]
    0m02.07s real     0m02.06s user     0m00.00s system

That's quite a boost glasses

The assoc-arr-test-$shell script is from here (the only difference between the two versions is the shebang):

http://www.tldp.org/LDP/abs/html/abs-gu … SOCARRTEST

I will have to see if I can convert the BunsenLabs bash scripts to ksh93 and test for any improvements.


“Et ignotas animum dimittit in artes.” — Ovid, Metamorphoses, VIII., 18.

Forum Rules   •   How to report a problem   •   Software that rocks

Offline

#2 2017-11-04 17:31:30

malm
jgmenu developer
Registered: 2016-10-13
Posts: 331
Website

Re: What the shell? Bash, KornShell & zsh

^ How does it compare with dash?

Offline

#3 2017-11-04 18:05:09

Head_on_a_Stick
Member
From: London
Registered: 2015-09-29
Posts: 8,759
Website

Re: What the shell? Bash, KornShell & zsh

malm wrote:

How does it compare with dash?

I did a (very limited) test in my mksh thread and it is even quicker than dash:

https://forums.bunsenlabs.org/viewtopic.php?id=3794

Quite astonishing for a shell that stopped being developed in 2012 cool

EDIT: having said that, it was proprietary (and rather expensive) before it was open-sourced so the code must be of very high quality.

Last edited by Head_on_a_Stick (2017-11-04 18:09:33)


“Et ignotas animum dimittit in artes.” — Ovid, Metamorphoses, VIII., 18.

Forum Rules   •   How to report a problem   •   Software that rocks

Offline

#4 2017-11-04 19:07:12

twoion
ほやほや
Registered: 2015-08-10
Posts: 2,393

Re: What the shell? Bash, KornShell & zsh

ksh93 has a number of unholy features begging the question why they aren't in Bash. It is also the better scripting language, but hasn't taken off on Linuy, sadly.


A silent kite against the blue, blue sky

Offline

#5 2017-11-04 21:06:32

Head_on_a_Stick
Member
From: London
Registered: 2015-09-29
Posts: 8,759
Website

Re: What the shell? Bash, KornShell & zsh

twoion wrote:

ksh93 has a number of unholy features

lol.

I particularly like the shcomp(1) utility that compiles ksh93 scripts into binary executables.

Last edited by Head_on_a_Stick (2017-11-04 21:07:42)


“Et ignotas animum dimittit in artes.” — Ovid, Metamorphoses, VIII., 18.

Forum Rules   •   How to report a problem   •   Software that rocks

Offline

#6 2017-11-04 21:43:07

malm
jgmenu developer
Registered: 2016-10-13
Posts: 331
Website

Re: What the shell? Bash, KornShell & zsh

@Head_on_a_Stick, @twoion: Thanks for the education. I had a quick browse of the code at https://github.com/att/ast/blob/master/ … /sh/main.c. Absolutely poetic. And I love the copyright - 1982-2012

Offline

#7 2017-11-06 14:31:43

ratcheer
Member
From: USA
Registered: 2015-10-05
Posts: 209

Re: What the shell? Bash, KornShell & zsh

I used ksh for a long time on Solaris and other enterprise UNIXs, but I've never really dug into it on Linux. I had no idea it might perform so much better.

Tim

Offline

#8 2017-11-06 21:36:55

Head_on_a_Stick
Member
From: London
Registered: 2015-09-29
Posts: 8,759
Website

Re: What the shell? Bash, KornShell & zsh

ratcheer wrote:

I had no idea it might perform so much better

It's not just performance: bash has a parlous history of security vulnerabilities, so much so that some in the BSD community refer to it as the broken again shell.

ksh93 has had 30+ years of development in professional UNIX environments and mksh is used as the login shell in Android, thus giving it many more "eyes on the code" than the alternatives, not to mention the much smaller codebase reducing the attack surface significantly.

Sadly, ksh93 is a tricky blighter to package because it needs AT&T's Software Technology (AST) toolkit and the associated (and very idiosyncratic) build system and that doesn't work with all operating systems.

Arch only has it in the AUR and the PKGBUILD is insanely complicated but it is a (slightly) newer version than the one available from Debian in their official repositories.

On the other hand, mksh has been ported to just about everything and so allows a uniform shell environment across all platforms, which is nice.


“Et ignotas animum dimittit in artes.” — Ovid, Metamorphoses, VIII., 18.

Forum Rules   •   How to report a problem   •   Software that rocks

Offline

#9 2017-11-06 21:57:10

Sector11
Tpyo Knig
From: 77345 ¡#
Registered: 2015-08-20
Posts: 5,274

Re: What the shell? Bash, KornShell & zsh

Can mksh replace "bash" easily in a distro?

Would existing bash scripts need modifying?


BunsenLabs Forum Rules ---== I'm a Conky 1.9'er ==---
System:    Host: s12 Kernel: 3.16.0-4-amd64 x86_64 (64 bit gcc: 4.8.4)
Desktop: Openbox 3.5.2 dm: (startx) Distro: Debian GNU/Linux 8

Offline

#10 2017-11-06 22:47:00

Head_on_a_Stick
Member
From: London
Registered: 2015-09-29
Posts: 8,759
Website

Re: What the shell? Bash, KornShell & zsh

Sector11 wrote:

Can mksh replace "bash" easily in a distro?

Yes, I use it as the default login shell for both root and all normal users in all my Linux systems, it only takes a single command.

Would existing bash scripts need modifying?

No.

Changing the user's default login shell does not mean that any system scripts need to be changed because they should all have #!/bin/bash as the first line and so will still call bash regardless of which login shell is set.

There may be some features of bash that you would miss for interactive use, you would have to try it and find out for yourself.

ProTip: use the supplied example configuration files, they provide many useful functions and aliases.


“Et ignotas animum dimittit in artes.” — Ovid, Metamorphoses, VIII., 18.

Forum Rules   •   How to report a problem   •   Software that rocks

Offline

#11 2017-11-06 23:06:51

Sector11
Tpyo Knig
From: 77345 ¡#
Registered: 2015-08-20
Posts: 5,274

Re: What the shell? Bash, KornShell & zsh

@HoaS

Thank you  ... food for thought.

I was wondering if it would 'replace' bash.


BunsenLabs Forum Rules ---== I'm a Conky 1.9'er ==---
System:    Host: s12 Kernel: 3.16.0-4-amd64 x86_64 (64 bit gcc: 4.8.4)
Desktop: Openbox 3.5.2 dm: (startx) Distro: Debian GNU/Linux 8

Offline

#12 2017-11-07 02:10:32

johnraff
nullglob
From: Nagoya, Japan
Registered: 2015-09-09
Posts: 5,229
Website

Re: What the shell? Bash, KornShell & zsh

Head_on_a_Stick wrote:

bash has a parlous history of security vulnerabilities

Just out of interest, are there any known unpatched vulnerabilities in the Debian Jessie bash 4.3-11+deb8u1 or Stretch 4.4-5?


John
--------------------
( a boring Japan blog , Japan Links, idle twitterings  and GitStuff )
In case you forget, the rules.

Offline

#13 2017-11-07 06:06:53

ohnonot
...again
Registered: 2015-09-29
Posts: 3,673
Website

Re: What the shell? Bash, KornShell & zsh

Head_on_a_Stick wrote:

Would existing bash scripts need modifying?

No.

let me rephrase that:
Would existing shell scripts (i.e. #/bin/sh) need modifying?
also, can you expand on differences/similarities between mksh and ksh? would mksh work as a drop-in replacement for ksh? is it equally lightweight?

Offline

#14 2017-11-07 07:02:30

Head_on_a_Stick
Member
From: London
Registered: 2015-09-29
Posts: 8,759
Website

Re: What the shell? Bash, KornShell & zsh

johnraff wrote:

Just out of interest, are there any known unpatched vulnerabilities in the Debian Jessie bash 4.3-11+deb8u1 or Stretch 4.4-5?

I wouldn't think so, no, Debian are exceptional at pushing fixes.

However, Shellshock was present and undetected for many years so it is the unknown and potential vulnerabilities that worry me — I think a proactive attitude is vital if security is a concern.

ohnonot wrote:

Would existing shell scripts (i.e. #/bin/sh) need modifying?

No because !#/bin/sh scripts should not be using bashisms anyway wink

Also, /bin/sh will remain symlinked to dash so that will be called irrespective of the user's login shell.

At any rate, the POSIX sh standard (IEE 1003.1-2008) is actually based on ksh88 so compatibility is pretty much guaranteed.

ohnonot wrote:

can you expand on differences/similarities between mksh and ksh? would mksh work as a drop-in replacement for ksh? is it equally lightweight?

ksh93 is a full-featured shell that aims to do everything, just like bash and zsh, whereas mksh is based on OpenBSD's public domain implementation of the KornShell programming language (ie, it shares no code with ksh93) and it inherits that operating system's tendency to strip out and remove any insecure or extraneous features resulting in a more "bare bones" shell but also one that is much lighter in respect of resource usage, see my mksh thread for some crude tests of memory usage:

https://forums.bunsenlabs.org/viewtopic.php?id=3794

So mksh can't be dropped in as a replacement for bash or ksh93 in scripts because it lacks some features (most notably associative arrays) but the codebase is smaller than that of the more able competition and it is faster than all but ksh93 (and dash).

Last edited by Head_on_a_Stick (2017-11-07 07:06:47)


“Et ignotas animum dimittit in artes.” — Ovid, Metamorphoses, VIII., 18.

Forum Rules   •   How to report a problem   •   Software that rocks

Offline

#15 2017-11-07 08:05:06

Steve
Member
Registered: 2017-01-03
Posts: 642

Re: What the shell? Bash, KornShell & zsh

@ Hoas, im on arch linux, Ive just chsh to mksh and made a few changes, added .mkshrc from /etc/skel/.mkshrc to ~/.mkshrc

Have a question on .xinitrc script im using in archbang, would i change it to look like below as far as calling the script?

#!/usr/bin/mksh

export DE=openbox

# Dbus fix
if [ -z "$DBUS_SESSION_BUS_ADDRESS" ]; then
    eval "$(dbus-launch --sh-syntax --exit-with-session)"
fi

/usr/bin/xdg-user-dirs-update

exec dbus-launch --sh-syntax --exit-with-session openbox-session

ive also changed bash_profile to .profile and it looks like this and working ok.

# System settings before starting X
. $HOME/.mkshrc

PATH=$PATH:$HOME/bin

# set up alsa
/usr/bin/amixer sset Master Mono 90% unmute  &> /dev/null
/usr/bin/amixer sset Master 90% unmute  &> /dev/null
/usr/bin/amixer sset PCM 90% unmute &> /dev/null

# Start x on login
if [[ -z $DISPLAY && ! -e /tmp/.X11-unix/X0 ]]; then
   exec xinit -- /usr/bin/X -nolisten tcp vt7
fi

Last edited by Steve (2017-11-07 08:16:10)

Offline

#16 2017-11-07 20:14:52

Head_on_a_Stick
Member
From: London
Registered: 2015-09-29
Posts: 8,759
Website

Re: What the shell? Bash, KornShell & zsh

Mod Note: I seemed to have developed a distinct topic in the completely off-topic thread and that was off-topic, even for Off Topic (or something). Anyway, I've split it out so people can either join the noise discussion or ignore the thread, I hope the participants don't mind.

Steve wrote:

on .xinitrc script im using in archbang, would i change it to look like below as far as calling the script?

Personally, I do not add a shebang to ~/.xinitrc because that ensures that the default login shell is used at all times but yes, adding #!/bin/mksh at the top will force the issue.

Side question: why are you so hung up on DBus?

I think you will find that systemd is almost obsessive about launching sessions for anything with a pulse so perhaps try removing all that stuff?

Your checks appear to use POSIX conditionals so you could in fact declare #!/bin/sh and gain a (tiny) speed advantage by re-linking /bin/sh from bash (yes, Arch does that as stock) to dash instead (as Debian does from stock):

https://wiki.archlinux.org/index.php/Da … 2Fbin.2Fsh

Steve wrote:

ive also changed bash_profile to .profile

In Arch, the bash package supplies ~/.bash_profile and if that file is present then bash will not read ~/.profile when it starts up.

However, in Debian bash has ~/.profile instead (which it will read automatically) thus unifying configuration across all shells so if you remove ~/.bash_profile in your Arch box then you can switch to ~/.profile for bash and mksh (and ksh93, zsh, ash, tcsh, rc, fish, etc) and simplify things a bit.

Other points:

  • quote your variables!

  • for PATH, place ~/bin before :$PATH to let custom programs be run in preference

  • no need to source ~/.mkshrc, it will do that automatically unless $ENV is set

  • the `xinit` command hasn't listened to port 80 for a while now

  • quote your variables! I know I've said this before but it's such an important point that it's worth mentioning twice

hth


“Et ignotas animum dimittit in artes.” — Ovid, Metamorphoses, VIII., 18.

Forum Rules   •   How to report a problem   •   Software that rocks

Offline

#17 2017-11-07 21:40:55

malm
jgmenu developer
Registered: 2016-10-13
Posts: 331
Website

Re: What the shell? Bash, KornShell & zsh

I like to write POSIX compliant shell.

Have found this useful https://manpages.debian.org/testing/dev … .1.en.html

Offline

#18 2017-11-07 21:45:30

Head_on_a_Stick
Member
From: London
Registered: 2015-09-29
Posts: 8,759
Website

Re: What the shell? Bash, KornShell & zsh

Actually, this reminds me...

Steve wrote:

im on arch linux, Ive just chsh to mksh

If you have the time, could you please check if this bug applies to your system:

https://bugs.archlinux.org/task/55053?p … tring=mksh

tl;dr: running `pkexec gparted` will throw an error unless a /usr/bin/mksh line is added to /etc/shells.

Also, my mksh AUR packages include a statically-linked version (using musl libc rather than the bloated GNU C libraries) which you may be interested in:

https://aur.archlinux.org/packages/?O=0&K=mksh


“Et ignotas animum dimittit in artes.” — Ovid, Metamorphoses, VIII., 18.

Forum Rules   •   How to report a problem   •   Software that rocks

Offline

#19 2017-11-07 22:19:55

malm
jgmenu developer
Registered: 2016-10-13
Posts: 331
Website

Re: What the shell? Bash, KornShell & zsh

Staying clear of GNU/bashisms can be tricky.

These are useful notes on writing 'portable' shell https://github.com/git/git/blob/master/ … elines#L41

Using dash helps.

A few that I've learnt the hard way:

  • Don't use arrays and [[ ]]

  • Avoid non-POSIX re-directions such as '&>', '>&-', '|&', etc
    For example, to re-direct both stdout and stderr to /dev/null, do:

    command >/dev/null 2>&1
  • Don't use echo -e "foo", instead use

    printf "%b\n" "foo"
  • Don't use sed -i, it is not POSIX

  • Don't use curly brace globbing such as "ls *.{c,h}"
    [*/]

Last edited by malm (2017-11-07 22:22:04)

Offline

#20 2017-11-07 22:21:11

Head_on_a_Stick
Member
From: London
Registered: 2015-09-29
Posts: 8,759
Website

Re: What the shell? Bash, KornShell & zsh

^ Nice tips, thanks!

I too prefer to use POSIX-compliant scripting.


“Et ignotas animum dimittit in artes.” — Ovid, Metamorphoses, VIII., 18.

Forum Rules   •   How to report a problem   •   Software that rocks

Offline

#21 2017-11-08 01:59:21

johnraff
nullglob
From: Nagoya, Japan
Registered: 2015-09-09
Posts: 5,229
Website

Re: What the shell? Bash, KornShell & zsh

Head_on_a_Stick wrote:
johnraff wrote:

Just out of interest, are there any known unpatched vulnerabilities in the Debian Jessie bash 4.3-11+deb8u1 or Stretch 4.4-5?

I wouldn't think so, no, Debian are exceptional at pushing fixes.

However, Shellshock was present and undetected for many years so it is the unknown and potential vulnerabilities that worry me — I think a proactive attitude is vital if security is a concern.

Of course this attitude is admirable, and one I also try to emulate when writing scripts.

I am left wondering, though, why most of the major systems - including Arch - are still using bash as the default user shell.
Surely they have some pretty security-conscious devs thinking about these things?

---
Anyway, I agree about using POSIX compliant code when writing system scripts, unless bash offers some overwhelming advantage in some specific case. For general user-level fun and games utilities, when performance is irrelevant, I find bash scripting much more congenial, but others are entitled to their preferences.

I very seldom use the full bash functionality on the terminal, however - mainly the { } syntax, tab completion, and history parsing. Do they also work on ksh? (The lack of tab completion etc when switching to dash to test a bit of code always annoys the **** out of me.)


John
--------------------
( a boring Japan blog , Japan Links, idle twitterings  and GitStuff )
In case you forget, the rules.

Offline

#22 2017-11-08 07:14:23

Steve
Member
Registered: 2017-01-03
Posts: 642

Re: What the shell? Bash, KornShell & zsh

Head_on_a_Stick wrote:

Side question: why are you so hung up on DBus?

I think you will find that systemd is almost obsessive about launching sessions for anything with a pulse so perhaps try removing all that stuff?

Those scripts came with the distro im using which is using openrc not systemd. Its an arch spin, i should have mentioned this sorry. Archbang https://distrowatch.com/table.php?distribution=archbang

Thanks for that other info.

Offline

#23 2017-11-08 07:18:44

Steve
Member
Registered: 2017-01-03
Posts: 642

Re: What the shell? Bash, KornShell & zsh

Head_on_a_Stick wrote:

Actually, this reminds me...

Steve wrote:

im on arch linux, Ive just chsh to mksh

If you have the time, could you please check if this bug applies to your system:

https://bugs.archlinux.org/task/55053?p … tring=mksh

tl;dr: running `pkexec gparted` will throw an error unless a /usr/bin/mksh line is added to /etc/shells.

Also, my mksh AUR packages include a statically-linked version (using musl libc rather than the bloated GNU C libraries) which you may be interested in:

https://aur.archlinux.org/packages/?O=0&K=mksh

I installed this arch package https://www.archlinux.org/packages/comm … 6_64/mksh/

And yes this bug applies.

sc@#!:~ $ pkexec gparted                                                                 
The value for the SHELL variable was not found the /etc/shells file

This incident has been reported.
127|sc@#!:~ $ 

Offline

#24 2017-11-08 07:27:52

Head_on_a_Stick
Member
From: London
Registered: 2015-09-29
Posts: 8,759
Website

Re: What the shell? Bash, KornShell & zsh

johnraff wrote:

I am left wondering, though, why most of the major systems - including Arch - are still using bash as the default user shell.

Yes, me too hmm

In respect of Arch, they like to stick with the upstream defaults so it makes sense for them to choose bash.

Anyway, Android use KornShell (mksh specifically) rather than bash and that operating system is _much_ more security focused than GNU/Linux.

Surely they have some pretty security-conscious devs thinking about these things?

Maybe they left to use OpenBSD devil

I very seldom use the full bash functionality on the terminal, however - mainly the { } syntax, tab completion, and history parsing. Do they also work on ksh?

{} works fine (if you mean functions?) as does tab completion, although I haven't found out how to make it case-insensitive (yet).

I don't use a HISTFILE with KornShell but it is possible to set one up, I think.

Anyway, the best fully-featured interactive shell is zsh, the extended globbing is so good that you don't need a file manager glasses

I am prepared to sacrifice some of zsh's awesome convenience for KornShell's speed and simplicity.


“Et ignotas animum dimittit in artes.” — Ovid, Metamorphoses, VIII., 18.

Forum Rules   •   How to report a problem   •   Software that rocks

Offline

#25 2017-11-08 07:32:57

Steve
Member
Registered: 2017-01-03
Posts: 642

Re: What the shell? Bash, KornShell & zsh

Is it just a matter of putting /usr/bin/mksh into etc/shells file as per below?

#
# /etc/shells
#

/bin/sh
/bin/bash
/usr/bin/mksh
# End of file
/usr/bin/git-shell
/bin/mksh

Offline

Board footer

Powered by FluxBB