What the shell? Bash, KornShell & zsh

Head_on_a_Stick · 2017-11-04 14:37:13

Quick demonstration of the inherent speed advantage that the original KornShell (ksh93) has over the latest version of bash:

$ time assoc-arr-test-bash
[...]
    0m19.01s real     0m18.99s user     0m00.01s system
$ time assoc-arr-test-ksh93
[...]
    0m02.07s real     0m02.06s user     0m00.00s system

That's quite a boost 8)

The assoc-arr-test-$shell script is from here (the only difference between the two versions is the shebang):

http://www.tldp.org/LDP/abs/html/abs-gu … SOCARRTEST

I will have to see if I can convert the BunsenLabs bash scripts to ksh93 and test for any improvements.

malm · 2017-11-04 17:31:30

^ How does it compare with dash?

Head_on_a_Stick · 2017-11-04 18:05:09

malm wrote:

How does it compare with dash?

I did a (very limited) test in my mksh thread and it is even quicker than dash:

https://forums.bunsenlabs.org/viewtopic.php?id=3794

Quite astonishing for a shell that stopped being developed in 2012

EDIT: having said that, it was proprietary (and rather expensive) before it was open-sourced so the code must be of very high quality.

Last edited by Head_on_a_Stick (2017-11-04 18:09:33)

Head_on_a_Stick · 2017-11-04 21:06:32

nobody wrote:

ksh93 has a number of unholy features

lol.

I particularly like the shcomp(1) utility that compiles ksh93 scripts into binary executables.

Last edited by Head_on_a_Stick (2017-11-04 21:07:42)

malm · 2017-11-04 21:43:07

@Head_on_a_Stick, @nobody: Thanks for the education. I had a quick browse of the code at https://github.com/att/ast/blob/master/ … /sh/main.c. Absolutely poetic. And I love the copyright - 1982-2012

ratcheer · 2017-11-06 14:31:43

I used ksh for a long time on Solaris and other enterprise UNIXs, but I've never really dug into it on Linux. I had no idea it might perform so much better.

Tim

Head_on_a_Stick · 2017-11-06 21:36:55

ratcheer wrote:

I had no idea it might perform so much better

It's not just performance: bash has a parlous history of security vulnerabilities, so much so that some in the BSD community refer to it as the broken again shell.

ksh93 has had 30+ years of development in professional UNIX environments and mksh is used as the login shell in Android, thus giving it many more "eyes on the code" than the alternatives, not to mention the much smaller codebase reducing the attack surface significantly.

Sadly, ksh93 is a tricky blighter to package because it needs AT&T's Software Technology (AST) toolkit and the associated (and very idiosyncratic) build system and that doesn't work with all operating systems.

Arch only has it in the AUR and the PKGBUILD is insanely complicated but it is a (slightly) newer version than the one available from Debian in their official repositories.

On the other hand, mksh has been ported to just about everything and so allows a uniform shell environment across all platforms, which is nice.

Sector11 · 2017-11-06 21:57:10

Can mksh replace "bash" easily in a distro?

Would existing bash scripts need modifying?

Head_on_a_Stick · 2017-11-06 22:47:00

Sector11 wrote:

Can mksh replace "bash" easily in a distro?

Yes, I use it as the default login shell for both root and all normal users in all my Linux systems, it only takes a single command.

Would existing bash scripts need modifying?

No.

Changing the user's default login shell does not mean that any system scripts need to be changed because they should all have #!/bin/bash as the first line and so will still call bash regardless of which login shell is set.

There may be some features of bash that you would miss for interactive use, you would have to try it and find out for yourself.

ProTip: use the supplied example configuration files, they provide many useful functions and aliases.

Sector11 · 2017-11-06 23:06:51

@HoaS

Thank you ... food for thought.

I was wondering if it would 'replace' bash.

johnraff · 2017-11-07 02:10:32

Head_on_a_Stick wrote:

bash has a parlous history of security vulnerabilities

Just out of interest, are there any known unpatched vulnerabilities in the Debian Jessie bash 4.3-11+deb8u1 or Stretch 4.4-5?

ohnonot · 2017-11-07 06:06:53

Head_on_a_Stick wrote:

Would existing bash scripts need modifying?
No.

let me rephrase that:
Would existing shell scripts (i.e. #/bin/sh) need modifying?
also, can you expand on differences/similarities between mksh and ksh? would mksh work as a drop-in replacement for ksh? is it equally lightweight?

Head_on_a_Stick · 2017-11-07 07:02:30

johnraff wrote:

Just out of interest, are there any known unpatched vulnerabilities in the Debian Jessie bash 4.3-11+deb8u1 or Stretch 4.4-5?

I wouldn't think so, no, Debian are exceptional at pushing fixes.

However, Shellshock was present and undetected for many years so it is the unknown and potential vulnerabilities that worry me — I think a proactive attitude is vital if security is a concern.

ohnonot wrote:

Would existing shell scripts (i.e. #/bin/sh) need modifying?

No because !#/bin/sh scripts should not be using bashisms anyway

Also, /bin/sh will remain symlinked to dash so that will be called irrespective of the user's login shell.

At any rate, the POSIX sh standard (IEE 1003.1-2008) is actually based on ksh88 so compatibility is pretty much guaranteed.

ohnonot wrote:

can you expand on differences/similarities between mksh and ksh? would mksh work as a drop-in replacement for ksh? is it equally lightweight?

ksh93 is a full-featured shell that aims to do everything, just like bash and zsh, whereas mksh is based on OpenBSD's public domain implementation of the KornShell programming language (ie, it shares no code with ksh93) and it inherits that operating system's tendency to strip out and remove any insecure or extraneous features resulting in a more "bare bones" shell but also one that is much lighter in respect of resource usage, see my mksh thread for some crude tests of memory usage:

https://forums.bunsenlabs.org/viewtopic.php?id=3794

So mksh can't be dropped in as a replacement for bash or ksh93 in scripts because it lacks some features (most notably associative arrays) but the codebase is smaller than that of the more able competition and it is faster than all but ksh93 (and dash).

Last edited by Head_on_a_Stick (2017-11-07 07:06:47)

Steve · 2017-11-07 08:05:06

@ Hoas, im on arch linux, Ive just chsh to mksh and made a few changes, added .mkshrc from /etc/skel/.mkshrc to ~/.mkshrc

Have a question on .xinitrc script im using in archbang, would i change it to look like below as far as calling the script?

#!/usr/bin/mksh

export DE=openbox

# Dbus fix
if [ -z "$DBUS_SESSION_BUS_ADDRESS" ]; then
    eval "$(dbus-launch --sh-syntax --exit-with-session)"
fi

/usr/bin/xdg-user-dirs-update

exec dbus-launch --sh-syntax --exit-with-session openbox-session

ive also changed bash_profile to .profile and it looks like this and working ok.

# System settings before starting X
. $HOME/.mkshrc

PATH=$PATH:$HOME/bin

# set up alsa
/usr/bin/amixer sset Master Mono 90% unmute  &> /dev/null
/usr/bin/amixer sset Master 90% unmute  &> /dev/null
/usr/bin/amixer sset PCM 90% unmute &> /dev/null

# Start x on login
if [[ -z $DISPLAY && ! -e /tmp/.X11-unix/X0 ]]; then
   exec xinit -- /usr/bin/X -nolisten tcp vt7
fi

Last edited by Steve (2017-11-07 08:16:10)

Head_on_a_Stick · 2017-11-07 20:14:52

Mod Note: I seemed to have developed a distinct topic in the completely off-topic thread and that was off-topic, even for Off Topic (or something). Anyway, I've split it out so people can either join the noise discussion or ignore the thread, I hope the participants don't mind.

Steve wrote:

on .xinitrc script im using in archbang, would i change it to look like below as far as calling the script?

Personally, I do not add a shebang to ~/.xinitrc because that ensures that the default login shell is used at all times but yes, adding #!/bin/mksh at the top will force the issue.

Side question: why are you so hung up on DBus?

I think you will find that systemd is almost obsessive about launching sessions for anything with a pulse so perhaps try removing all that stuff?

Your checks appear to use POSIX conditionals so you could in fact declare #!/bin/sh and gain a (tiny) speed advantage by re-linking /bin/sh from bash (yes, Arch does that as stock) to dash instead (as Debian does from stock):

https://wiki.archlinux.org/index.php/Da … 2Fbin.2Fsh

Steve wrote:

ive also changed bash_profile to .profile

In Arch, the bash package supplies ~/.bash_profile and if that file is present then bash will not read ~/.profile when it starts up.

However, in Debian bash has ~/.profile instead (which it will read automatically) thus unifying configuration across all shells so if you remove ~/.bash_profile in your Arch box then you can switch to ~/.profile for bash and mksh (and ksh93, zsh, ash, tcsh, rc, fish, etc) and simplify things a bit.

Other points:

quote your variables!
for PATH, place ~/bin before :$PATH to let custom programs be run in preference
no need to source ~/.mkshrc, it will do that automatically unless $ENV is set
the `xinit` command hasn't listened to port 80 for a while now
quote your variables! I know I've said this before but it's such an important point that it's worth mentioning twice

hth

malm · 2017-11-07 21:40:55

I like to write POSIX compliant shell.

Have found this useful https://manpages.debian.org/testing/dev … .1.en.html

Head_on_a_Stick · 2017-11-07 21:45:30

Actually, this reminds me...

Steve wrote:

im on arch linux, Ive just chsh to mksh

If you have the time, could you please check if this bug applies to your system:

https://bugs.archlinux.org/task/55053?p … tring=mksh

tl;dr: running `pkexec gparted` will throw an error unless a /usr/bin/mksh line is added to /etc/shells.

Also, my mksh AUR packages include a statically-linked version (using musl libc rather than the bloated GNU C libraries) which you may be interested in:

https://aur.archlinux.org/packages/?O=0&K=mksh

malm · 2017-11-07 22:19:55

Staying clear of GNU/bashisms can be tricky.

These are useful notes on writing 'portable' shell https://github.com/git/git/blob/master/ … elines#L41

Using dash helps.

A few that I've learnt the hard way:

Don't use arrays and [[ ]]
Avoid non-POSIX re-directions such as '&>', '>&-', '|&', etc
For example, to re-direct both stdout and stderr to /dev/null, do:
```
command >/dev/null 2>&1
```
Don't use echo -e "foo", instead use
```
printf "%b\n" "foo"
```
Don't use sed -i, it is not POSIX
Don't use curly brace globbing such as "ls *.{c,h}"
[*/]

Last edited by malm (2017-11-07 22:22:04)

Head_on_a_Stick · 2017-11-07 22:21:11

^ Nice tips, thanks!

I too prefer to use POSIX-compliant scripting.

johnraff · 2017-11-08 01:59:21

Head_on_a_Stick wrote:

johnraff wrote:
Just out of interest, are there any known unpatched vulnerabilities in the Debian Jessie bash 4.3-11+deb8u1 or Stretch 4.4-5?
I wouldn't think so, no, Debian are exceptional at pushing fixes.
However, Shellshock was present and undetected for many years so it is the unknown and potential vulnerabilities that worry me — I think a proactive attitude is vital if security is a concern.

Of course this attitude is admirable, and one I also try to emulate when writing scripts.

I am left wondering, though, why most of the major systems - including Arch - are still using bash as the default user shell.
Surely they have some pretty security-conscious devs thinking about these things?

---
Anyway, I agree about using POSIX compliant code when writing system scripts, unless bash offers some overwhelming advantage in some specific case. For general user-level fun and games utilities, when performance is irrelevant, I find bash scripting much more congenial, but others are entitled to their preferences.

I very seldom use the full bash functionality on the terminal, however - mainly the { } syntax, tab completion, and history parsing. Do they also work on ksh? (The lack of tab completion etc when switching to dash to test a bit of code always annoys the **** out of me.)

#1 2017-11-04 14:37:13

What the shell? Bash, KornShell & zsh

#2 2017-11-04 17:31:30

Re: What the shell? Bash, KornShell & zsh

#3 2017-11-04 18:05:09

Re: What the shell? Bash, KornShell & zsh

#4 2017-11-04 21:06:32

Re: What the shell? Bash, KornShell & zsh

#5 2017-11-04 21:43:07

Re: What the shell? Bash, KornShell & zsh

#6 2017-11-06 14:31:43

Re: What the shell? Bash, KornShell & zsh

#7 2017-11-06 21:36:55

Re: What the shell? Bash, KornShell & zsh

#8 2017-11-06 21:57:10

Re: What the shell? Bash, KornShell & zsh

#9 2017-11-06 22:47:00

Re: What the shell? Bash, KornShell & zsh

#10 2017-11-06 23:06:51

Re: What the shell? Bash, KornShell & zsh

#11 2017-11-07 02:10:32

Re: What the shell? Bash, KornShell & zsh

#12 2017-11-07 06:06:53

Re: What the shell? Bash, KornShell & zsh

#13 2017-11-07 07:02:30

Re: What the shell? Bash, KornShell & zsh

#14 2017-11-07 08:05:06

Re: What the shell? Bash, KornShell & zsh

#15 2017-11-07 20:14:52

Re: What the shell? Bash, KornShell & zsh

#16 2017-11-07 21:40:55

Re: What the shell? Bash, KornShell & zsh

#17 2017-11-07 21:45:30

Re: What the shell? Bash, KornShell & zsh

#18 2017-11-07 22:19:55

Re: What the shell? Bash, KornShell & zsh

#19 2017-11-07 22:21:11

Re: What the shell? Bash, KornShell & zsh

#20 2017-11-08 01:59:21

Re: What the shell? Bash, KornShell & zsh

Board footer