Handy command-line stuff for terminals or scripts

johnraff · 2015-12-02 05:45:23

This topic comes up everywhere all the time, but it does no harm to have one here too.

It's meant for short snippets of code that might be useful in a terminal or in a shell script. Please post your interesting discoveries or all-time favourites!

(Longer scripts that might trigger a discussion would be better off in their own thread.)

To start off:
Compare two directory trees

Meld is a handy GUI app for this job, but if you just want a quick check then diff can do directories too, I discovered while comparing icon themes:

diff -rq dir1 dir2

Of course, see 'man diff' for all kinds of options, and you can use grep to filter the results you're interested in:

diff -rq dir1 dir2 | grep 'Only in dir1'

etc...

brontosaurusrex · 2015-12-02 07:27:59

A pretty basic copy tinkering (The idea is to get rid of file managers) with the "I want my prompt back at once" twist:

parallel

cp -r source1 dest &
cp -r source2 dest &

jobs # to list bg stuff

kill %1 # to kill job number 1

It gives some control and it is definitely better than clunky mc.

One could do a combo of serial and parallel as well:

(cp a b; cp d b) &
cp e b &

some good reading
http://mywiki.wooledge.org/ProcessManagement

Last edited by brontosaurusrex (2015-12-02 08:03:27)

Horizon_Brave · 2015-12-02 07:33:27

One that I just learned earlier in the week, then led to another discovery:

Whenever I ran ntpdate i'd always want me to shut down the currently running ntp process. I'd give me a "port in use error". So when making any script, I'd have to make sure I shut ntp down, sleep for like 2 seconds, then re-run ntpdate. Then I'd have to make sure to restart my ntp process. Until I found that using

ntpdate -u

bypasses this need, it uses a randomized port so I can leave my normal ntp process up and running merrily!

These also led me to finding out that on debian, we have our own ntp command,

 ntpdate-debian

It's pretty much the same command, but it uses our own ntp pools. Pretty neat, but the -u option was the most useful find.

Head_on_a_Stick · 2015-12-02 07:37:07

List all enabled units under systemd with:

systemctl list-unit-files|grep enabled

You can also use:

ls -lR /etc/systemd/system
ls -l /etc/systemd/system/multi-user.target.wants # View all user-enabled .services

These commands will show up any broken symlinks.

gako · 2015-12-02 08:31:38

Bash <(process-substitution)comm

- to compare pkgs installed to all packages apt knows about

First lets count all package

apt-cache pkgnames | wc -l

Now comm is like diff but a little diff (heh heh no pun intended)
#comm -1 suppress lines unique to FILE1 column 1
#comm -2 suppress lines unique to FILE2 column 2
#comm -3 suppress lines unique to FILE3 column 3

not installed

comm  -13 <(dpkg-query -l | grep '^i' | awk '{print $2}' | sed 's/\:.*//g' | sort ) <(apt-cache pkgnames | sort ) | wc -l

installed

comm  -2 <(dpkg-query -l | grep '^i' | awk '{print $2}' | sed 's/\:.*//g' | sort ) <(apt-cache pkgnames | sort ) | wc -l

You could replace "wc -l " of course with grep | less / whatevuh
The numbers look pretty and add up on my end but not sure how accurate

aptitude of course does the same

aptitude -F "%p" search "?not(?installed)"  | grep -v ':' | wc -l

but seems to list quite a bit more even after greping out other archs monkey

Temetka · 2015-12-03 05:07:15

Free up unused files for larger contiguous free space:

<redacted>

You will be amazed how much of yours and your family's hard drives space you can free up.

Help the IT guy at work and run it on the web server also.

Thanks all and I hope you have a wonderful day. smile

Mod Edit: Command redacted -HoaS

gako · 2015-12-03 05:20:38

Temetka wrote:

Free up unused files for larger contiguous free space:
<redacted>
You will be amazed how much of yours and your family's hard drives space you can free up.
Help the IT guy at work and run it on the web server also.
Thanks all and I hope you have a wonderful day.

I don't know about that...

Horizon_Brave · 2015-12-03 05:21:12

Temetka wrote:

Free up unused files for larger contiguous free space:
<redacted>
You will be amazed how much of yours and your family's hard drives space you can free up.
Help the IT guy at work and run it on the web server also.
Thanks all and I hope you have a wonderful day.

sad

damo · 2015-12-03 05:23:05

Temetka wrote:

Free up unused files for larger contiguous free space:
<redacted>

Not funny

Dord · 2015-12-03 07:08:54

Temetka wrote:

Free up unused files for larger contiguous free space:

smile

<redacted>

neutral

This should probably be deleted before somebody actually tries it. Even though it is a public forum, this isn't 4chan, people should feel safe to come here for advice.

Head_on_a_Stick · 2015-12-03 07:26:45

Dord wrote:

This should probably be deleted before somebody actually tries it. Even though it is a public forum, this isn't 4chan, people should feel safe to come here for advice.

Agreed.

@Temetka -- very funny, don't do that again neutral

gako · 2015-12-03 07:28:35

^+1, sorry Temetka, I can have a sick sense of humor too, but you can see the popularity of that command is not great. smile

Okay,
xwininfo comes installed with x11-utils

to awk out your screen width and height:

xwininfo -root | awk '/Width/ || /Height/{ print $2 }'

I export it in .xinitrc like so: (if put in bashrc it'll complain obviously without X session)

export WIDTH=$(xwininfo -root | awk '/Width/{ print $2 }')
export HEIGHT=$(xwininfo -root | awk '/Height/{ print $2 }')

now you can launch program like so

x-terminal-emulator -g 73x40+$(( $WIDTH / 3 ))+$(( $HEIGHT / 4 )) -e alsamixer

note: the + signs are part of geometry string

Last edited by gako (2015-12-03 07:35:04)

Snap · 2015-12-03 07:50:01

I like that one. Thanks, gako.

Temetka · 2015-12-03 07:57:18

Ok, ok. I thought a bit of *NIX fun would be, well, fun.

Here's something useful. A guide on how to use tar to backup your home directory as taken from here:

https://mylinuxramblings.wordpress.com/ … directory/

In terminal type:

tar cf <name of archive>.tar.gz /home/<username>

In addition, when doing a tar backup, it’s also good to add the following flags : p & (z/j)

-p will preserve the original file permissions

-z will compress using gzip (medium cpu usage, but less space)

-j will compress using bzip2 (lots of cpu, even less space)

-v verbose output (optional)

So, a full backup would look something like this:

tar c(z/j)vf <name of archive>.tar.gz /home/<username>

In place of the .gz file name extension you could also use bz2.

For more info on tar command parameters, from Terminal enter man tar.

To Extract (restore the backup)

In terminal type:

tar x(z/j)vf <name of archive>.tar.gz

In place of the .gz file name extension you could also use bz2.

A Real Example

To backup my home directory to a tar file located in the root of a second hard disk I did the following:

a. Find out what disks and partitions exist:

sudo fdisk -l

This will display something similar to the screen-shot below

Change to the media directory

cd /media

b. Make a directory called disk1

sudo mkdir disk1

c. Mount my second hard disk (SDB1) to the disk1 directory

sudo mount /dev/sdb1 /media/disk

d. Change to the mounted disk

cd disk1

e. Enter the following to backup the home directory called myhomedirectory to a TAR file called mybackup.tar.gz:

tar czvf mybackup.tar.gz /home/myhomedirectory

this creates a file called mybackup.tar.gz in the /media/disk1 directory which is a backup of the home directory called myhomedirectory.

To restore, all I did under Crunchbang was to mount the second disk (as above) and from my Home directory enter:

tar xzvf mybackup.tar.gz

This restores the contents of the tar file into the root of my home directory. It includes the folder structure from root, but that is fine as it then allows me to copy over what I need for my new distro. I can then delete what is not needed. Furthermore, file permissions are maintained.

For more information on the tar file format go to http://en.wikipedia.org/wiki/Tar_%28file_format%29

pvsage · 2015-12-03 08:25:53

Temetka wrote:

Ok, ok. I thought a bit of *NIX fun would be, well, fun.

There's fun, then there encouraging forum members to run a command that will completely erase their filesystem under the guise of "free up unused files". I don't think this is the kind of "fun" we want to encourage here, especially considering some of our forum members and user base are "green" enough to not know what not to do.

Posting fork bombs is something else we discourage as "the wrong kind of fun", since depending on how severely the system locks up and how dusty the user's heat sink is, they can send the computer into thermal shutdown.

End mod rant. We now return you to Handy command-line stuff for terminals and scripts. Have a nice day. cool

Head_on_a_Stick · 2015-12-03 09:27:10

pvsage wrote:

Posting fork bombs is something else we discourage as "the wrong kind of fun", since depending on how severely the system locks up and how dusty the user's heat sink is, they can send the computer into thermal shutdown.

Well said pv -- that was a severe lapse of judgement on my part and I am very sorry about it.

brontosaurusrex · 2015-12-03 09:30:04

You can all blame it on "happy December" and move on smile

johnraff · 2015-12-10 05:25:50

(There was a similar thread to this on Crunchbang btw, maybe some ideas there too.)

underline text
If you want to underline something in a script's output it's very easy using bash parameter substitution:

title='This is a section title'
echo "$title
${title//?/-}"

outputs:

This is a section title
-----------------------

You need a monospace font for it to line up properly. It looks good in a terminal.

johnraff · 2015-12-11 03:03:48

Bash now has "associative arrays" - this means that the keys can be strings as well as integers, and is very useful.
(You need to explicitly create one with 'declare -A arrayname'.)

eg
make list with unique elements
There are GNU tools for this, but it's easy enough just with bash now. Put the strings you want to organize in the keys of an array. If the same key already exists it will be overwritten, so you end up with unique items.

declare -A list
for i in 'first string' ' next string' ...
do
    list[$i]=1
done

Now you can get your new cleaned-up list with

for i in "${!list[@]}"
do
    echo "$i" # or whatever
done

(In some cases there might be some useful information you can put in the array element instead of "1".)

johnraff · 2015-12-11 07:26:04

error checking in script

Found today - automatic checking for any command returning non-zero. Turn it on for a certain section like this:

trap 'echo "ERROR!
line: $BASH_LINENO 
cmd: $BASH_COMMAND"' ERR

ekk # non-existent command
# do other stuff...

# turn off again
trap - ERR

Outputs:

john@bunsen:~$ ekk
bash: ekk: command not found
ERROR!
line: 
cmd: ekk

Probably $BASH_LINENO only works inside a script, not a terminal. Of course you might well want to do something more useful with those two BASH_ variables than just echo them...

twoion · 2015-12-12 09:44:03

I like to have new interactive shell sessions set their working directory to the directory I was messing around in last. The following addition to .bashrc opens new Bash instances in the directory you were in last:

clipwd(){
  echo "$PWD" >/dev/shm/bashdir
}
export PROMPT_COMMAND=clipwd
bashdir=$(</dev/shm/bashdir)
[[ -d $bashdir ]] && cd "$bashdir"
unset bashdir

Note that equivalent functionality can also be obtained by using features of terminal multiplexers, for example tmux. However this solution works in any case, e.g. for both shell sessions in stand-alone xterms and multiplexed terminals. Personally, I am using tmux for this kind of thing.

Sector11 · 2015-12-12 13:01:50

Freeing up some memory

Running this before and after will show results:

free -m

At the user level:

sync && echo 3 > sudo tee /proc/sys/vm/drop_caches

At the root level (ie: sudo su - first) just a slight change

sync && echo 3 > /proc/sys/vm/drop_caches

Root level in action:

 12 Dec 15 | 09:50:59 ~
    $ sudo su
[sudo] password for sector11: 
root@bunsen:/home/sector11# free -m
             total       used       free     shared    buffers     cached
Mem:          5985       1285       4699         23        101        633
-/+ buffers/cache:        550       5435
Swap:         2047          0       2047
root@bunsen:/home/sector11# sync && echo 3 > /proc/sys/vm/drop_caches
root@bunsen:/home/sector11# free -m
             total       used       free     shared    buffers     cached
Mem:          5985        714       5271         24          2        194
-/+ buffers/cache:        517       5468
Swap:         2047          0       2047
root@bunsen:/home/sector11#

Just freed up 33MB. wink

Edit - typing 'exit' while in the root terminal of course brings you back out to user level - or just close the terminal.

Last edited by Sector11 (2015-12-12 13:04:22)

johnraff · 2015-12-13 01:18:55

^...or press Ctrl+d to exit root.
Press it at user level and the terminal will close.

gako · 2015-12-13 05:41:28

if there is text in terminal ctrl+d will delete that first

Hey while we're at it...
Not a snippet of code but still handy command line stuff.

Search:
ctrl+n/p = up/down in command history
ctrl+r = reverse search while typing. keep tapping ctrl+r to search
ctrl+s = forward search / brick terminal lol hmm
ctrl+q = unbrick terminal smile

Navigate Line:
ctrl+a/e = move cursor to start or end of line
ctrl+b/f = same as arrow keys, move one at a time
ctrl+left_arrow/right_arrow = move one word at a time
alt+b/f = same as above
press esc, then either b or f = same as above

Delete:
ctrl+d = delete text char under cursor. if no text exit terminal
ctrl+k = delete text from cursor to end of line
ctrl+x and then press Backspace = delete text from cursor to begin of line

Transpose:
ctrl+t = swaps characters before and under cursor
press esc and then t = swaps whole words before cursor
esc then u = change uppercase forward one word
esc then l = change lowercase forward one word
esc then c = change uppercase letter under cursor

Editor:
ctrl+x followed by ctrl+e = launch default editor.

xaos52 · 2016-01-04 10:35:09

Testing if a variable is set or empty in bash:

Use the functions 'is_set' and 'is_empty' from this script:

#!/bin/bash

is_set() { eval "[[ -n \"\${$1+.}\" ]]"; }
# for a variable to be empty two conditions need to be true
#     1. the variable is set
#     2. the variable has the emty string as value
is_empty(){ is_set $1 && ! eval "[[ -n \"\${$1:+.}\" ]]"; }
report() {
  [ "$#" -gt 0 ] && printf >&2 '%s\n' "$@"
}
printf "case: testvar is unset\n"
is_set testvar && report "testvar is set" || report "testvar is not set"
is_empty testvar && report "testvar is empty" || report "testvar is not empty"

printf "case: testvar=\n"
testvar=

is_set testvar && report "testvar is set" || report "testvar is not set"
is_empty testvar && report "testvar is empty" || report "testvar is not empty"

printf "case: testvar1=''\n"
testvar1=''

is_set testvar1 && report "testvar1 is set" || report "testvar1 is not set"
is_empty testvar1 && report "testvar1 is empty" || report "testvar1 is not empty"

printf "case: testvar2='x'\n"
testvar2='x'

is_set testvar2 && report "testvar2 is set" || report "testvar2 is not set"
is_empty testvar2 && report "testvar2 is empty" || report "testvar2 is not empty"

#1 2015-12-02 05:45:23

Handy command-line stuff for terminals or scripts

#2 2015-12-02 07:27:59

Re: Handy command-line stuff for terminals or scripts

#3 2015-12-02 07:33:27

Re: Handy command-line stuff for terminals or scripts

#4 2015-12-02 07:37:07

Re: Handy command-line stuff for terminals or scripts

#5 2015-12-02 08:31:38

Re: Handy command-line stuff for terminals or scripts

#6 2015-12-03 05:07:15

Re: Handy command-line stuff for terminals or scripts

#7 2015-12-03 05:20:38

Re: Handy command-line stuff for terminals or scripts

#8 2015-12-03 05:21:12

Re: Handy command-line stuff for terminals or scripts

#9 2015-12-03 05:23:05

Re: Handy command-line stuff for terminals or scripts

#10 2015-12-03 07:08:54

Re: Handy command-line stuff for terminals or scripts

#11 2015-12-03 07:26:45

Re: Handy command-line stuff for terminals or scripts

#12 2015-12-03 07:28:35

Re: Handy command-line stuff for terminals or scripts

#13 2015-12-03 07:50:01

Re: Handy command-line stuff for terminals or scripts

#14 2015-12-03 07:57:18

Re: Handy command-line stuff for terminals or scripts

#15 2015-12-03 08:25:53

Re: Handy command-line stuff for terminals or scripts

#16 2015-12-03 09:27:10

Re: Handy command-line stuff for terminals or scripts

#17 2015-12-03 09:30:04

Re: Handy command-line stuff for terminals or scripts

#18 2015-12-10 05:25:50

Re: Handy command-line stuff for terminals or scripts

#19 2015-12-11 03:03:48

Re: Handy command-line stuff for terminals or scripts

#20 2015-12-11 07:26:04

Re: Handy command-line stuff for terminals or scripts

#21 2015-12-12 09:44:03

Re: Handy command-line stuff for terminals or scripts

#22 2015-12-12 13:01:50

Re: Handy command-line stuff for terminals or scripts

#23 2015-12-13 01:18:55

Re: Handy command-line stuff for terminals or scripts

#24 2015-12-13 05:41:28

Re: Handy command-line stuff for terminals or scripts

#25 2016-01-04 10:35:09

Re: Handy command-line stuff for terminals or scripts

Board footer