You are not logged in.

  1. Index
  2. » Dev Discussions
  3. » Best algorithm for bl-user-setup

Pages: 1 2

#21 2020-01-22 07:41:55

johnraff
nullglob
From: Nagoya, Japan
Registered: 2015-09-09
Posts: 12,557
Website

Re: Best algorithm for bl-user-setup

Hmm... maybe this is getting somewhere near close. smile

The paths set at the top are still for testing, but it seems to be doing something like what I wrote in the posts above.

Launched with --install option before starting X so initial import doesn't need a logout/in. After that, blocked by the same flagfile ~/.config/bunsen/bl-setup as at present.

Launched with --auto in every user login: in this case checks if skel has changed, and if not exits immediately.

I tried to make those two options exit as fast as possible when there's nothing new in skel - which is the case most of the time. On an old slow laptop - Centrino 1GHx CPU, 1GB RAM - they both exit in ~ 40ms. Of course on a modern computer, much faster. I guess that's an acceptable slowdown of the login time?

The interactive options have to wait for user response, so time is less important. The default for a user is to run 'bl-user-setup' with no options, though '--refresh' is also accepted for compatibility.

See 'bl-user-setup --help' for a bit more info.

There is a cache directory created in ~/.cache/bunsen/bl-setup to hold the previous skel info, but since this is in the user's domain, and might be deleted, I tried to make the script proofed against that by reverting to some user prompts when necessary.

I've tried to make the code as readable as I could (and thanks to Johan for some formatting ideas) but please ask about anything that seems not to make sense.

There's a tarball below with a test directory ready to play with without messing up your real system.

#!/bin/bash
#
#    bl-user-setup: a script to populate a new user's HOME directory
#    using template files from /usr/share/bunsen/skel/
#    Copyright: 2015-2020 John Crawley <john@bunsenlabs.org>
#               2019      Johan Malm <jgm323@gmail.com>
#
#    This program is free software: you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation, either version 3 of the License, or
#    (at your option) any later version.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program.  If not, see <http://www.gnu.org/licenses/>.

USAGE="bl-user-setup: a script to populate a new user's HOME directory
using files from /usr/share/bunsen/skel

This script is run non-interactively on a user's first login,
and on subsequent logins to check for new default configurations
in /usr/share/bunsen/skel that may have come from a package upgrade.
User set configurations will not be modified without permission.

It can also be run manually with custom options.
The most common manual use case is to run with no options,
in which case the algorithm described below is followed.

In all cases backups will be made of any files which are overwritten.

Options:
-h --help
    Show this message.
--refresh
    This is the default behaviour with no options passed:
    '--refresh' is kept only for backward compatibility.
    If any files in /usr/share/bunsen/skel have changed,
    follow the algorithm below to determine whether to update them
    and whether to prompt the user first.
--addonly --no
    Assume all prompts answered 'no'.
--force --yes
    Assume all prompts answered 'yes'.
--reset
    Implies --force.
    Restore all files to the default state in /usr/share/bunsen/skel.

The following two options are not usually of interest to users:
--auto
    This is run on each user login.
    Do a quick check for any change to skel contents,
    exit if there is nothing new,
    or apply the same algorithm as --refresh if there are changes.
--install
    Implies --force.
    Run non-interactive file import only on first login.

ALGORITHM

Compare the contents of $HOME/.config/bunsen/skel (here called 'skel')
with the contents on the last run of this script.
If there has been no change, exit immediately.
If there is new content:
Copy in any new directories.
Copy in any new symlinks, if they do not already exist (no overwrites).
Copy in any new files.
If file in skel has changed, and user's file is still in default state,
silently update file.
If skel file and user file have both changed (or user has removed file),
 ask user for instructions.
"

### RESTORE THESE SETTINGS BEFORE RELEASE ###

readonly USER=${USER}

#readonly TARGETDIR="${HOME}"
readonly TARGETDIR="./home"
#readonly skeldir="/usr/share/bunsen/skel"
readonly skeldir="./skel"
#readonly confdir="$HOME/.config/bunsen"
readonly confdir="./config"
#readonly cachedir="$HOME/.cache/bunsen/bl-setup"
readonly cachedir="./cache"

readonly flagfile="$confdir/bl-setup"
readonly logfile="$cachedir/log"

########################################

main() {

    g_addonly=f
    g_force=f
    g_reset=f
    copy_all=f
    wholeskelsum=

    pre_checks

    case "$1" in
    --install)
        [[ -f "$flagfile" ]] && exit 0
        g_force=t
        ;;
    --auto)
        check_skel_sum
        ;;
    ''|--refresh)
        :
        ;;
    --addonly|--no)
        g_addonly=t
        ;;
    --force|--yes)
        g_force=t
        ;;
    --reset)
        g_reset=t
        g_force=t
        ;;
    -h|--help)
        printf '%b\n' "$USAGE"
        exit 0
        ;;
    *)
        die "$1 is not a valid option"
        ;;
    esac

    mkdir -p "$TARGETDIR" "$confdir" "$cachedir"

    bkp_sfx="~$( date +%FT%T )~"

    log "############
Running $0 at $(date)"

    check_skel_contents

    copy_dirs
    copy_links
    copy_files

    echo "Do not delete this file.

If it is not present, the import of config files from $skeldir
will be repeated, overwriting any user modifications.
(However, backups will be made of overwritten files.)" > "$flagfile"

    # Update cache of skel checksum if script finishes with no errors
    [[ -z $wholeskelsum ]] && {
        wholeskelsum=$( tar -cf - -C "${skeldir%/*}" "${skeldir##*/}" | md5sum )
        wholeskelsum=${wholeskelsum%% *}
    }
    echo "$wholeskelsum" > "$cachedir/skelsum"
}

##############

log () {
    echo "$@" >> "$logfile"
    [[ -t 1 ]] && {
        printf '\033[32m=>\033[m %s\n' "$@"
    }
}

die  () {
    log "ERROR: $*"
    printf '%b\n' "$0: ERROR: $*" >&2
    exit 1
}

ask_yesno() {
    local message="$1"
    if [[ -t 0 && -t 1 ]] # on terminal
    then
        echo "$message"
        while true
        do
            echo "y: Yes
n: No
(Default is Yes.)"
            read -r -p 'Y/n: '
            case "${REPLY^}" in
            Y|'') return 0;;
            N) return 1;;
            esac
        done
    else
        title="BunsenLabs User Setup"
        winicon="distributor-logo-bunsenlabs"
        msg="<b>User Setup</b>

$message"
        yad --center --undecorated --borders=20 --window-icon="$winicon"\
        --title="$title" --text="$msg"\
        --button='Yes':0 --button='No':1
    fi
}

ask_overwrite() {
    local file="$1"
    if [[ -t 0 && -t 1 ]] # on terminal
    then
        echo "You have modified this file:
$file
Overwrite with the new version?"
        while true
        do
            echo "y: overwrite
n: skip this one
a: overwrite all
s: skip all
(Default is skip.)"
            read -r -p 'y/N/a/s: '
            case "${REPLY^}" in
            Y) return 0;;
            N|'') return 1;;
            A) return 2;;
            S) return 3;;
            esac
        done
    else
        title="BunsenLabs User Setup"
        winicon="distributor-logo-bunsenlabs"
        msg="<b>User Setup</b>

You have modified this file:
$file
Overwrite with the new version, or skip?"
        yad --center --undecorated --borders=20 --window-icon="$winicon"\
        --title="$title" --text="$msg"\
        --button='Overwrite':0 --button='Skip':1 --button='Do All':2 --button='Skip All':3
    fi
}

##############

pre_checks () {
    # Do not apply BL configs to the root account.
    [[ "$USER" = root ]] && die "USER is root, abort"

    # Check for the user home directory
    [[ -d "$HOME" ]] || die "User home directory <$HOME> is not set or not a directory."

    # Check for source (skel) directory
    [[ -d "$skeldir" ]] || die "Source directory (skel) is not set or not a directory."
}

check_skel_sum() {
    if [[ -f $cachedir/skelsum ]]
    then
        wholeskelsum=$( tar -cf - -C "${skeldir%/*}" "${skeldir##*/}" | md5sum )
        wholeskelsum=${wholeskelsum%% *}
        # If skel directory unchanged since last run, exit.
        [[ $wholeskelsum = $( <"$cachedir/skelsum" ) ]] && exit 0
        ask_yesno "Some default configuration files have been updated
in a recent package upgrade.
Would you like to import these changes into your HOME directory?
(Files you have edited will not be overwritten without permission.)"
        [[ $? = 1 ]] && {
            log "User declined a config file update"
            echo "$wholeskelsum" > "$cachedir/skelsum" # so user doesn't get repeated prompts
            exit 0
        }
    fi
}

# Directories and links will only be added to the "to_copy" lists
# if they are new arrivals in skeldir.
#
# Files will also be added if the md5sum has changed.

check_skel_contents() {
    dirs_to_copy=()
    mapfile -t new_skel_dirs < <( find "$skeldir" -type d )
    if [[ -f $cachedir/skel.dirs && $g_reset = f ]]
    then
        mapfile -t old_skel_dirs < "$cachedir/skel.dirs"
        mapfile -t dirs_to_copy < <(comm -13 <(printf '%s\n' "${old_skel_dirs[@]}" | sort) <(printf '%s\n' "${new_skel_dirs[@]}" | sort))
    else
        dirs_to_copy=("${new_skel_dirs[@]}")
    fi

    links_to_copy=()
    mapfile -t new_skel_links < <( find "$skeldir" -type l )
    if [[ -f $cachedir/skel.links && $g_reset = f ]]
    then
        mapfile -t old_skel_links < "$cachedir/skel.links"
        mapfile -t links_to_copy < <(comm -13 <(printf '%s\n' "${old_skel_links[@]}" | sort) <(printf '%s\n' "${new_skel_links[@]}" | sort))
    else
        links_to_copy=("${new_skel_links[@]}")
    fi

    files_to_copy=()
    mapfile -t new_skel_files < <( find "$skeldir" -type f -exec md5sum '{}' '+' )
    if [[ -f $cachedir/skel.files && $g_reset = f ]]
    then
        mapfile -t old_skel_files < "$cachedir/skel.files"
        mapfile -t files_to_copy < <(comm -13 <(printf '%s\n' "${old_skel_files[@]}" | sort) <(printf '%s\n' "${new_skel_files[@]}" | sort))
    else
        copy_all=t
        files_to_copy=("${new_skel_files[@]}")
    fi
}

copy_dirs() {
    for dir in "${dirs_to_copy[@]}"
    do
        destdir="$TARGETDIR/${dir#$skeldir/}"
        mkdir -p "$destdir"
    done
    # Update cache after copying directories
    printf '%s\n' "${new_skel_dirs[@]}" > "$cachedir/skel.dirs"
}

copy_links() {
    for link in "${links_to_copy[@]}"
    do
        destlink="$TARGETDIR/${link#$skeldir/}"
        if [[ -e $destlink ]]
        then
            log "$destlink exists: not importing symlink."
        else
            cp --no-dereference "$link" "$destlink"
        fi
    done
    # Update cache after copying links
    printf '%s\n' "${new_skel_links[@]}" > "$cachedir/skel.links"
}

copy_file() {
    local src dest
    src=$1
    dest=$2
    if [[ -e "${dest%.template}" ]] # if dest has .template ending, remove it
    then
        log "Overwriting ${dest%.template}, backing up old file as ${dest%.template}${bkp_sfx}"
        mv "${dest%.template}" "${dest%.template}${bkp_sfx}"
    else
        log "Importing new file ${dest%.template}"
    fi
    if [[ $dest = *.template ]]
    then
        sed "s|%USERHOME%|$HOME|g" "$src" > "${dest%.template}"
    else
        cp "$src" "$dest"
    fi
}

copy_files() {
    for file in "${files_to_copy[@]#* ?}"
    do
        destfile="$TARGETDIR/${file#$skeldir/}"

        cachesum=0
        for j in "${old_skel_files[@]}"
        do
            [[ $j = *\ "$file" ]] && {
                cachesum=${j%% *}
                break
            }
        done

        skelsum=0
        for k in "${new_skel_files[@]}"
        do
            [[ $k = *\ "$file" ]] && {
                skelsum=${k%% *}
                break
            }
        done

        usersum=0
        if [[ -f $destfile ]]
        then
            usersum=$(md5sum "$destfile")
            usersum=${usersum%% *}
        elif [[ -f "${destfile%.template}" ]]
        then # reverse sed to get md5sum of original template file
            usersum=$( sed "s|$HOME|%USERHOME%|g" "${destfile%.template}" | md5sum )
            usersum=${usersum%% *}
        fi

        if [[ $copy_all = t && -f "${destfile%.template}" ]]
        then
            [[ $usersum = "$skelsum" ]] && continue # files are identical
        fi

        if [[ $g_force = t ]]
        then
            copy_file "$file" "$destfile"
            continue
        fi

        if [[ $cachesum = 0 ]] && [[ $usersum = 0 ]]
        then
            log "New file: $file, copying"
            copy_file "$file" "$destfile"
        elif [[ ${cachesum} = "${usersum}" ]]
        then
            log "User file ${destfile%.template} unchanged from default, copying new file."
            copy_file "$file" "$destfile"
        else
            log "User has modified or removed ${destfile%.template}."
            if [[ $g_addonly = t ]]
            then
                log "Skipping all user modified files"
                continue
            fi
            ask_overwrite "${destfile%.template}"
            case $? in
            0)
                copy_file "$file" "$destfile"
                ;;
            1)
                log "Not overwriting ${destfile%.template}"
                ;;
            2)
                copy_file "$file" "$destfile"
                g_force=t
                ;;
            3)
                g_addonly=t
                ;;
            esac
        fi

    done
    # Update cache after copying files
    printf '%s\n' "${new_skel_files[@]}" > "$cachedir/skel.files"
}

main "$@"

This tarball's a bit big because it's got a copy of the Lithium skel in it:
https://drive.google.com/open?id=15fF0D … 5jfAE2BZNo
Unpack, cd into test_user-setup and run './bl-user-setup --install' to create a populated home directory inside. (It won't touch your real $HOME.)
Then try the other options. Check execution times. Change files in skel or home and see what happens. To test the yad popups divert STDOUT to /dev/null: 'bl-user-setup >/dev/null'
Any feedback most welcome!

Last edited by johnraff (2020-01-22 08:05:06)

...elevator in the Brain Hotel, broken down but just as well...
( a boring Japan blog (currently paused), now on Bluesky, there's also some GitStuff )

Introduction to the Bunsenlabs Boron Desktop

Offline

#22 2020-01-22 20:52:21

malm
jgmenu developer
Registered: 2016-10-13
Posts: 735
Website

Re: Best algorithm for bl-user-setup

I get it now big_smile  The --auto option checks if the skel/ files have changed. Nice!

Seems to work well.

Offline

#23 2020-01-23 08:10:40

johnraff
nullglob
From: Nagoya, Japan
Registered: 2015-09-09
Posts: 12,557
Website

Re: Best algorithm for bl-user-setup

A couple of comment corrections and associative arrays to hold md5sums. A bit faster but it doesn't really matter for interactive sessions anyway. Just feels cleaner. (The test directory posted before will still do for testing the algorithm though.)

#!/bin/bash
#
#    bl-user-setup: a script to populate a new user's HOME directory
#    using template files from /usr/share/bunsen/skel/
#    Copyright: 2015-2020 John Crawley <john@bunsenlabs.org>
#               2019      Johan Malm <jgm323@gmail.com>
#
#    This program is free software: you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation, either version 3 of the License, or
#    (at your option) any later version.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program.  If not, see <http://www.gnu.org/licenses/>.

USAGE="bl-user-setup: a script to populate a new user's HOME directory
using files from /usr/share/bunsen/skel

This script is run non-interactively on a user's first login,
and on subsequent logins to check for new default configurations
in /usr/share/bunsen/skel that may have come from a package upgrade.
User set configurations will not be modified without permission.

It can also be run manually with custom options.
The most common manual use case is to run with no options,
in which case the algorithm described below is followed.

In all cases backups will be made of any files which are overwritten.

Options (only one is accepted):
-h --help
    Show this message.
--refresh (default)
    '--refresh' is kept only for backward compatibility.
    If any files in /usr/share/bunsen/skel have changed,
    follow the algorithm below to determine whether to update them
    and whether to prompt the user first.
--addonly --no
    Assume all prompts answered 'no'.
--force --yes
    Assume all prompts answered 'yes'.
--reset
    Implies --force.
    Restore all files to the default state in /usr/share/bunsen/skel.

The following two options are not usually of interest to users:
--auto
    This is run on each user login.
    Do a quick check for any change to skel contents,
    exit if there is nothing new,
    or apply the same algorithm as --refresh if there are changes.
--install
    Implies --force.
    Run non-interactive file import only on first login.

ALGORITHM

Compare the contents of /usr/share/bunsen/skel (here called 'skel')
with the contents on the last run of this script.
If there has been no change, exit immediately.
If there is new content:
Copy in any new directories.
Copy in any new symlinks, if they do not already exist (no overwrites).
Copy in any new files.
If file in skel has changed, and user's file is still in default state,
silently update file.
If skel file and user file have both changed (or user has removed file),
 ask user for instructions.
If a file has been removed from skel, do not remove it from HOME.
"

### RESTORE THESE SETTINGS BEFORE RELEASE ###

readonly USER=${USER}

#readonly TARGETDIR="${HOME}"
readonly TARGETDIR="./home"
#readonly skeldir="/usr/share/bunsen/skel"
readonly skeldir="./skel"
#readonly confdir="$HOME/.config/bunsen"
readonly confdir="./config"
#readonly cachedir="$HOME/.cache/bunsen/bl-setup"
readonly cachedir="./cache"

readonly flagfile="$confdir/bl-setup"
readonly logfile="$cachedir/log"

########################################

main() {

    g_addonly=f
    g_force=f
    g_reset=f
    wholeskelsum=

    pre_checks

    case "$1" in
    --install)
        [[ -f "$flagfile" ]] && exit 0
        g_force=t
        ;;
    --auto)
        check_skel_sum
        ;;
    ''|--refresh)
        :
        ;;
    --addonly)
        g_addonly=t
        ;;
    --force)
        g_force=t
        ;;
    --reset)
        g_reset=t
        g_force=t
        ;;
    -h|--help)
        printf '%b\n' "$USAGE"
        exit 0
        ;;
    *)
        die "$1 is not a valid option"
        ;;
    esac

    mkdir -p "$TARGETDIR" "$confdir" "$cachedir"

    bkp_sfx="~$( date +%FT%T )~"

    log "############
Running $0 at $(date)"

    check_skel_contents

    copy_dirs
    copy_links
    copy_files

    echo "Do not delete this file.

If it is not present, the import of config files from $skeldir
will be repeated, overwriting any user modifications.
(However, backups will be made of overwritten files.)" > "$flagfile"

    # Update cache of skel checksum if script finishes with no errors
    [[ -z $wholeskelsum ]] && {
        wholeskelsum=$( tar -cf - -C "${skeldir%/*}" "${skeldir##*/}" | md5sum )
        wholeskelsum=${wholeskelsum%% *}
    }
    echo "$wholeskelsum" > "$cachedir/skelsum"
}

##############

log () {
    echo "$@" >> "$logfile"
    [[ -t 1 ]] && {
        printf '\033[32m=>\033[m %s\n' "$@"
    }
}

die  () {
    log "ERROR: $*"
    printf '%b\n' "$0: ERROR: $*" >&2
    exit 1
}

ask_yesno() {
    local message="$1"
    if [[ -t 0 && -t 1 ]] # on terminal
    then
        echo "$message"
        while true
        do
            echo "y: Yes
n: No
(Default is Yes.)"
            read -r -p 'Y/n: '
            case "${REPLY^}" in
            Y|'') return 0;;
            N) return 1;;
            esac
        done
    else
        title="BunsenLabs User Setup"
        winicon="distributor-logo-bunsenlabs"
        msg="<b>User Setup</b>

$message"
        yad --center --undecorated --borders=20 --window-icon="$winicon"\
        --title="$title" --text="$msg"\
        --button='Yes':0 --button='No':1
    fi
}

ask_overwrite() {
    local file="$1"
    if [[ -t 0 && -t 1 ]] # on terminal
    then
        echo "You have modified or removed this file:
$file
Overwrite with the new version?"
        while true
        do
            echo "y: overwrite
n: skip this one
a: overwrite all
s: skip all
(Default is skip.)"
            read -r -p 'y/N/a/s: '
            case "${REPLY^}" in
            Y) return 0;;
            N|'') return 1;;
            A) return 2;;
            S) return 3;;
            esac
        done
    else
        title="BunsenLabs User Setup"
        winicon="distributor-logo-bunsenlabs"
        msg="<b>User Setup</b>

You have modified or removed this file:
$file
Overwrite with the new version, or skip?"
        yad --center --undecorated --borders=20 --window-icon="$winicon"\
        --title="$title" --text="$msg"\
        --button='Overwrite':0 --button='Skip':1 --button='Do All':2 --button='Skip All':3
    fi
}

##############

pre_checks () {
    # Do not apply BL configs to the root account.
    [[ "$USER" = root ]] && die "USER is root, abort"

    # Check for the user home directory
    [[ -d "$HOME" ]] || die "User home directory <$HOME> is not set or not a directory."

    # Check for source (skel) directory
    [[ -d "$skeldir" ]] || die "Source directory (skel) is not set or not a directory."
}

check_skel_sum() {
    if [[ -f $cachedir/skelsum ]]
    then
        wholeskelsum=$( tar -cf - -C "${skeldir%/*}" "${skeldir##*/}" | md5sum )
        wholeskelsum=${wholeskelsum%% *}
        # If skel directory unchanged since last run, exit.
        [[ $wholeskelsum = $( <"$cachedir/skelsum" ) ]] && exit 0
        ask_yesno "Some default configuration files have been updated
in a recent package upgrade.
Would you like to import these changes into your HOME directory?
(Files you have edited will not be overwritten without permission.)"
        [[ $? = 1 ]] && {
            log "User declined a config file update"
            echo "$wholeskelsum" > "$cachedir/skelsum" # so user doesn't get repeated prompts
            exit 0
        }
    fi
}

# Directories and links will only be added to the "to_copy" lists
# if they are new arrivals in skeldir.
#
# Files will also be added if the md5sum has changed.

check_skel_contents() {
    dirs_to_copy=()
    mapfile -t new_skel_dirs < <( find "$skeldir" -type d )
    if [[ -f $cachedir/skel.dirs && $g_reset = f ]]
    then
        mapfile -t old_skel_dirs < "$cachedir/skel.dirs"
        mapfile -t dirs_to_copy < <(comm -13 <(printf '%s\n' "${old_skel_dirs[@]}" | sort) <(printf '%s\n' "${new_skel_dirs[@]}" | sort))
    else
        dirs_to_copy=("${new_skel_dirs[@]}")
    fi

    links_to_copy=()
    mapfile -t new_skel_links < <( find "$skeldir" -type l )
    if [[ -f $cachedir/skel.links && $g_reset = f ]]
    then
        mapfile -t old_skel_links < "$cachedir/skel.links"
        mapfile -t links_to_copy < <(comm -13 <(printf '%s\n' "${old_skel_links[@]}" | sort) <(printf '%s\n' "${new_skel_links[@]}" | sort))
    else
        links_to_copy=("${new_skel_links[@]}")
    fi

    files_to_copy=()
    declare -Ag new_skel_files old_skel_files
    while read -r sum file # assuming no files in skel have linebreaks in names
    do
        new_skel_files[$file]="$sum"
    done < <( find "$skeldir" -type f -exec md5sum '{}' '+' )
    if [[ -f $cachedir/skel.files && $g_reset = f ]]
    then
        while read -r sum file
        do
            old_skel_files[$file]="$sum"
        done < "$cachedir/skel.files"
        for i in "${!new_skel_files[@]}"
        do
            [[ "${new_skel_files[$i]}" = "${old_skel_files[$i]}" ]] && continue
            files_to_copy+=("$i")
        done
    else
        files_to_copy=("${!new_skel_files[@]}")
    fi
}

copy_dirs() {
    for dir in "${dirs_to_copy[@]}"
    do
        destdir="$TARGETDIR/${dir#$skeldir/}"
        mkdir -p "$destdir"
    done
    # Update cache after copying directories
    printf '%s\n' "${new_skel_dirs[@]}" > "$cachedir/skel.dirs"
}

copy_links() {
    for link in "${links_to_copy[@]}"
    do
        destlink="$TARGETDIR/${link#$skeldir/}"
        if [[ -e $destlink ]]
        then
            log "$destlink exists: not importing symlink."
        else
            cp --no-dereference "$link" "$destlink"
        fi
    done
    # Update cache after copying links
    printf '%s\n' "${new_skel_links[@]}" > "$cachedir/skel.links"
}

copy_file() {
    local src dest
    src=$1
    dest=$2
    if [[ -e "${dest%.template}" ]] # if dest has .template ending, remove it
    then
        log "Overwriting ${dest%.template}, backing up old file as ${dest%.template}${bkp_sfx}"
        mv "${dest%.template}" "${dest%.template}${bkp_sfx}"
    else
        log "Importing new file ${dest%.template}"
    fi
    if [[ $dest = *.template ]]
    then
        sed "s|%USERHOME%|$HOME|g" "$src" > "${dest%.template}"
    else
        cp "$src" "$dest"
    fi
}

copy_files() {
    for file in "${files_to_copy[@]}"
    do
        destfile="$TARGETDIR/${file#$skeldir/}"

        cachesum=${old_skel_files[$file]:-0}
        skelsum=${new_skel_files[$file]:-0}
        [[ $skelsum = 0 ]] && die "$file not found in $skeldir"

        usersum=0
        if [[ -f $destfile ]]
        then
            usersum=$(md5sum "$destfile")
            usersum=${usersum%% *}
        elif [[ -f "${destfile%.template}" ]]
        then # reverse sed to get md5sum of original template file
            usersum=$( sed "s|$HOME|%USERHOME%|g" "${destfile%.template}" | md5sum )
            usersum=${usersum%% *}
        fi
        [[ $usersum = "$skelsum" ]] && continue # files are identical

        if [[ $g_force = t ]]
        then
            copy_file "$file" "$destfile"
            continue
        fi

        if [[ $cachesum = 0 ]] && [[ $usersum = 0 ]]
        then
            log "New file: $file, copying"
            copy_file "$file" "$destfile"
        elif [[ ${cachesum} = "${usersum}" ]]
        then
            log "User file ${destfile%.template} unchanged from default, copying new file."
            copy_file "$file" "$destfile"
        else
            log "User has modified or removed ${destfile%.template}."
            if [[ $g_addonly = t ]]
            then
                log "Skipping all user modified files"
                continue
            fi
            ask_overwrite "${destfile%.template}"
            case $? in
            0)
                copy_file "$file" "$destfile"
                ;;
            1)
                log "Not overwriting ${destfile%.template}"
                ;;
            2)
                copy_file "$file" "$destfile"
                g_force=t
                ;;
            3)
                g_addonly=t
                ;;
            esac
        fi

    done
    # Update cache after copying files
    for i in "${!new_skel_files[@]}"
    do
        printf '%s  %s\n' "${new_skel_files[$i]}" "$i"
    done > "$cachedir/skel.files"
}

main "$@"

Time to put this in a github branch to keep track of tweaks.
https://github.com/BunsenLabs/bunsen-co … user-setup

...elevator in the Brain Hotel, broken down but just as well...
( a boring Japan blog (currently paused), now on Bluesky, there's also some GitStuff )

Introduction to the Bunsenlabs Boron Desktop

Offline

#24 2020-01-25 03:05:13

johnraff
nullglob
From: Nagoya, Japan
Registered: 2015-09-09
Posts: 12,557
Website

Re: Best algorithm for bl-user-setup

I think it's usable now, but needs testing of the corner cases and some user feedback on whether they appreciate it or not (hope they do).
So, soon I'll put it in bunsen-configs and release it to the exp repo so our testers will be exposed to it...  yikes

There is more that can be done though - @malm I liked your idea of showing a file diff before making decisions. Maybe best at the point when a prompt comes up about a specific file?

But maybe that could go on a DO LATER list?

...elevator in the Brain Hotel, broken down but just as well...
( a boring Japan blog (currently paused), now on Bluesky, there's also some GitStuff )

Introduction to the Bunsenlabs Boron Desktop

Offline

#25 2020-01-25 10:09:12

malm
jgmenu developer
Registered: 2016-10-13
Posts: 735
Website

Re: Best algorithm for bl-user-setup

Yes, showing the diff would be quite powerful.
A bit like

git add -p

Offline

#26 2020-02-27 01:27:56

johnraff
nullglob
From: Nagoya, Japan
Registered: 2015-09-09
Posts: 12,557
Website

Re: Best algorithm for bl-user-setup

TODO
User feedback: https://forums.bunsenlabs.org/viewtopic.php?id=6510

Before that came up I was thinking about these two cases when there was no user filelist cache:
1) Upgrade from Helium.
2) User has deleted their cache deliberately (or accidentally) and wants to rebuild it.
In both these cases there is an existing $HOME with config files the user might have tweaked, so getting a message and rebuilding the cache with one-at-a-time checks of each file might be reasonable enough.

BUT 3)  with a brand new install this is not the case - the files in HOME and skel are identical, and all that is needed is to build the cache. So now I want to think about some test for that situation...

...elevator in the Brain Hotel, broken down but just as well...
( a boring Japan blog (currently paused), now on Bluesky, there's also some GitStuff )

Introduction to the Bunsenlabs Boron Desktop

Offline

#27 2020-02-27 18:31:36

malm
jgmenu developer
Registered: 2016-10-13
Posts: 735
Website

Re: Best algorithm for bl-user-setup

^ I struggle to tune into the problem. Isn’t it bl-user-setup that copies the files from skel to $HOME in the first place?

Offline

#28 2020-02-27 22:19:11

damo
....moderator....
Registered: 2015-08-20
Posts: 6,734

Re: Best algorithm for bl-user-setup

malm wrote:

^ I struggle to tune into the problem. Isn’t it bl-user-setup that copies the files from skel to $HOME in the first place?

bl-user-setup --auto
    This is run on each user login.
    Do a quick check for any change to skel contents,
    exit if there is nothing new,
    or apply the same algorithm as --refresh if there are changes.

Be Excellent to Each Other...
The Bunsenlabs Lithium Desktop » Here
FORUM RULES and posting guidelines «» Help page for forum post formatting
Artwork on DeviantArt  «» BunsenLabs on DeviantArt

Offline

#29 2020-02-27 23:24:19

johnraff
nullglob
From: Nagoya, Japan
Registered: 2015-09-09
Posts: 12,557
Website

Re: Best algorithm for bl-user-setup

@Johan you're absolutely right of course.
@damo I think the help text might need tweaking slightly - refresh is now just a dummy option, same effect as no options, ie default.

In cases 1) & 2) popup confirmation messages are inevitable. I guess once the new bunsen-configs is in the iso then popups on new installs will disappear.

...elevator in the Brain Hotel, broken down but just as well...
( a boring Japan blog (currently paused), now on Bluesky, there's also some GitStuff )

Introduction to the Bunsenlabs Boron Desktop

Offline

#30 2021-08-09 09:19:04

johnraff
nullglob
From: Nagoya, Japan
Registered: 2015-09-09
Posts: 12,557
Website

Re: Best algorithm for bl-user-setup

Update - a diff option has been pushed to github and will appear in the next version of bunsen-configs.
Some other improvements too.

...elevator in the Brain Hotel, broken down but just as well...
( a boring Japan blog (currently paused), now on Bluesky, there's also some GitStuff )

Introduction to the Bunsenlabs Boron Desktop

Offline

Pages: 1 2

  1. Index
  2. » Dev Discussions
  3. » Best algorithm for bl-user-setup

Board footer

Powered by FluxBB