/r/bash
Wake me up when September ends.
A subreddit dedicated to Bash scripting. Now complete with a Discord Server.
Content must be Bash related. This rule is interpreted generously; general shell scripting content is mostly accepted. However, the post should not be specific to another shell.
No reposts. This is meant with regards to content, not just “the same link was submitted earlier” – it’s okay to resubmit an old link in some new context (e. g. because you’d like to discuss another part of it, or because something has changed since the last time it was submitted, or because the link was updated since then). Links from the sidebar count as having been submitted already, so posting them without new context is also considered a repost.
You can choose one of these four flairs for your post:
If you don’t flair your post, the moderators will set the most appropriate flair.
/r/unix – for everything Unix
Other Shells: /r/zsh, /r/fishshell, /r/oilshell, /r/batch
BashGuide – A Bash guide for beginners.
Beginner's Guide to Command Line – A crash course for some common unix and shell commands. Update 2022-01-14: Course is currently being rewritten
Google's Shell Style Guide – Reasonable advice about code style.
Explainshell - Explain complex shell operations.
ShellCheck – Automatically detects problems with shell scripts.
BashFAQ – Answers most of your questions.
BashPitfalls – Lists the common pitfalls beginners fall into, and how to avoid them.
(Archived) The Bash-Hackers Wiki – Extensive resource.
#bash – IRC channel on Libera. The main contributors of the BashGuide, BashFAQ, BashPitfalls and ShellCheck hang around there.
/r/bash
Curious if there's any way to hook into the error condition 'command not found' and run a script/function? Basically, I'd like to do something similar to "thefuck" but have it run automatically.
$ doesnotexist
-bash: doesnotexist: command not found
# how to (automatically) call some custom function/script/etc?
# preferably with access to bash history so I can run a
# fuzzy find with target command vs my defined aliases
So far my searches keep coming up with irrelevant stuff so I'm not sure if I'm just using bad search terms or if this is something that is just not possible under bash.
Hello,
i have a lot of folders containing files and more sobfolders with files. I want to have all that files in the root folder and the filename should contain the folder name. For example the file /testdir1/testdir2/testfile,txt should be in /testdir1_-_testdir2_-_testfile.txt
The thing is, some years ago i had done this by accident (i think i tried just to remove bad characters from filename but by accident also replaces the / but i can't get it together again :-( )
while read -r line
do
echo "$line"
done <file.txt
Here, the condition read -r line
has nothing to read the first time the loop runs, why it doesn't break the first time?
Hi, I have been learning Bash the last two days as my first scripting language. I saw the advent of code started this year, and I thought why not try to solve it with Bash (since it's the only language I know so far." I managed to solve most of it by myself, had only to look for the sort command.
A text file with the 2 lists is presented in the following format
18944 47230
94847 63037
93893 35622
Save the numbers in a text file called input.txt"
#!/bin/bash
# Generate an array from the input
list=(`cat input.txt`)
# Save the even elements into list.left.txt and the odd elements into list.right.txt
for el in "${!list[@]}"
do
rem=$((${el} % 2))
if [[ rem -eq 0 ]]
then
echo "${list[$el]}" >> list.left.txt
else
echo "${list[$el]}" >> list.right.txt
fi
done
# Sorting the numbers
sort list.left.txt > list.left.sorted.txt
sort list.right.txt > list.right.sorted.txt
# create arrays from the two files
left=(`cat list.left.sorted.txt`)
right=(`cat list.right.sorted.txt`)
# calculate the difference and save it to a text file.
for ele in "${!left[@]}"
do
diff=$(("${left[$ele]}"-"${right[$ele]}"))
if [ $diff -ge 0 ]
then
echo "$diff" >> diffs.txt
else
diff=$(($diff * -1))
echo "$diff" >> diffs.txt
fi
done
# Import the differences as an array
di=(`cat diffs.txt`)
total=0
for elem in ${di[@]}
do
total=$(($total + $elem))
done
echo "$total"
Hi,
Newbie here, apologies in advance if my question is not appropriate.
I have a bash script that installs some software, and I would like to generate a networkd-dispatcher script.
The networkd-dispatcher script should contain placeholders such as "$IFACE" and "$UNIT_NAME", but the installation script interprets them as undeclared variables, and the networkd-dispatcher scripts ends up with empty spaces.
How can I escape these "$"?
This is what I have at the moment in the installation script:
create_networkd_script() {
cat << EOF > $HOME/BirdNET-Pi/templates/50-birdweather-publication
#!/bin/bash
UNIT_NAME="birdweather_publication@$IFACE.service"
# Check if the service is active and then start it
if systemctl is-active --quiet "$UNIT_NAME"; then
echo "$UNIT_NAME is already running."
else
echo "Starting $UNIT_NAME..."
systemctl start "$UNIT_NAME"
fi
EOF
chmod +x $HOME/BirdNET-Pi/templates/50-birdweather-publication
chown root:root $HOME/BirdNET-Pi/templates/50-birdweather-publication
ln -sf $HOME/BirdNET-Pi/templates/50-birdweather-publication /etc/networkd-dispatcher/routable.d
systemctl enable systemd-networkd
}
create_networkd_script
I want to use ctrl+c like I use in my editor to enter normal mode
I am trying to have an understanding of what these things actually mean and have an understanding of it.
The more I read the more confused I get, if someone could explain it so a child could understand it I would appreciate it.
Hello, I'm confused about the output of this script:
Foo="bar"
cat << EOF
a $Foo
$Foo
EOF
This outputs:
a bar
Foo
It looks like variables at the start of a line don't get substituted. Can I work around that?
A few scripts I wrote have "byte count" as an [optional] input. Id like these to accept using prefixes (e.g., 64 kb or 128 MiB). But, there are 2 competing systems at play here.
Is there some universally agreed upon syntax for which prefic abbreviations map to 1000^n vs which map to 1024^N?
NOTE: for my use cases it doesnt make sense to specify bit count, so wshether or not there is a trailing b
or B
it will always refer to bytes.
My intuition here is that
1000^N:
1024^N:
Are there any commonly used programs that would conflict with this mapping?
As far as the actual implementation, I use something like
getBytes() {
local +i nn
local -A byteMap
byteMap=([k]=1 [m]=2 [g]=3 [t]=4 [p]=5 [e]=6 [z]=7 [y]=8 [r]=9 [q]=10)
for nn in "${@}"; do
nn="${nn//[bB ]/}"
case "${nn}" in
*[kmgtpezyrq])
echo "$(( ${nn//[^[0-9]/} * ( 1000 ** ${byteMap[${nn//[0-9]/}]} ) ))"
;;
*[KMGTPEZYRQIi])
nn="${nn,,}"
nn="${nn%i}"
echo "$(( ${nn//[^[0-9]/} << ( 10 * ${byteMap[${nn//[0-9]/}]} ) ))"
;;
*)
echo "${nn//[^0-9]/}"
;;
esac
done
}
but if anyone has a better implementation please do suggest it!
EDIT: updated function with a slightly more efficient version.
I got a coupon to attempt the certificate exam SC103 from The Linux Foundation. Wondering if anyone has given this exam? How should I prepare specifically for this exam as this would be online proctored exam. I have few months before the voucher expires. Any suggestions would be appreciated.
Hello! I am a relatively new Linux user and I spent the better part of a month working on a project called clicraft. It is available at https://github.com/DontEvenTalkToMe/clicraft ! Please do check it out and give me some feedback as I would like to develop my skills further, thanks!
Hello everyone,
I am a newbie Ble-sh user. I installed it using all default configurations. I think it's a bit slow, and that latency bothers me a lot. I would like to know some good tips to tune the performance. Do you mind sharing them with me?
I appreciate any help you can provide.
PS: I also use Atuin integrated with it. I would greatly appreciate any performance tunes upon it as well.
Need a repo updater and need to implement in your custom bash scripts to make your script up-to-date and monitor for the updates??, here it is called repo-updater
Needs a code update for better use
It was originally created for Android Sysinfo script to check updates here
I'm trying to make the script support the usage described below and am having trouble passing $DIRS
(directory names as arguments) to fzf as a string. Pretty sure converting array to string should be avoided, but what are alternatives? A directory could contain a space.
# Usage: re <pattern> [dirs]
trap 'rm /tmp/.rg-fzf-{f,r} >/dev/null 2>&1' EXIT INT QUIT TERM
INITIAL_QUERY="$1"
shift
DIRS=( "$@" )
RG_PREFIX="rg --column --line-number --no-heading --color=always --smart-case"
fzf --ansi --disabled --query "$INITIAL_QUERY" \
--bind "start:reload:$RG_PREFIX {q} ${DIRS[*]}" \
--bind "change:reload:sleep 0.1; $RG_PREFIX {q} ${DIRS[*]} || true" \
--bind 'ctrl-r:transform:[[ ! $FZF_PROMPT =~ ripgrep ]] &&
echo "rebind(change)+change-prompt(1. ripgrep> )+disable-search+transform-query:echo \{q} > /tmp/.rg-fzf-f; cat /tmp/.rg-fzf-r" ||
echo "unbind(change)+change-prompt(2. fzf> )+enable-search+transform-query:echo \{q} > /tmp/.rg-fzf-r; cat /tmp/.rg-fzf-f"' \
... \
--bind 'enter:become(nvim {1} +{2})'
Basically I'm trying to tweak this fzf command that uses rg
(grep-like alternative) command to support taking the rest of the arguments starting from the second argument as directories to search for, with the first argument being the string to search for.
im working on a bash script that takes two text files, input file contains some text and dictionary.txt contains a list of 4 letter words that exist in the input file. im trying to find all 4 letter words in file and compare then to the words in dictionary.txt, if a word in input does not exist in dictionary, print that four letter word. here is my script:
#!/bin/bash
# Check if the input file and dictionary file are provided
if [ $# -eq 0 ]; then
echo "input file and dictionary missing"
exit 1
fi
# Check if the input file is valid
input_file=$1
if [ ! -f "$input_file" ]; then
echo "$input_file is not a file"
exit 1
fi
# Check if the dictionary file is valid
dictionary_file=$2
if [ ! -f "$dictionary_file" ]; then
echo "$dictionary_file is not a file"
exit 1
fi
# Read the dictionary into an array
mapfile -t dictionary < "$dictionary_file"
# Convert dictionary array to lowercase for case-insensitive comparison
dictionary=("${dictionary[@],,}")
# Check the input file for 4-letter words
grep -o '\b[a-zA-Z]\{4\}\b' "$input_file" | while read word; do
# Convert the word to lowercase
word=$(echo "$word" | tr '[:upper:]' '[:lower:]')
# Check if the word is NOT in the dictionary
if ! [[ " ${dictionary[@]} " =~ " ${word} " ]]; then
echo "$word"
fi
done
#!/bin/bash
# Check if the input file and dictionary file are provided
if [ $# -eq 0 ]; then
echo "input file and dictionary missing"
exit 1
fi
# Check if the input file is valid
input_file=$1
if [ ! -f "$input_file" ]; then
echo "$input_file is not a file"
exit 1
fi
# Check if the dictionary file is valid
dictionary_file=$2
if [ ! -f "$dictionary_file" ]; then
echo "$dictionary_file is not a file"
exit 1
fi
# Read the dictionary into an array
mapfile -t dictionary < "$dictionary_file"
# Convert dictionary array to lowercase for case-insensitive comparison
dictionary=("${dictionary[@],,}")
# Check the input file for 4-letter words
grep -o '\b[a-zA-Z]\{4\}\b' "$input_file" | while read word; do
# Convert the word to lowercase
word=$(echo "$word" | tr '[:upper:]' '[:lower:]')
# Check if the word is NOT in the dictionary
if ! [[ " ${dictionary[@]} " =~ " ${word} " ]]; then
echo "$word"
fi
doneim working on a bash script that takes two text files, input file contains some text and dictionary.txt contains a list of 4 letter words that exist in the input file. im trying to find all 4 letter words in file and compare then to the words in dictionary.txt, if a word in input does not exist in dictionary, print that four letter word. here is my script: #!/bin/bash
# Check if the input file and dictionary file are provided
if [ $# -eq 0 ]; then
echo "input file and dictionary missing"
exit 1
fi
# Check if the input file is valid
input_file=$1
if [ ! -f "$input_file" ]; then
echo "$input_file is not a file"
exit 1
fi
# Check if the dictionary file is valid
dictionary_file=$2
if [ ! -f "$dictionary_file" ]; then
echo "$dictionary_file is not a file"
exit 1
fi
# Read the dictionary into an array
mapfile -t dictionary < "$dictionary_file"
# Convert dictionary array to lowercase for case-insensitive comparison
dictionary=("${dictionary[@],,}")
# Check the input file for 4-letter words
grep -o '\b[a-zA-Z]\{4\}\b' "$input_file" | while read word; do
# Convert the word to lowercase
word=$(echo "$word" | tr '[:upper:]' '[:lower:]')
# Check if the word is NOT in the dictionary
if ! [[ " ${dictionary[@]} " =~ " ${word} " ]]; then
echo "$word"
fi
done
#!/bin/bash
# Check if the input file and dictionary file are provided
if [ $# -eq 0 ]; then
echo "input file and dictionary missing"
exit 1
fi
# Check if the input file is valid
input_file=$1
if [ ! -f "$input_file" ]; then
echo "$input_file is not a file"
exit 1
fi
# Check if the dictionary file is valid
dictionary_file=$2
if [ ! -f "$dictionary_file" ]; then
echo "$dictionary_file is not a file"
exit 1
fi
# Read the dictionary into an array
mapfile -t dictionary < "$dictionary_file"
# Convert dictionary array to lowercase for case-insensitive comparison
dictionary=("${dictionary[@],,}")
# Check the input file for 4-letter words
grep -o '\b[a-zA-Z]\{4\}\b' "$input_file" | while read word; do
# Convert the word to lowercase
word=$(echo "$word" | tr '[:upper:]' '[:lower:]')
# Check if the word is NOT in the dictionary
if ! [[ " ${dictionary[@]} " =~ " ${word} " ]]; then
echo "$word"
fi
done
This is a problem I run into frequently, but I'll describe the current application.
So, I have a list of subtitle files for all the episodes of a program called "Forged in Fire". I'm trying to review each file that contains something about "meeting parameters" to compile a list of the episodes where there has been a "parameter failure". I thought it would be as simple as...
egrep -o "./Forged.in.Fire.S.*E.*_extracted_sub*" ./matching_episodes | uniq | sort | while read file ; do less -FX "$file" ; reset ; read -p "Did that episode have a parameter failure?: yes_no" ; if [ "$yes_no" = "yes" ] ; then echo "$file" >> ./episodes_with_parameter_failures ; fi ; done
However it turns out that between piping information into "while", the way "less" blocks and how "read" blocks for input, this isn't working. All that happens is 'less' runs, and when I exit, the next instance of 'less' runs immediately instead of my prompt. I've tried a whole host of things like trying to run 'clear', or 'reset', or other more direct tty options to no avail.
I'm not really sure how to change my approach to this because it seems like it's just simply not feasible due to the way 'while' is creating a subshell thanks to the standard-input redirection, and then with 'less' and 'read' both blocking for input. But I'm not sure what other tools in bash I might be able to use.
I need to be able to
I'm wondering if I could somehow used 'xargs' to avoid piped input, but I still think there's an underlying issue of competing blocking going on between "less" and "read" that won't resolve? Perhaps not, because as a workaround I did this...
echo '#!/bin/bash' > ./script.sh ; egrep -o "./Forged.in.Fire.S.*E.*_extracted_sub*" ./matching_episodes | uniq | sort | while read file ; do echo -ne "less "$file"\n./review.sh "$file"\n"; done >> ./script.sh
That allows me to run 'script.sh' afterwards, and works as I want, but I would really like to understand this to not have to rely on such a hacky workaround for next time I encounter something like this, because there are many occasions where I would like to run a loop that presents me the contents of something in a pager program, and then be prompted about what to do about it. But the current ways I know how to skin this cat really suck.
So long story short, I really want to be able to do something like this...
*produce list of files* | while read file ; do less "$file" ; read -p "Question about file" user_input ; if *expression evaluating $user_input* ; then *run some code* ; fi ; done
As a quick one-liner and have it actually work.
Is there ever a good reason to use exit 1
in a function (title is wrong)? You should always use return 1
and let the caller handle what to do after? The latter is more transparent, e.g. you can't assume exit 1
from a function always exits the script if the function is run inside a subshell like command substitution? Or is exit 1
in a function still fine and the maintainer of the script should be mindful of this, e.g. depending on whether it's run in a subshell in which case it won't exit the script?
I have an abort
function:
abort() {
printf "%b\n" "${R}Abort:${E} $*" >&2
exit 1
}
which I intended to use to print message and exit the script when it's called.
But I have a function running in a command substition that uses this abort
function so I can't rely on it to exit the script.
Instead, change exit 1
to return 1
and var=$(func) || exit $?
? Or can anyone recommend better practices? It would be neater if the abort function can handle killing the script (with signals?) instead of handling at every time abort
gets called but not sure if this introduces more caveats or is more prone to error.
I guess I shouldn't take "exit" to typically mean exit the script? I believe I also see typical abort
/die
with exit 1
instead of return 1
, so I suppose the maintainer of the script should simply be conscious of calling it in a subshell and handling that specific case.
Hello,
I faced a real-life challenge by trying to run a Unix binary installed on another partition of my SSD. The execution failed with the "Segmentation error" message which usually points to an incompatibility. Switching to the partition with a newer macOS that hosts the binary allows me to run it as intended.
I suspect it's because of the paths to dependencies hardcoded in the binary. My question is, is it possible to make it use these paths even if I'm currently working from the other partition?
I'm running my scripts on ubuntu.
I've tried to read an array using read command and it's as follows:
read -a arr
which is working when I execute it as a standalone command and not working when I'm trying it use it in a shell script file.
Source code:
read -p "Enter array elements: " -a arr
largest=${arr[0]}
for ele in ${arr[@]}; do
if [ $ele -gt $largest ]; then
largest=$ele
fi
done
echo "Largest is $largest"
I already understand how mostly everything works in bash, however, I am looking for a course to learn how to more effectively format scripts. My scripts are so messy and hard to read. Any ideas?
curl -sSL https://install.python-poetry.org | python3 - --version 1.6.0
printf "\npyproject.toml\ncz_conventional_commits\npoetry: Get and set version from pyproject.toml:tool.poetry.version field\nsemver\nv$major.$minor.$patch$prerelease\nY\nY\ncommit-msg" | /root/.local/bin/poetry run cz init
Welcome to commitizen!
Answer the questions to configure your project.
For further configuration visit:
https://commitizen-tools.github.io/commitizen/config/
Warning: Input is not a terminal (fd=0).
? Please choose a supported config file: pyproject.toml
? Please choose a cz (commit rule): (default: cz_conventional_commits) cz_customize
? Choose the source of the version: poetry: Get and set version from pyproject.toml:tool.poetry.version field
No Existing Tag. Set tag to v0.0.1
? Choose version scheme: semver
? Please enter the correct version format: (default: "$version") semver
? Create changelog automatically on bump Yes
? Keep major version zero (0.x) during breaking changes Yes
? What types of pre-commit hook you want to install? (Leave blank if you don't want to install) done
You can bump the version running:
cz bump
Configuration complete 🚀
poetry run cz init <<EOF
pyproject.toml
cz_conventional_commits
poetry: Get and set version from pyproject.toml:tool.poetry.version field
semver
v\$major.\$minor.\$patch\$prerelease
Y
Y
commmit-msg
EOF
Welcome to commitizen!
Answer the questions to configure your project.
For further configuration visit:
https://commitizen-tools.github.io/commitizen/config/
Warning: Input is not a terminal (fd=0).
? Please choose a supported config file: .cz.toml
? Please choose a cz (commit rule): (default: cz_conventional_commits) cz_conventional_commits
? Choose the source of the version: scm: Fetch the version from git and does not need to set it back
No Existing Tag. Set tag to v0.0.1
? Choose version scheme: pep440
? Please enter the correct version format: (default: "$version") v$major.$minor.$patch$prerelease
? Create changelog automatically on bump Yes
? Keep major version zero (0.x) during breaking changes Yes
? What types of pre-commit hook you want to install? (Leave blank if you don't want to install) done
You can bump the version running:
cz bump
Configuration complete 🚀
Welcome to commitizen!
Answer the questions to configure your project.
For further configuration visit:
https://commitizen-tools.github.io/commitizen/config/
? Please choose a supported config file: pyproject.toml
? Please choose a cz (commit rule): (default: cz_conventional_commits) cz_conventional_commits
? Choose the source of the version: poetry: Get and set version from pyproject.toml:tool.poetry.version field
No Existing Tag. Set tag to v0.0.1
? Choose version scheme: semver
? Please enter the correct version format: (default: "$version") v$major.$minor.$patch$prerelease
? Create changelog automatically on bump Yes
? Keep major version zero (0.x) during breaking changes Yes
? What types of pre-commit hook you want to install? (Leave blank if you don't want to install) [commit-msg]
commitizen pre-commit hook is now installed in your '.git'
You can bump the version running:
cz bump
Configuration complete 🚀
commitizen version: 3.30.0 python version: 3.10.11 docker version: Docker version 27.2.0, build 3ab4256 cz init is running inside a docker container very specifically the python 3.10.11 container
I was recently tasked with creating some resources for students new to computational research, and part of that included some material on writing bash scripts to automate various parts of their computational workflow. On the one hand: this is a little bit of re-inventing the wheel, as there are many excellent resources already out there. At the same time, it's sometimes helpful to have guides that are somewhat limited in scope and focus on the most common patterns that you'll encounter in a particular domain.
With that in mind, I tried to write some tutorial material targeted at people who, in the context of their research, are just realizing they want to do something better than babysit their computer as they re-run the same code over and over with different command line options. Most of the Bash-related information is on this "From the command line to simple bash scripts" page, and I also discuss a few scripting strategies (running jobs in parallel, etc) on this page on workload and workflow management.
I thought I would post this here in case folks outside of my research program find it helpful. I also know that I am far from the most knowledgeable person to do this, and I'd be more than happy to get feedback (on the way the tutorial is written, or on better/more robust ways to do script things up) from the experts here!
I use Debian as my daily driver (I've been using Linux in some for or the other as my daily driver since 2006). I also us Debian for all my servers. I also running a webhosting business and in my 20+ years in the game, I have learned that...
On my daily driver machine, I also create a lot of digital artwork. I recently found myself rendering a finished artfile that was 9.5GB (24000px by 13700px) in Gimp, and half way through I started running out of RAM and SWAP. So I quickly created the needed SWAP file, and Gimp was able to finish the render without crashing.
So I created a script to automate the process, along with two supporting script.
I am wanting some guys to test it out, maybe in a VM if you don't want to risk a production or daily driver machine.
I am running it currently on my laptop. It has 12GB RAM and 4GB SWAP. I opened several of my art pieces 12000px wide to 24000px wide, several times into various programs. I watched my desktop widget show the RAM getting used up, and then the SWAP, then suddenly, more SWAP. When I closed all the images from all the programs, I watched all the extra SWAP space disappear?
Looking for constructive criticism and feedback. The Git Repo is https://git.zaks.web.za/thisiszeev/linux-server-tools and it's all in the folder swap-management.
Thanks in advanced, as I am wanting to rework it in to a solution that can be installed via a package manager, to simplify life for others...
Hi, I am using dirdiff, grsync but dirdiff show the same files like differents when they are the same. grsync will copy over the same file in destiny.
I will do the backup manually so,
I need a tool for compare 2 dirs side by side...
I have pending to see yours complete replies to my last post here.
Thank you and Regards!
Hello is have script, it works when I run it manually. Problem is when I want to run it with cron, backup is not created. From log seems script stuck on password. Any help appreciated
#!/usr/bin/expect -f
log_file /tmp/debug.log
spawn echo "cron started"
spawn rm /home/admin/backup-restore/mls_backup/mls-backup.tar.gz
set password {password}
spawn /usr/sbin/exec /home/admin/backup-restore/backup-restore --target /home/admin/backup-restore/mls_backup/mls-backup.tar.gz --no-encryption
expect "admin password:"
send "$password\r"
interact
Edited: I did a mistake: hi, doing ls I have some files named "name'", why do not I can rm them?
when I tipe rm name nothing pass. rm nam<tab> nothing pass...
these names have " '" note ' before last "
Thank you and Regards!
Thank you every of you repliers for your help