Derrick.Blarg

Tag: automation

  • Garbage Sysadmin: Easily Make CIFS Mounts

    Garbage Sysadmin: Easily Make CIFS Mounts

    I’ve been rebuilding my Raspberry Pi collection from scratch, and moving from Ubuntu Server to Debian/Raspbian Bookworm. One of the tasks that I quickly tried to automate was reconnecting my CIFS mounts. I wanted to do it better, and came across this method, with the help of ChatGPT, to mount them at boot:

    #!/bin/bash

# Check if the script is run as root
if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root"
   exit 1
fi

# Check for correct number of arguments
if [ "$#" -ne 4 ]; then
    echo "Usage: $0 <RemoteDirectory> <MountDirectory> <Username> <Password>"
    exit 1
fi

REMOTE_DIR="$1"
MOUNT_DIR="$2"
USERNAME="$3"
PASSWORD="$4"
CREDENTIALS_PATH="/etc/samba/credentials-$(basename "$MOUNT_DIR")"

# Escape the mount directory for systemd
UNIT_NAME=$(systemd-escape -p --suffix=mount "$MOUNT_DIR")

# Create mount directory
mkdir -p "$MOUNT_DIR"

# Create credentials file
touch "$CREDENTIALS_PATH"
echo "username=$USERNAME" > "$CREDENTIALS_PATH"
echo "password=$PASSWORD" >> "$CREDENTIALS_PATH"
chmod 600 "$CREDENTIALS_PATH"

# Create systemd unit file
UNIT_FILE_PATH="/etc/systemd/system/$UNIT_NAME"
echo "[Unit]
Description=Mount Share at $MOUNT_DIR
After=network-online.target
Wants=network-online.target

[Mount]
What=$REMOTE_DIR
Where=$MOUNT_DIR
Type=cifs
Options=_netdev,iocharset=utf8,file_mode=0777,dir_mode=0777,credentials=$CREDENTIALS_PATH
TimeoutSec=30

[Install]
WantedBy=multi-user.target" > "$UNIT_FILE_PATH"

# Reload systemd, enable and start the unit
systemctl daemon-reload
systemctl enable "$UNIT_NAME"
systemctl start "$UNIT_NAME"

echo "Mount setup complete. Mounted $REMOTE_DIR at $MOUNT_DIR"Code language: Bash (bash)

    I’m sure this is totally insecure and a terrible idea, but it works for me so back off, buddy!

    Please don’t follow me as an example of what to do, but take this code for anything you need.

  • Bash Script: Calculate before/after 2: Calculate Harder

    Bash Script: Calculate before/after 2: Calculate Harder

    As an update, or an evolution of my earlier script that did some simple math for me, I’ve made one that will full-on test a URL while I’m making changes to see what the impact performance is of my updates.

    $ abtesturl.sh --url=https://example.com/ --count=10
Press any key to run initial tests...
Initial average TTFB: 3.538 seconds
Press any key to re-run tests...

Running second test...
Second average TTFB: 1.975 seconds
Before TTFB: 3.538 seconds
After TTFB: 1.975 seconds
Change in TTFB: -1.563 seconds
Percentage Change: -44.00%Code language: JavaScript (javascript)

    It makes it much simpler to gather data to write reports or figure out of a change is worth the effort.

    Well, that’s about it so here’s the script:

    #!/bin/bash

function show_usage() {
	echo "Usage: $0 --url=<URL> [--count=<number of requests>]"
	echo "  --url        Specifies the URL to test."
	echo "  --count      Optional. Specifies the number of requests to send. Default is 6."
	echo
	echo "Example: $0 --url=https://example.com/ --count=5"
	exit
}

function average_ttfb() {
	local URL=""
	local COUNT=6 # Default COUNT to 6 if not supplied
	local CURL_OPTS="-s"

	# Parse arguments
	for arg in "$@"; do
		case $arg in
			--url=*)
			URL="${arg#*=}"
			shift # Remove argument from processing
			;;
		--count=*)
			COUNT="${arg#*=}"
			shift # Remove argument from processing
			;;
		*)
			# Unknown option
			;;
		esac
	done

	if [[ -z "$URL" ]]; then
		exit 1
	fi

	local total_time=0
	local count_success=0

	for ((i=1; i<=COUNT; i++))
	do
		# Perform the curl command, extracting the time to first byte
		ttfb=$(curl $CURL_OPTS -o /dev/null -w "%{time_starttransfer}\n" $URL)
		
		# Check if the curl command was successful
		if [ $? -eq 0 ]; then
			total_time=$(echo "$total_time + $ttfb" | bc)
			((count_success++))
		else
			echo "Request $i failed." >&2
		fi
	done

	if [ $count_success -eq 0 ]; then
		echo "All requests failed." >&2
		return 1
	fi

	# Calculate the average TTFB
	average_time=$(echo "scale=3; $total_time / $count_success" | bc)
	echo $average_time # This line now simply outputs the average time
}

function ab_test_ttfb() {
	# Run initial test
	read -p "Press any key to run initial tests..." -n 1 -r
	initial_ttfb=$(set -e; average_ttfb "$@"; set +e)
	echo "Initial average TTFB: $initial_ttfb seconds"
	
	# Wait for user input to proceed
	read -p "Press any key to re-run tests..." -n 1 -r
	echo # Move to a new line

	# Run second test
	echo "Running second test..."
	second_ttfb=$(average_ttfb "$@")
	echo "Second average TTFB: $second_ttfb seconds"

	# Calculate and output the difference and percentage change
	difference=$(echo "$second_ttfb - $initial_ttfb" | bc)
	percent_change=$(echo "scale=2; ($difference / $initial_ttfb) * 100" | bc)

	echo "Before TTFB: $initial_ttfb seconds"
	echo "After TTFB: $second_ttfb seconds"
	echo "Change in TTFB: $difference seconds"
	echo "Percentage Change: $percent_change%"
}

# Check if help is requested or no arguments are provided
if [[ " $* " == *" --help "* ]] || [[ "$#" -eq 0 ]]; then
	show_usage
fi

# Check if --url is in the arguments
url_present=false
for arg in "$@"; do
	if [[ $arg == --url=* ]]; then
		url_present=true
		break
	fi
done

if [ "$url_present" = false ]; then
	echo "Error: --url argument is required."
	show_usage
fi

ab_test_ttfb "$@"Code language: Bash (bash)

    Don’t break anything!

  • Bash Script: Calculate before/after averages

    Bash Script: Calculate before/after averages

    I’ve been doing some performance testing, and wanted a quick way to test how well or poorly changes affect a site. Normally I’d whip out the ol’ calculator app and do this manually. That got tiring after a while, so instead with the help of ChatGPT, I made this little bash script that will do the work for you:

    #!/bin/bash

# Function to calculate the average of a list of numbers
average() {
	local sum=0
	local count=0

	for num in "$@"; do
		sum=$(echo "$sum + $num" | bc -l)
		count=$((count+1))
	done

	echo "$sum / $count" | bc -l
}

# Parse arguments
for i in "$@"; do
	case $i in
		--before=*)
		BEFORE="${i#*=}"
		shift
		;;
		--after=*)
		AFTER="${i#*=}"
		shift
		;;
		*)
		# unknown option
		;;
	esac
done

# Check if both BEFORE and AFTER parameters are provided
if [ -z "$BEFORE" ] || [ -z "$AFTER" ]; then
	echo "Error: Missing required parameters."
	echo "Usage: $0 --before=<comma-separated-values> --after=<comma-separated-values>"
	exit 1
fi

IFS=',' read -ra BEFORE_LIST <<< "$BEFORE"
IFS=',' read -ra AFTER_LIST <<< "$AFTER"

# Calculate average for before and after lists
BEFORE_AVG=$(printf "%.2f\n" $(average "${BEFORE_LIST[@]}"))
AFTER_AVG=$(printf "%.2f\n" $(average "${AFTER_LIST[@]}"))

echo "Before average: $BEFORE_AVG"
echo "After average: $AFTER_AVG"

# Calculate average percent increase, decrease or no change for the list
if [ "$BEFORE_AVG" != "0.00" ]; then
	PERCENT_CHANGE=$(echo "(($AFTER_AVG - $BEFORE_AVG) / $BEFORE_AVG) * 100" | bc -l)
	if [ "$(echo "$PERCENT_CHANGE > 0" | bc -l)" -eq 1 ]; then
		printf "Average percent increased: %.2f%%\n" "$PERCENT_CHANGE"
	elif [ "$(echo "$PERCENT_CHANGE < 0" | bc -l)" -eq 1 ]; then
		printf "Average percent decreased: %.2f%%\n" "$PERCENT_CHANGE" | tr -d '-'
	else
		echo "No change in average."
	fi
else
	echo "Percent change from before to after: undefined (division by zero)"
fi
Code language: Bash (bash)

    It runs like this:

    $ average.sh --before=13.07,9.75,16.14,7.71,10.32 --after=1.22,1.28,1.13,1.19,1.26
Before average: 11.40
After average: 1.22
Average percent decreased: 89.30%

    In this instance, it was calculating seconds–but you need to remember that it only goes to two decimal places, so if you need something finer you’ll need to adjust the code or your inputs.

    Happy Slacking!

  • Supercharge Your Clipboard with a Shell Function for iTerm2

    Supercharge Your Clipboard with a Shell Function for iTerm2

    Well, I’ve been playing with ChatGPT more, and I had ChatGPT (GPT-4) write this blog post for me. I told it that I was going to give it some of my blog posts to determine my writing style, and to have it write me a blog post about this bash function. It’s …. meh … not really “my style” but whatever.

    The idea of this shell function is to integrate with iTerm2’s proprietary escape sequences to allow you to pipe content right to your clipboard. I’m using the name pbcopy from MacOS so that I don’t need to think about it, and can use pbcopy anywhere. Now I can work on a remote server and cat stupid-stuff.log | pbcopy and get a copy of it locally to mess with, without having to click and drag or download the file. Super lazy ❇

    Enjoy the AI blog post:

    Hey folks, today we’re going to explore a neat shell function that can make your life a lot easier when you’re working in iTerm2 on macOS. The purpose of this function is to make the built-in pbcopy command work seamlessly in iTerm2, even when the command isn’t available in your shell environment.

    Without further ado, let’s dive into the code:

    # pbcopy via iTerm2
function pbcopy() {
	if which pbcopy >/dev/null 2>&1; then
		pbcopy "$@"
	else
		# Replace ^[ with the ASCII escape character
		local start="\e]1337;CopyToClipboard\a"
		local end="\e]1337;EndCopy\a"
		printf "${start}$(cat)${end}"
	fi
}
Code language: PHP (php)

    You might be wondering how this magic works. Let’s break it down step by step:

    1. The function first checks if the pbcopy command is available in your shell environment using the which command. If it is, it simply calls the built-in pbcopy command with the provided arguments.
    2. If the pbcopy command isn’t available, the function creates two local variables, start and end, containing special escape sequences that iTerm2 understands.
    3. The printf command is then used to concatenate the start escape sequence, the contents of the input (read using cat), and the end escape sequence. This formatted string is what gets sent to iTerm2.

    When iTerm2 receives this specially formatted string, it recognizes the escape sequences and takes care of copying the text between them to the system clipboard.

    And that’s it! With this function in your shell environment, you can copy text to your clipboard using pbcopy without any hassle, whether the command is natively available or not. To use this function, simply add it to your shell startup file (like .bashrc or .zshrc), and you’re good to go.

    In a practical scenario, let’s say you’re working on a project and you want to share the contents of a log file with a colleague. Instead of manually opening the log file, selecting its content, and copying it to your clipboard, you can use this shell script to streamline the process. With the pbcopy function added to your shell startup file, simply open your terminal and run the command cat logfile.txt | pbcopy. This command will read the content of the logfile.txt file and pipe it directly to the pbcopy function, which in turn places the content into your clipboard. Now you can easily paste the log file content into an email, chat application, or any other medium you choose, all without leaving the comfort of your terminal. This not only saves time but also enhances your overall productivity when working with text-based data in a command-line environment.

    In conclusion, this shell function is a fantastic little tool to enhance your iTerm2 experience on macOS. I hope you find it as useful as I do. As always, if you have any questions or comments, feel free to drop them below. Happy coding!

    Happy coding, indeed 🤖

  • More Garbage Sysadmin: Reboot Linux Server on Kernel Panic

    More Garbage Sysadmin: Reboot Linux Server on Kernel Panic

    Just like restarting a server when the memory is low, I’ve had a recent problem with kernel panics on my Raspberry Pi, and I’ve found a terrible solution to fix it: Just reboot.

    Setting the /proc/sys/kernel/panic file contents to a non-zero integer will reboot the server on kernel panic after that many seconds.

    Because I’m lazy, I asked ChatGPT to write me up a startup script to do this for me, and here’s what I have now:

    To set the panic timeout value on Ubuntu Server 20.04 and later versions, you can create a systemd service unit file.

    Here are the steps to create a systemd service unit file:

    1. Open a terminal window on your Ubuntu Server.
    2. Create a new service unit file with the following command:

      sudo nano /etc/systemd/system/panic-timeout.service

      This will open a new file named panic-timeout.service in the nano text editor with superuser privileges.
    3. Add the following lines to the file:
    [Unit]
Description=Panic Timeout

[Service]
Type=oneshot
ExecStart=/bin/bash -c "echo 60 > /proc/sys/kernel/panic"

[Install]
WantedBy=multi-user.target
Code language: JavaScript (javascript)

    This service unit file sets the panic timeout to 60 seconds.

    1. Save the file by pressing Ctrl+O, then exit nano by pressing Ctrl+X.
    2. Reload the systemd daemon to recognize the new service unit file with the following command:

      sudo systemctl daemon-reload
    3. Enable the service unit file to run at boot time with the following command:

      sudo systemctl enable panic-timeout.service
    4. Reboot the server to test the service unit file. After the server reboots, the panic-timeout.service will automatically run the echo command and set the panic timeout to 60 seconds.

    That’s it! With these steps, you can set the panic timeout value on the latest versions of Ubuntu Server.

    Well there you have it! Don’t forget to follow along for more terrible ideas!

  • Get First Date of Week in Google Sheets, When the Week of the Year is Related to the Row Number

    Get First Date of Week in Google Sheets, When the Week of the Year is Related to the Row Number

    *phew* what a title

    For a recent project, I wanted to set up a Google Sheet that tracked something daily/weekly. I wanted one column to show the starting day of the week to help keep me on track, and I ended up with this:

    I used ChatGPT to help me make the formula to determine the first day of the week, indexed to the current row:

    =DATE(YEAR(TODAY()), MONTH(DATE(YEAR(TODAY()),1,1)+(ROW()-2)*7-WEEKDAY(DATE(YEAR(TODAY()),1,1))+1), DAY(DATE(YEAR(TODAY()),1,1)+(ROW()-2)*7-WEEKDAY(DATE(YEAR(TODAY()),1,1))+1))

    This is how ChatGPT explains how the function works:

    This Google Sheets function returns the date that corresponds to the first day of the week for a given row number. The row number is specified by the ROW() function, which returns the row number of the current cell.

    The function uses the following steps to calculate the date:

    1. It uses the YEAR(TODAY()) function to get the current year.
    2. It uses the DATE(YEAR(TODAY()),1,1) function to get the first day of the current year (e.g. January 1).
    3. It uses the WEEKDAY(DATE(YEAR(TODAY()),1,1)) function to get the day of the week of the first day of the year as a number (e.g. 1 for Monday, 2 for Tuesday, etc.).
    4. It uses the (ROW()-2)*7 expression to calculate the number of days to skip from the first day of the year to get to the desired week. For example, if the ROW() function returns 2, then the expression will evaluate to 0, which means that no days need to be skipped. If the ROW() function returns 3, then the expression will evaluate to 7, which means that one week needs to be skipped.
    5. It uses the DATE(YEAR(TODAY()),1,1)+(ROW()-2)*7-WEEKDAY(DATE(YEAR(TODAY()),1,1))+1 expression to calculate the date that corresponds to the first day of the week for the given row number.
    6. It uses the MONTH(DATE(YEAR(TODAY()),1,1)+(ROW()-2)*7-WEEKDAY(DATE(YEAR(TODAY()),1,1))+1) function to get the month of the calculated date.
    7. It uses the DAY(DATE(YEAR(TODAY()),1,1)+(ROW()-2)*7-WEEKDAY(DATE(YEAR(TODAY()),1,1))+1) function to get the day of the month of the calculated date.
    8. It uses the DATE() function to combine the year, month, and day into a single date.

    For example, if the ROW() function returns 2 and the current year is 2022, the function will return the date January 3, 2022. If the ROW() function returns 3 and the current year is 2022, the function will return the date January 10, 2022.

    It took a lot of trial and error, and manual adjustments to get the function to work, and it was really fun to play with ChatGPT and work together to get this working. While I could have trial and errored myself to the end functionality eventually, this did end up saving me quite a bit of time.

    If you, random Internet stranger, want to use this function yourself, but your rows aren’t lined up the same as mine, you’ll just need to adjust the (ROW()-2) type areas to change the row offset you’re currently starting at.

  • iOS Reminders to Habitica To Do’s via IFTTT

    iOS Reminders to Habitica To Do’s via IFTTT

    After digging around for a while trying to see how I could link up iOS’s Reminders with Habitica‘s To Do’s to help keep me organized, I finally found an easy way through IFTTT.

    This works easily because Habitica offers a wonderful API💥

    Specifically we’re looking at the “Create a new task belonging to the user” API endpoint:

    https://habitica.com/api/v3/tasks/user

    With this, we’ll need to make a POST request with some special headers to authenticate and then a body payload made of JSON:

    Headers:

    X-Client: my-user-id-IFTTTiOSRemindersSync
X-API-User: my-user-id
X-API-Key: my-api-keyCode language: HTTP (http)

    Body:

    {
	"text": "{{Title}}",
	"type": "todo",
	"notes": "{{Notes}} (Imported from iOS Reminders via IFTTT)"
}Code language: JSON / JSON with Comments (json)

    From here, IFTTT will fill in the title, notes, and ship it off to Habitica for me to check off for some sweet XP!

  • Raspberry Pi: February 2021

    Raspberry Pi: February 2021

    I often feel like I don’t get enough use out of my Raspberry Pi devices. I have a Raspberry Pi Model B Revision 2.0 (512MB) and a Raspberry Pi 4 (4GB).

    Their current uses are as follows

    Raspberry Pi Model B:

    Raspberry Pi 4:

    I’d love to do more, but I’m really not sure what else I need. I don’t need some grandiose home server but I absolutely love playing with these little things!

  • Pi-hole, Google Wifi, and Device Names

    Pi-hole, Google Wifi, and Device Names

    One of the things that bothered me for quite some time with my Pi-Hole was that using it with Google Wifi (first gen), it wouldn’t automatically detect device hostnames. I’d done a lot of googling and never could get it to work even after a lot of different trials with multiple settings.

    Eventually I gave up and instead wrote a command that would use nmap to help fill in the gaps, and output to /etc/pihole/custom.list:

    #!/bin/bash
if [ "$(id -u)" != "0" ]; then
	echo "This script must be run as root" 1>&2
	exit 1
fi

echo -n "Looking up MAC and IPs"
for ip in "192.168.1.0" "192.168.2.0" "192.168.3.0"; do
	echo -n .
	# This is very slow.
	nmap -sP "$ip"/20 > "$ip"-nmap.txt
done
echo

# Mega-command to turn the nmap output into a CSV.
cat 192.168.?.0-nmap.txt \
	| sed '/^Starting Nmap/ d' \
	| sed '/^Host is up/ d' \
	| sed '/^Stats:/ d' \
	| sed '/^Ping Scan Timing:/ d' \
	| sed '/^Nmap done:/ d' \
	| sed -z 's/\nMAC/,MAC/g' \
	| sed -e 's/Nmap scan report for //g' \
	| sed -e 's/MAC Address: //g' \
	| sed -e 's/ (/,(/g' \
	| grep -Ev $'^[0-9.]+$' \
	| sort -u > ip-mac-mapping.csv

rm /etc/pihole/custom.list 2> /dev/null
while IFS=, read -r col1 col2 col3
do
	# Strip out opening and closing parenthesis.
	col3="${col3//[\(\)]/}"

	# Replace all non-alphanumeric characters with dashes.
	col3="${col3//[^[:alnum:]]/-}"

	# Manually name some of the MACs I already know.
	case "$col2" in
	"24:05:88:XX:XX:XX")
		col3="Google-Wifi"
		;;
	"B0:19:C6:XX:XX:XX")
		col3="Derricks-iPhone"
		;;
	"CC:44:63:XX:XX:XX")
		col3="iPad-Pro"
		;;
	"C8:D0:83:XX:XX:XX")
		col3="Apple-TV-Den"
		;;
	"50:32:37:XX:XX:XX")
		col3="Apple-TV-Bedroom"
		;;
	"DC:A6:32:XX:XX:XX")
		col3="Ubuntu-Server"
		;;
	"38:F9:D3:XX:XX:XX")
		col3="Derrick-MBP"
		;;
	*)
		echo -n
		;;
	esac

	# For some reason, this one is still funky, so I'm adding in a special case for it.
	# Could have just been weird caching during my testing.
	case "$col1" in
	"192.168.1.1")
		col3="Google-Wifi"
		;;
	*)
		echo -n
		;;
	esac

	# The PiHole custom.list is supposed to follow the hosts standard, but it seems that
	# it is not happy with tabs and comments :sadpanda:
	echo "$col1	$col3 # $col2"
	echo "$col1 $col3" >> /etc/pihole/custom.list
done < ip-mac-mapping.csvCode language: PHP (php)

    This will attempt to grab some info about all the devices on your network via nmap, but also allow you to manually override that per IP or per MAC. I have of course stripped out some devices and semi-anonymized my MACs in the above example.

    The nmap can be pretty slow, especially if you’re running this on a first gen Raspberry Pi like I am.

Close

Ad-Blocker Not Detected 🛑

Consider installing a browser extension that blocks ads and other malicious scripts in your browser to protect your privacy and security, conserve your expensive data plan, and cut your online carbon footprint. Learn more.