Favorite Linux: Shell Aliases -- gitupdate

 I hope you love these as much as I do.  Here are a few of my favorite aliases (ref: man alias):

1.    Git Update

    Personally, I like to update (commit) my work frequently and often.  Many moons ago, when God was still a baby, we used WordPerfect on the mono-chrome Wyse terminal.  Too often, our documents were completely lost before completion, and it was always our own damn fault because we hadn't saved our work frequently enough.  This brings us back to habit #1 and today.  Now we use Git/GitLab to save much of our work (also from a terminal).  

    To save work (commit) there are 3 commands which perform: 1) add new files, 2) save the state with a message (what you did), and 3) upload to the central repository.  While the message is important for collaboration with other team members, it's not necessary for solo work.  This macro conserves time, encourages you to save, and includes the relevant info needed (whodunit, hostname, and time of day).  We find it informative and time saving.

256 color directory listing

 LS_COLORS='rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:';
export LS_COLORS

Randomly Colorful Unix Terminal Prompt

#!/usr/bin/env bash
### ./gen_prompt.sh
### Output: bash script that configures a randomly colorful prompt (PS1).
###   matt.a.feenstra@gmail.com
### For Example:
###   eval $(gen_prompt.sh)

echo 'PROMPT_DIRTRIM=4'

PS_STRING="'\[\033[38;5;$(shuf -i 27-231 -n 1)m\]\u\[\033[38;5;$(shuf -i 27-231 -n 1)m\]@\[\033[38;5;$(shuf -i 27-231 -n 1)m\]\h\[\033[38;5;$(shuf -i 27-231 -n 1)m\]:\[\033[38;5;$(shuf -i 27-231 -n 1)m\]\j\[\033[38;5;$(shuf -i 27-231 -n 1)m\]:\[\033[38;5;$(shuf -i 27-231 -n 1)m\]\w\[\033[38;5;$(shuf -i 27-231 -n 1)m\]\\$\[\033[00m\] '"

echo export PS1=$PS_STRING
 

HOWTO: Use Ruby IO with shell execution (STDIN, STDOUT, STDERR, PID)

# 1. IO.popen

# Runs the specified command as a subprocess; the subprocess's standard

# input and output will be connected to the returned IO object. This

# allows you to provide STDIN input and get STDOUT output easily.

# Asynchronous (IO objects)

# Return: IO object, (IO#pid, IO#read)

# Docs: https://www.rubydoc.info/stdlib/core/IO.popen

io = IO.popen("echo 'hi'") # Or IO.popen(["echo", "hi"])

io = IO.popen(cmd)

IO.popen(["echo", "'hi'"]) do |io|

# ...

end

# 2. open3

# Runs the specified command as a subprocess; the subprocess's standard

# input, stdout, and stderr IO objects are available. There is also

# an "open4" gem to more easily get the PID of the child process.

# Synchronous (get strings) or Asynchronous (IO objects)

# Return: Strings (capture*) or IO objects (popen*)

# Docs: https://docs.ruby-lang.org/en/2.5.0/Open3.html#method-c-popen3

require 'open3'

stdin_io, stdout_io, stderr_io, process_waiter = Open3::popen3(cmd)

stdout_str, stderr_str, process_info = Open3::capture3(cmd)

require 'open4'

pid, stdin, stdout, stderr = Open4::popen4(cmd);

# Extra Advice - Exit Code

# $? which is the same as $CHILD_STATUS (if you require 'english')

# Accesses the status of the last system executed command if

# you use the backticks, system() or %x{}.

# You can then access the ``exitstatus'' and ``pid'' properties

# Docs: https://ruby-doc.org/core-2.7.1/Process/Status.html#method-i-exitstatus

$?.exitstatus

HOWTO: Shell redirection (pipe) and output into Ruby scripts

    Ruby syntax is a little different than most programming languages.  However I have found that it is worth the time and effort required in learning the style.  Here is an easy way to use Ruby and Linux Bash shell seamlessly through redirection, giving a world of capabilities, simply:

$ echo 'stuff' | ruby -e 'puts "this -> #{ARGF.read.chomp} <- is it."'
 

this -> stuff <- is it.
 

$

    Here we see the ARGF constant is assigned data from the left hand side as a String:

 

$ echo 'stuff' | ruby -e 'puts "this -> #{ARGF.read.class} <- is it."'

this -> String <- is it.

$

That's all there is to it!  Hope you've learned something, and I welcome your comments.  Have a great day!

--Matt

HOWTO: Control Spotify and your MacOS volume from the command line

I use the following script called "~/.music.automation.sh" and load it from my "~/.bash_profile" with "source ~/.music.automation.sh".
---

# ~/.music.automation.sh

# Volume stuff
alias vol="osascript -e 'output volume of (get volume settings)'"
alias 0="osascript -e 'set Volume 0'"
alias 1="osascript -e 'set Volume 0.3'"
alias 2="osascript -e 'set Volume 0.6'"
alias 3="osascript -e 'set Volume 0.9'"
alias 4="osascript -e 'set Volume 1.2'"
alias 5="osascript -e 'set Volume 1.5'"
alias 6="osascript -e 'set Volume 1.8'"
alias 7="osascript -e 'set Volume 2.1'"
alias 8="osascript -e 'set Volume 2.4'"
alias 9="osascript -e 'set Volume 2.7'"
alias 10="osascript -e 'set Volume 3.0'"

# Synonyms for above
alias m="osascript -e 'set Volume 0'"


# Spotify stuff
alias next="osascript -e 'tell application \"spotify\" to next track'"
alias n="osascript -e 'tell application \"spotify\" to next track'"
alias previous="osascript -e 'tell application \"spotify\" to previous track'"
alias prev="osascript -e 'tell application \"spotify\" to previous track'"
alias p="osascript -e 'tell application \"spotify\" to previous track'"
alias pp="osascript -e 'tell application \"spotify\" to playpause'"



---
Then, I can SSH into MacOS and control the music remotely, from my phone for example.  Enjoy!

HOWTO: Add loopback as a service

This can be easily done by adding an "ip" command to system startup with RHEL8 "systemctl".

1. Create a run script for ip to setup a loopback device on 192.168.10.1:

echo 'ip addr add 192.168.10.1/24 dev lo' >/usr/local/sbin/loopback2.sh
chmod 700 /usr/local/sbin/loopback2.sh


2. Create the service file:

vi /lib/systemd/system/loopback2.service
or
systemctl edit loopback2


2. Have it start as service with systemctl:


[Unit]
after=network

[Service]
ExecStart=/usr/local/sbin/loopback2.sh

[Install]
WantedBy=default.target


3. Enable and auto start:

systemctl enable loopback2.service
systemctl start loopback2.service


4. Confirm the setup:

ip addr
systemctl status loopback2

HOWTO: PostgresDB & pgAdmin4 setup on RHEL 8

https://yum.postgresql.org

https://yum.postgresql.org/howtoyum.php

---

dnf -y install https://download.postgresql.org/pub/repos/yum/reporpms/EL-8-x86_64/pgdg-redhat-repo-latest.noarch.rpm

dnf install postgresql12-server postgresql12-contrib

---

yum install pgadmin4

/usr/pgadmin4/bin/./pgadmin4-web-setup.sh

service httpd start

Handy UNIX diff: Exclusive match

Super handy tool for selectively removing serial contents from a text file:

#!/bin/bash
# xdiff <new_file> <old_file>
# Exclusive diff - just show what's different from the first file
#       example: xdiff SYMBOLS.txt remove_symbols.txt >SYMBOLS2.out
diff -u $1 $2 | grep '^-[^-]' | sed 's/^-//'

Best prompt ever, Improved!

#!/bin/bash

u_color="\e[38;5;212m"

amp_color="\e[38;5;183m"

h_color="\e[38;5;51m"

col_color="\e[1;34m"

dir_color="\e[1;32m"

ps_color="\e[38;5;226m"

reset="\e[0m"

PS1="$u_color\u$amp_color@$h_color\h$col_color:$dir_color\w$ps_color\$$reset "

echo "PS1=\"$PS1\""

Chef HOWTO: Encrypted Data Bags

---

Create a new encrypted data bag

---

1. Create a secret for the data bag

$ openssl rand -base64 1024 > db_secret_key.txt

---

2. Make an encrypted data bag:

$ export EDITOR=vim

$ knife data bag create BAG_NAME --secret-file db_secret_key.txt

Created data_bag[BAGNAME]

---

3. Create data bag entry and edit contents:

$ knife data bag create BAG_NAME ITEM_NAME --secret-file db_secret_key.txt

Data bag BAG_NAME already exists

Created data_bag_item[ITEM_NAME]

---

4. (optional) Download data bag as json:

knife download data_bags/BAG_NAME

---

5. (optional) Upload data bag from json file:

knife data bag from file BAG_NAME BAG_FILENAME.json

---

6. (optional) Access data bag from a recipe:

db_secret = Chef::EncryptedDataBagItem.load_secret('/path/to/db_secret_key.txt')

private_key_value = Chef::EncryptedDataBagIten.load('BAG_NAME', 'ITEM_NAME', db_secret)['ITEM_KEY']

### note: ITEM_KEY is often an Environment name

---

Super Easy Encryption w/ GPG

To import somebody's public key:

gpg --import your_friend.pub

To encrypt a text file for them:

gpg -a -se -r 'friend name in address book' FILENAME.txt

To list the keys (your friends') in your address book:

gpg --list-keys

To import a new friend's key:

gpg --import friend_key.pub


To generate a new private key for yourself:

gpg --full-generate-key
or
gpg --default-new-key-algo rsa4096 --gen-key

Most important SED syntax

This is how Unix SED was meant to be used.

Take a text file and substitute 1 regex match for a new one, in place.

My file:
-rw-r--r-- 1 matt matt  7338 May  6 00:05 merge.rb


sed -i s/OLDVALUE/NEWVALUE/g merge.rb


-i means in-place
-s means substitute
-g means globally

Ruby - How to make your program run from any directory (with libraries)

#!/usr/bin/ruby

### Save the folder name that this lives in
folder = File.expand_path('.',__dir__)

### Add to the Ruby default load path the pwd this was executed from
$LOAD_PATH.unshift(folder) unless $LOAD_PATH.include?(folder)

### Here we can now interpolate a relative path
load "#{folder}/../lib/modules.rb"

puts "stub.rb pwd: #{Dir.pwd}"

Ruby - Check for a regexp in a file

True/False

File.readlines(text_filename).grep(/myRegexp/).any?

An even better PS1

\[\033[38;5;201m\]\u\[\033[38;5;51m\]@\[\033[38;5;183m\]\h\[\033[1;34m\]:\[\033[1;32m\]\w\[\033[1;31m\]\$\[\033[00m\]

Ruby: Best way to boolean grep for a string contained in a file

File.readlines('./mystuff.txt').grep(/Word123/).any?

Best vim Color Mod

Put these files in your (unix) home directory:

~/.vim/autoload/palenight.vim
https://github.com/drewtempelmeyer/palenight.vim/blob/master/autoload/palenight.vim



~/.vim/colors/palenight.vim
https://github.com/drewtempelmeyer/palenight.vim/blob/master/colors/palenight.vim



Autoload?  Add line this to your ~/.vimrc

colorscheme palenight

Super Easy Ruby ERB template

#!/opt/chef-workstation/embedded/bin/ruby -w

 

require 'erb'

 

#template = File.read('./mytemplate.txt.erb')

 

template = 'i want to put stuff ===> <%= "here" %> <==='

 

obj_out = ERB.new(template)

 

STDOUT.puts obj_out.result

GitLab: Clone all the repos!

#!/home/matt_feenstra/.rvm/rubies/ruby-2.3.1/bin/ruby -w
require 'json'
gitlab        = '10.252.1.229'
private_token = 'gog3AkrumotxU2m6wqSP'
###
json_obj = JSON.parse(`curl --header "PRIVATE-TOKEN: #{private_token}" "http://#{gitlab}/api/v3/projects/all?per_page=1000"`)
repos = Array.new
# has format "Group_name/Project_name"
json_obj.each do |record|
repos.push(record['path_with_namespace'])
end
repos.sort!
git_paths = Array.new
groups    = Array.new
# construct git clone command
repos.each do |repo|
groupname, projectname = repo.split('/')
groups.push(groupname)
git_paths.push("git clone git@#{gitlab}:#{groupname}/#{projectname}.git")
end
groups.uniq!
# put them in separate directories per group
groups.each do |dir|
system("mkdir #{dir}")
end
# git clone them all into 'groupname' subdirs
git_paths.each do |clone|
groups.each do |groupname|
# grab the project name
projectname = String
if clone =~ /#{groupname}\/(.*)\.git/ then
projectname = $1
end
# drop into subdirectories and clone
if clone =~ /#{groupname}/ then
system("cd #{groupname} && #{clone}")
# if clone_cookbooks.rb exists, run it for sub-cookbooks
subdir = "./#{groupname}/#{projectname}"
if File.exist?("#{subdir}/clone_cookbooks.rb") then
puts "\tRunning #{subdir}/clone_cookbooks.rb.."
system("cd #{subdir} && ./clone_cookbooks.rb")
end
end
end
end