Introduction to Unix

Introduction to Unix, with an emphasis on Linux

• RedHat family: RedHat Enterprise Linux (RHEL), Fedora, CentOS, Rocky Linux
• Debian family: Debian, Ubuntu, Mint
• SUSE Linux Enterprise Server (SLES)

• Long-form options are --long-option
  when a value is expected, the equal sign is used: --long-option=<value>
• Usually short options take the form -X and can be aggregated, eg: -XYZ is -X -Y -Z
  when a value is expected, it is preceeded by a space -X <value>

• bash: (Bourne Again Shell) the standard for Linux
• zsh: the standard for MacOS, compatible with bash
• csh, tcsh: legacy shell, installed on Linux for backward compatibility

• ls *.jpg shows only the files ending with the extension .jpg
• ls *jwst* shows all the files which contain jwst in their name

• cp -r <source> <destination> copies a directory recursively
• touch <filename> creates an empty file, or changes its date to the current date and time

[user1@ws01 tests]$ file test_db.c
test_db.c: C source, ASCII text
[user1@ws01 tests]$ file test_db.o
test_db.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
[user1@ws01 tests]$ file test_db
test_db: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, not stripped

• To have a shortcut to a directory
• When multiple versions of the same file exist, to create a default one
• To share read-only data from a common source

[user1@ws01 folder]$ ln -s code/bin/test_db .
[user1@ws01 folder]$ ls -l test_db 
lrwxrwxrwx 1 user1 group1 16 Jul 24 22:10 test_db → code/bin/test_db

• find <directory> -name <pattern> looks in directory for files with the given pattern
• It can be useful to look only for regular files, in this case, use the option -type f

[user1@ws01 folder]$ find . -name "*.o" # Find object files
./code/src/tests/test_db.o
./code/src/database/db.o

• Read set to true (1) is 4 in decimal
• Write set to true is 2 in decimal
• Execute set to true is 1 in decimal
• The total is the mask. For instance r-x would be 1*4 (r) + 0*2 (w) + 1*1 (x) = 5
• chmod 775 will correspond to rwx rwx r-x

• chown <username> <file_or_dir> will change the owner of the file or directory
• chgrp <groupname> <file_or_dir> will change the group of the file or directory

• head <filename> shows the beginning of a file
• tail <filename> shows the end of a file
  tail -f <filename> keeps the file open and lets you see lines being added

• wc -l <filename> counts the number of lines in a file
• <command> | wc -l counts the number of results returned by command

I like cats: they are great animals, and their name is a Unix command.
Mine is called Cathy.
On the other hand, dogs are friendly and not as reclusive.
Overall, cats and dogs are good pets.

[user1@ws01 input]$ grep cat animals.txt 
I like cats: they are great animals, and their name is a Unix command.
Overall, cats and dogs are good pets.
# If we use -i, the search becomes case-insensitive:
[user1@ws01 input]$ grep -i cat animals.txt
I like cats: they are great animals, and their name is a Unix command.
Mine is called Cathy.
Overall, cats and dogs are good pets.

[user1@ws01 input]$ sed 's/cat/bear/' animals.txt
I like bears: they are great animals, and their name is a Unix command.
Mine is called Cathy.
On the other hand, dogs are friendly and not as reclusive.
Overall, bears and dogs are good pets.

• The sort <filenames> command sorts (alphabetically) concatenated text files line by line
• uniq <filename> is used to remove duplicate lines from a single text file
• To combine sorting and duplicate removals, use sort -u <filenames>

Paul            Peterson        34      paul@fi.com     London
Julia           Smith           23      jsmith@sf.com   Boston
Mary-Jane       Allgood         54      mj@gmail.com    Vancouver
Peter           Maxwell         45      pete@fi.com     Atlanta

[user1@ws01 input]$ awk '{print $1","$3}' users.dat
Paul,34
Julia,23
Mary-Jane,54
Peter,45

• .tar.gz files: tar czvf <archive_name>.tar.gz <dir_to_archive>
• .tar.bz2 files: tar cjvf <archive_name>.tar.bz2 <dir_to_archive>
• .tar.xz files: tar cJvf <archive_name>.tar.gz <dir_to_archive>

[user1@ws01 tmp]$ ls
folder
[user1@ws01 tmp]$ tar czvf folder.tar.gz folder
folder/
folder/code/
folder/code/bin2/
folder/code/bin2/hello_world
folder/code/src/
folder/code/src/tests/
folder/code/src/tests/hello_world.c
…
[user1@ws01 tmp]$ ls
folder                folder.tar.gz   

[user1@ws01 ~]$ my_file=firstfile
[user1@ws01 ~]$ echo $my_file
firstfile
[user1@ws01 ~]$ echo $my_file_prev   # Unknown variable my_file_prev

[user1@ws01 ~]$ echo ${my_file}_prev # Will properly append
firstfile_prev

• VISUAL, EDITOR the default editor used when files need to be changed
• DISPLAY: used when remote programs display on that computer screen. Usually :0
• PATH: directories containing executables (see details here)
• LD_LIBRARY_PATH: directories containing dynamic libraries

[user1@ws01 folder]$ cat myenv.sh 
export MY_VERSION=4.5
export MY_GCC=/usr/bin/gcc
[user1@ws01 folder]$ bash myenv.sh # Executing the code
[user1@ws01 folder]$ echo "$MY_VERSION $MY_GCC"

[user1@ws01 folder]$ source myenv.sh # Source'ing it
[user1@ws01 folder]$ echo "$MY_GCC $MY_VERSION"
/usr/bin/gcc 4.5

• rsync -a <source> <destination>
• rsync -arz user2@server:/mnt/home/user2/data/ /home/user1/data --delete syncs the content of a remote directory from server to the local computer, recursively (-r), using compression (-z), and deleting removed files (--delete)

[user1@ws01 ~]$ ping www.google.com
PING www.google.com (142.250.176.196) 56(84) bytes of data.
64 bytes from XXXX-YY-ZZZZ (142.250.176.196): icmp_seq=1 ttl=115 time=1.04 ms
64 bytes from XXXX-YY-ZZZZ (142.250.176.196): icmp_seq=2 ttl=115 time=0.992 ms
^C
--- www.google.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4004ms
rtt min/avg/max/mdev = 0.930/0.977/1.035/0.040 ms

[user1@ws01 bin]$ ls # Let's check what files are in this directory
hello_world
[user1@ws01 bin]$ hello_world # Launch the program
bash: hello_world: command not found
[user1@ws01 bin]$ ./hello_world
Hello World # It worked!

[user1@ws01 bin]$ pwd
/tmp/folder/code/bin
[user1@ws01 bin]$ ls
hello_world
[user1@ws01 bin]$ hello_world
bash: hello_world: command not found
[user1@ws01 bin]$ echo $PATH
/usr/local/bin:/usr/bin:/bin  # aka /usr/local/bin, /usr/bin, /bin
[user1@ws01 bin]$ export PATH=/tmp/folder/code/bin:$PATH # Do not forget to put $PATH at the end, otherwise you will lose standard commands!
[user1@ws01 bin]$ echo $PATH
/tmp/folder/code/bin:/usr/local/bin:/usr/bin:/bin
[user1@ws01 bin]$ hello_world 
Hello World

[user1@ws01 code]$ ls
bin  bin2  include  lib  src
[user1@ws01 code]$ ls bin
hello_world
[user1@ws01 code]$ ls bin2
hello_world
[user1@ws01 code]$ which hello_world # Which version is used?
/tmp/folder/code/bin/hello_world
[user1@ws01 code]$ export PATH=/tmp/folder/code/bin2:$PATH  # Prepend the location of the new executable
[user1@ws01 code]$ echo $PATH
/tmp/folder/code/bin2:/tmp/folder/code/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin
[user1@ws01 code]$ which hello_world # Which version is used?
/tmp/folder/code/bin2/hello_world

• You can use the logical AND && to conditionally run each command only if the previous command succeeds (the return code is 0). a && b runs command a, and then only runs command b if a succeeds

• stdout is used for the results of the operation
• stderr is used for error messages, warnings, information

• > redirects stdout
• 2> redirects stderr
• &> redirects both of them, as they would appear in a terminal

[user1@ws01 ~]$ command > command.out 2> command.err

[user1@ws01 folder]$ my_var=`ls -l`
[user1@ws01 folder]$ echo $my_var # New lines are replaced by spaces!
total 0 drwxrwxr-x 7 user1 group1 66 Jun 24 12:23 code drwxrwxr-x 5 user1 group1 47 Jun 23 14:38 data -rw-r--r-- 1 user1 group1  0 Jun 24 17:03 README.txt

[user1@ws01 ~]$ echo $((3*6 - 1 + 5**2)) # ** is the exponentiation
42

[user1@ws01 ~]$ for prime in 1 2 3 5 7 11
> do
> echo "A prime: $prime"
> done
A prime: 1
A prime: 2
A prime: 3
A prime: 5
A prime: 7
A prime: 11
[user1@ws01 ~]$ for vowel in a e i o u y; do echo "m$vowel"; done # as a single line
ma
me
mi
mo
mu
my

if [[ $i -eq 0 ]]; then # EQual for numbers
   echo "Null value"
elif [[ $i -lt 5 ]]; then # Less Than
   echo "The number is 5"
else
   echo "The number is larger than 5"
fi

#!/bin/bash
# The first line above is called the shebang, and tells Unix how to interpret the instructions contained in that file
# Comments start with the pound/hashtag sign

echo "Hello World!"

[user1@ws01 ~]$ ./hello_world.sh
Hello World!

[user1@ws01 ~]$ ssh -Y remote_username@remote_computer
remote_username@remote_computer's password: 
[remote_username@remote_computer ~]$ xterm # this will launch a new terminal which will be opened on your local computer, but whose commands are executed on the remote one

Daemon

a program that runs as a background process

Service

a program that gets requests from other processes. Usually implemented as a daemon. Eg: sshd

• RedHat family use yum
  yum info <package_name> is used to query if a package exists or is installed (can use regular expressions)
  yum info <package_name> installed a package Need to be root
• Debian and related use apt
  apt info <package_name> is used to query packages
  apt install <package_name> installed a package Need to be root