Scripting        2

What is scripting        2

Shell        3

Shell Scripting        3

Bash        3

Difference between sh and bash        3

Bash commands syntax        3

General commands        3

To change shell in mac        4

To list shells        4

Udemy course Content starts from here        4

Directory Commands        4

Move around CD        5

Copy commands        5

Remove & Cut commands        6

Cat, Less and More        6

echo commands        6

Mkdir & Touch        7

Wildcards        7

Variables        8

Extraction of the string & count characters of the variables        8

Variables manipulation        9

Permanent Variable inside Login script        9

Read command        10

Redirect the output        10

REDIRECT ERROR        11

Pipe command        12

Grep command        13

Brace expansion        13

Permissions        14

Permissions in the terminal        14

Files’s relation with permissions        14

Change permission on files        15

Section 2        15

Run Bash script from terminal        15

Conditions        15

Conditions in Bash scripts        15

If on strings        16

If on Numbers        16

If on Files        16

Logic condition        17

Loops        17

Subshell        17

Source command        18

Type command        18

Alias in Bash        19

Command substitution        19

Parameters in Bash        20

Back script        20

Exit status        20

Functions        20

Variable scope in functions        20

CALC script        20

BC Environment        21

Redirect        22

Redirect input : here document        22

Redirect input : here string        22

Parameter SHIFT        22

CALC Script        23

Read multiple variables        23

Grep command        23

Internal field separator        23

Intro to the IFS        23

sed = stream editor        23

AWK        25

Scripting

Link: https://www.geeksforgeeks.org/bash-script-difference-between-bash-script-and-shell-script/ 

What is scripting

In computer programming, a script is defined as a sequence of instructions that is executed by another program.

Shell
Shell Scripting
Bash

Difference between sh and bash

Bash commands syntax

Command [options] [arguments]

Every command has a name

Command can have options (it’s like flag -a, -h, -iv).

Commands can have arguments

Ex; command = ls (list all content of the directory)

General commands
To change shell in mac

To list shells

Udemy course Content starts from here

Directory Commands

Commands

Function

pwd

cd ~

gives current directory

Home directory

ls // access particular folder

  1. ls 
  2. ls folderName
  3. ls -l
  4. ls -F
  5. ls -ld foldername

1.  List of items in the folder

2. Content of the folder

3. Gives you list in long format

  • If it starts with - // it is file ex; -rw-r--r--@
  • If it starts with d // it is folder ex; drwx------@

4. To know which one is executable and which one is a folder  

  • Adds * to the executable
  • Adds / to the folder
  • File doesn’t have anything

5. Information about folder // time

#

comments

Clear

Clear your screen

cd folderName

Change directory

Move around CD

Command

Function

Relative path

Relative path because it is relative to where you are right now.

Ex; if you are at  /gautampambhar/work/

cd Pace/shared  // this is the path is relative to where you are right now

It does not start with root(/). It always starts with folder name

Absolute path

Always starts with the root folder path (/). Ex; cd /user/gautampambhar/work/PACE

cd ..

Back one folder  

cd -

It will take you where you were before starting to use cd command

TAB

It will complete the incomplete folder name

ls -a

ls -af

To see hidden files

To see hidden folders

Copy commands 

Commands

Function

ls .

Gives list of items in current which is similar to ls command

  1. cp -v demo(file) folder(folderName)

  1. cp demo demo2
  2. cp demo demo2 demo3 folder(folderName)
  3. cp -i demo folder(folderName)
  4. cp -iv demo folder
  5. cp -iv demo ..
  1. Copy the file into the given directory/folder at last. -v stands for verbose, it prints copied file into mentioned folder on the command line  
  2. Copy the the existing file(demo) and give a new name at last of copied file // rename and create a new file
  3. Copy multiple files inside folder name you provide at last
  4. If you copy an existing file inside a foldr, -i will ask you to overwrite the file. -i stands for interactive option
  5. Use multiple option together (overwrite & verbose)
  6. Copy file to previous directory

cp ../demo.zip .

Copy file(in the parent directory) to present directory

cp -r folder1 folder2

Copies one folder(folder1) inside another folder(folder2)

-r stands for copy(recursively) everything inside folder1 into folder2  

Remove & Cut commands

Commands

Function

  1. rm -v demo
  2. rm -v -r folder
  3. rm -i demo
  1. Remove file // remove means it is deleted forever, can’t find in the trash
  2. Remove folder
  3. Remove with interactive mode

mv -v play/demo.txt .

Moves the file from play folder into current directory

mv -v folder1 folder2

Moves one folder into another // don’t require recursive -r in moving folders  

mv demo NewName.txt

Rename file from demo to NewName.txt

Cat, Less and More

Commands

Function

cat gautam

Print file content

cat -n  gautam

Prints all the line(in the file) in number

cat gautam newfile

It concatenates the file content of newfile into gautam file

more gautam

If a gautam file has too many lines you can use the “more” command which always starts printing content from the top. And you can go down by down arrow and quit the terminal by entering Q

less gautam

It opens a file content in a separate window. No content will be shown in the same terminal unlike More and cat commands

/

Type searchable(pattern) text and by “N” you can go to the next word and by “B” you can go to previous word

echo commands

        To print outputs in a different way

Commands

Function

echo

Print whatever you write inside parentheses

  1. echo -e “hi \n there”
  2. echo -e “hi \t there”
  1. Print words in a new line using \n(stands for new line)
  1. o/p:

Hi

There 

  1. Prints a words using tab \t(stand for tab space)
  1. o/p:

Hi        There

echo -e “hi \v there”

Print the second word using tab space. \v(stands for vertical tab)

o/p:

Hi

      there

Which is similar to echo -e “hello \n \t there”

Mkdir & Touch

Commands

Function

touch filename

To create a file

  1. touch demo
  2. touch -c asdfg

  1. This will update the existing file time with current time without changing file content.(update existing file time)(view the changed file time by ls -l)
  2. If the file doesn’t exist and you don’t want to create a new file then use it with -c
  1. mkdir foldername
  2. mkdir -p folder1/folder2
  1. Creates a folder
  2. To create 2 folder(parent and child)
Wildcards

Commands

Function

*

Any character in the file name

Usage:

  1. ls *.txt  // list all the files that has a .txt extension
  2. ls d*   // list all the files that starts with d character
  3. Copy: cp  *d folder1 // copy file that starts with d character inside folder 1                                        

ls ??.txt

ls ???.txt

Any single character

Usage:

List all the files that have only 2 characters: ls ??.txt

ls [abc]*.txt

List all the files that starts with either a,b or c character

o/p: afile.txt, bob.txt, cat.txt

ls [abc][ab].txt

Combination of characters.

First character has to be either a,b or c and second character has to be a or b

o/p: ab.txt

[!abc]

To exclude character. ! stands for bang

Any character that shouldn’t have a,b,c

Usage:

ls [!abc].txt  // will output file that doesn’t have a starting character from a,b,c

ls [[:alpha:]]*.txt

Lists files that starts with letter in the file name

ls [[:digit:]]*.txt

Lists files that starts with digits in the file name

ls [[:lower/upper:]]*.txt

Lists files that starts with lower/upper case character in the file name

Variables

Commands

Function

bob=103

To declare variable and it’s value

declare -i num=10

#Declare variable with integer type

  1. echo $bob
  2. echo ${bob}
  1. ${bob}_a

  1. To access variable’s value
  2. To precisely access variable’s value
  3. To append value to variable
  1. o/p: 103_a  // only possible if you use curly bracket

c=”this is string”

Use “” to assign string value to variable

unset c

Unset the value of variable

${#c}

TO know the length of value(how many character in the value)

o/p: 14

Extraction of the string & count characters of the variables

{a:4:5}

1st character denotes the character you want to start from // always starts with position 0

2nd character denotes the length you want from the 1st character’s output // always starts with position 1 // total length of the value

Commands

Function

a=0987654

${a:4}

Gives output start from 4th character // cut character from 1 to 4

o/p: 654

b=”hello there”

${b:4:5}

Gives output from 4th character and takes only 3 character from that output

o/p: o the

a=123456789

${a: -3}

To list the last 3 characters. // space is required // starts with 1 position

o/p: 789     // {a:6} = {a: -3}

${a: -3:2}

To list length of 1st character’s output

o/p: 78

Variables manipulation

a=”hello there”

b=123456789

Commands

Function

${a#h}

To remove character from the starting point use #.

If you want to remove character that doesn’t there it doesn’t affect the output: ${a#w} // still prints hello there

${a#hell}

${b#123}

o/p: o there.

o/p: 456789

${a#*t}

${b#*7}

Remove everything that starts with something and arrive till t

o/p: here

o/p: 89

${b%789}

If you want to remove something from the end of the string or character use %.  

o/p: 123456

${b%3*}

${a%l*}

o/p: 12

o/p: hel

${a%ere}

o/p: hello th

Permanent Variable inside Login script

You can store permanent variables inside the login script.

You can find this login script in .profile, .bashrc, .login files. And there you can add or change existing variable sets in this file. So you don’t need to set that variable every time you open the terminal.

Read command

Read command takes input and uses a variable to store it

Commands

Function

read a

12345

Read command takes input and uses a variable to store it

o/p:

echo $a   // 12345  

read -p “give me the value of the variable: ” q

Give me the vale of the variable: 12345

You can put the message to input the value

o/p:

echo $q    // 12345

read

PAPA

If you don’t give anything to read command and want to access value.  This variable will always put inside $REPLY

$REPLY // PAPA

Redirect the output

How to redirect output of the command to whatever file we want(and not just to terminal)

  1. How commands send message to the screen

Commands

Function

ls -l 1> fileName

Redirect output to some file name you provide // you don’t need 1 every time.

ls -l 1>> fileName

Append text to the file. So existing file content stays as it is

cat file1 file 2 1> file3

Append content of 2nd file and create a new file

REDIRECT ERROR

Commands

Function

ls -l file1 filejwdnjq 2> file2

To redirect error message to the file

o/p: filejwdnjq is not found  // this output will be directed in the file2  & file 1 will be shown in the terminal

ls -l file1 filejwdnjq &> file2

To redirect both error message and standard message to a file

ls -l file1 filejwdnjq 1> output.txt 2> error.txt

To redirect standards output to one file and error to different file

ls -l file1 filejwdnjq 2>  /dev/null

To redirect error output to null directory.

Why? - bcz some error messages are not important to us to show. In that case we can take out the error message and put it in the another directory  

Pipe command

Commands

Function

ls folder1 | less

Takes first command output and use it as an input of an another command

o/p:

Folder1

File1

Folder2

Tst.txt

(end)

tail -3 file.txt

Outputs last 3 line of that file

ls folder1 | tail -3

Outputs last 3 folder name

o/p:

File1

Folder2

Tst.txt

ls folder1 | tail -3 | less

o/p:

File1

Folder2

Tst.txt

(end)

ls folder1 | tail -3 | sort

Sort output alphabetically

o/p:

Abc.txt

Bcd.txt

cde.txt

ls folder1 | tail -3 | sort > file.txt

Redirect output in some file

Grep command

Commands

Function

Grep some filename.txt

Use grep command To find something inside the file. Basically will output the line that contains “some”

This is case sensitive  

o/p:

There is something

Grep -i some filename.txt

Avoid case-sensitivity on your findings

o/p:

There is something

Son

ls | grep g

Grep with pipe command to list all files that has “g” character in it

ls | grep -v g

It looks everything except line or file contains “g” in it

Brace expansion

Commands

Function

echo {1,2,3}

echo {1,2,3}_file

Brace expansion works with numbers and letters

o/p:

1 2 3

1_file 2_file 3_file

echo example{1..4}

o/p:

Example1 Example2 Example3 Example4

Touch file{1..4}

Mkdir folder{1..4}

Creates file1 file2 file3 file4 files/folders

echo {a,b,c}_{1,2,3}

Create the combination of 2 braces

o/p:

A_1 a_2 a_3 b_1 b_2 b_3 c_1 c_2 c_3

Permissions

Permissions in the terminal

In the pic below every files belongs to gautampambhar is the “USER/OWNER” and staff is the “GROUP”

Files’s relation with permissions

Commands

Function

whoami

Gives you user name of the user(terminal)

Change permission on files

Commands

Function

Chmod u=rw file.txt

Change the file permission to user/owner

Chmod g=rw file.txt

Change the file permission to group

Chmod o=rw file.txt

Change the file permission to “Other” users

Chmod +x file.txt

Give everyone's execute permission (user/group/others)

Chmod -x file.txt

Take out everyone's execute permission (user/group/others)

Section 2

Run Bash script from terminal

#! /bin/bash  =  telling your program that you want to interpret everything you find in this file; read as a bash. = telling the system that this program is bash and run in the bash shell

 

Commands

Function

./filename

To run bash script from the terminal

Conditions

Conditions in Bash scripts

If →  else condition // to check one condition

If  → elif(n) → else condition // to check multiple condition

Condition can be grouped in 3 ways

If → strings

If → numbers

If → files  

If on strings

Commands

Function

If [ “$d” == “hello” ]

Strings equals to hello

If [ -z “$d” ]

If string is null

If [ -n “$d” ]

If the string is not null  // something’s in there. Don’t know what

If [ “$d” != “hello” ]

If string does not equals to specific string

If on Numbers

Commands

Function

If [ “$numb” -eq “12” ]

If number equals to 12

If [ “$numb” -ne “12” ]

If number does not equals to 12

If [ “$numb” -lt “12” ]

If number is lower than 12

If [ “$numb” -gt “12” ]

If number is greater than 12

If [ “$numb” -le “12” ]

If number is lower tor equal to 12

If [ “$numb” -ge “12” ]

If number is greater or equal to 12

If on Files

Commands

Function

If [  -e “$myfile”  ]

If file is exist

If [  -d “$myfile”  ]

If file is exist and it’s a directory

If [  -f  “$myfile”  ]

If file is exist and it’s a regular file // not directory and link

If [  -s “$myfile”  ]

If file is exist and has a length > 0 // non empty file

Logic condition

Commands

Function

If [ $numb -gt 3 -a $numb -lt 9 ]

Checks 2 logic with AND operator

If [ $numb -gt 3 -o $numb -lt 9 ]

Checks 2 logic with OR operator

If [ -s “$myfile” -a -x “$myfile”]

Checks 2 condition with AND : file is non empty and executable

If [[ $numb -gt 3 && $numb -lt 9 ]]

MODERN WAY TO WRITE LOOPS WITH SYMBOLS

AND condition

If [[ $numb -gt 3 || $numb -lt 9 ]]

OR condition

Loops

Commands

Function

For i in {1..3}; do

echo {command}

done

Commands get executed by element of the loop

For i in {1,”hey”,2,”CAT”,”END”}; do

If [ “$i” ==  “CAT” ]; then

Break

fi

done

To stop element getting executed in the list

For (( i = 0; i < 25; i=i+1 )); do

echo “command”

done

Modern way to loop with numbers

Subshell

Commands

Function

exit

To exit the current shell // you’ll lose variables once you exit the shell you are in. you have to open again and assign variables again

Bash

  • Exit

To enter subshell;  

Exit To get out of subshell and you’ll get back to main shell

File create in subshell

Whatever file you create in the subshell will remain in the current shell after you exit from the subshell

Executing subshell script from the current shell

When you write script it becomes subshell when you try to execute from the current shell (by ./somescript). Everything you have inside somescript remains in the subshell. And when you exit from the subshell you can’t access any variable from the current shell

Source command

Commands

Function

Source script.sh

To execute subshell script and retain values of it inside the current shell

. script.sh

Alternative of using source command

Type command

Three types of commands

Commands

Function

Type cat

To know which type of command cat is

o/p: cat is /bin/cat

type cd

cd is a shell builtin

Type lf

Lf is aliased to “ ls -Flha”

Type type

Type is a shell builtin

Alias in Bash

Commands

Function

Alias ll=‘ls -l’

Set alias for command you frequently use/want to use.

F.e; you can set command “ls -l” to whatever name you want to give as an alias  

Alias abc=’ls ../’

Abc is aliased to ls ../

Alias abc

To see the content of the command

o/p: alias abc=’ls ../’

ls; cd; echo “hey”

Use 3 command in one line

Alias cpl=’clear; pwd; ls -l’

Use alias command for which you use command many times

Alias

To see list of alias you set in the shell

Unalias yellow

To unset alias

Command substitution

Commands

Function

Which cd

To know the path of cd command

o/p: /usr/bin/cd

ls -l $(which cd)

To store and use the previous output in the next command

a=$(pwd)

Storing output of the command inside a variable

Date

To see the date of today

d=$(date)

echo $d

To store the date value in the variable named d

o/p:

When you call $d anytime it will give you the initial static value of the date set initially

$d  ne $(date)

$(date) will give you current date and time

Parameters in Bash

Back script

Commands

Function

Source ./script param1

To access subshell directory // see the script for more info

Exit status

Why?

Commands

Function

Pwd

echo $?

It will return the last command’s status. If it executed correctly it will return 0, if not then return 1. Some also returns >1 number as an output  

Functions

Variable scope in functions

CALC script

Commands

Function

echo $(( 3+5 ))  = 8

echo $(( 3**2 ))  = 9

echo $(( 10/3 ))  = 3

Bc

3+8  =  11

To get into another environment type “bc” in the terminal

 

BC Environment

Commands

Function

Bc

10/3  == 3

scale=3

o/p: 3.333

To solve this problem you have to use scale=2 // where 2 is decimal point you want to get result in // 3.33 // set the decimal value

quit

To get out of that environment

Bc -l

Set decimal point to default value // meaning you get result of above example is 3.3333333333  // you can set scale nonetheless

3^3  = 27

3^(-2) = .11

sqrt(16)  = 4

Square root of value

3+ 5^2 -12*sqrt(18) = -18.47

(10<20) || (1<2)

Will return 1. If both condition fails then it will return 0

(10<20) && (10<3)

Will return 0.

Length(123.22)  = 5

How many digit the value has

Scale(123.345) = 3

Count how many decimal number the value has

Bc -q

Open environment without licence // -q stands for quiet. // open your env, do your work quietly

Redirect

Redirect input : here document

Commands

Function

cat

If you type cat without any file, it will wait for your input ; you can close it with ctrl+c ctrl+d. // it will print whatever you write

cat << EOF

Use to change/redirect input of cat command // “<<” stands for here document. // EOF is the symbol that is end of the file // last line will be EOF because program has to know the last line or where the file is going to finish  

<<

Meaning whatever you write; << will redirect tand goes inside the cat command(or whatever command you write)

Bc <<

Use << command with bc

Redirect input : here string

Commands

Function

cat <<< “hello there”

cat will take input from the here string which is “hello there”

s=”het there”

cat <<< $s

Takes the input for set variable

Bc <<< “3*4”

Read w <<< “hey there”

echo $w

q=”3*7 +5”

Bc $q

Parameter SHIFT

Commands

Function

Shift

Shift 4

All the parameter are shifted by 1 // second parameter assigned to $1 in the script

All the parameter are shifted by 4 // 4th parameter assigned to $1 in the script

$0

Will give you absolute path of the current directory where you calling the script

CALC Script

Commands

Function

Calc 3+4

o/p: 7

Calc -p 5 12/7

Use with -p stands for precision. How many decimal points you want with the result.

o/p: 1.71428

Read multiple variables

Grep command

Grep whatYouAreLookingFor nameOfTheFile // grep something abc.txt

Internal field separator

Intro to the IFS

sed = stream editor

Why: helps you extract data in file(ex: database file, data: username, password, age, email)

Why = print the whole file content

sed -n ‘1,3’p test.sh

Printing file lines
p is print

sed “s/gautam/alien/g” test.sh

Replace a value in a file
-i = ignore case sensitive
s = substitute

g = globally substitute

awk “{print}” test.sh

Print the whole file content

awk ‘/gautam/{print}’ test.sh

Print lines with “gautam” word

 

curl = command in a shell script, which will hit the target endpoint and get the JSON data that that endpoint has

Sed

  1. Replace text in echo with other names
  1. Answer: echo “hi gautam” | sed s/gautam/John/g
  1. Replace multiple words(I, Me, me, My, my) in a file with Gautam using Sed
  1. You’ll need 2 files
  1. One with the content (takeonme.txt)
  2. One sed file(with sed Extention = gautam.sed) with noting the value you want to replace with
  1. Answer: cat takeonme.txt | sed -f gautam.sed // this will replace any value(I, Me, me, My, my) with Gautam
  1. -f stands for open a script as a file

REGEX(regular expression) allows you to define the pattern in text using special or complicated syntax

  1. use regex to print out every line that doesn't start with "Take"
  1. Answer: cat takeonme.txt | sed /^Take/d

Practice 

Delimiter: commonly used /.

sed ‘s/I/Gautam/g’ file.txt

sed -i ‘s/I/Gautam/g’ file.txt

sed -i ‘.bak’ ‘s/I/Gautam/g’ file.txt

Globally replace I a word with Gautam

To overwrite file

To overwrite file in MAC

sed ‘s/I/Gautam’ file.txt

Replace first I in line with Gautam

sed ‘s/I/Gautam/2’ file.txt

Replace second I in line with Gautam

AWK

  1. print all the lines that are longer than 20 characters.
  1. Answer: awk ' length($0) > 20 ' takeonme.txt
  1. List all the lines that are longer than 20 characters using awk and remove lines started with Take using sed
  1. Answer: awk ' length($0) > 20 ' takeonme.txt | sed/^Take/d

Practice 

Delimiter: spaces are delimiter by default

awk ‘{print}’ file.txt

To print the file content. This is same as using cat command

awk ’{print $1}’ file.txt

To prints the first field of every line; increment number to print the line field ahead

awk ‘{print $1,$3}’ file.txt

To print in combination of 1st and 3rd filed of every line

awk ‘{print $NF}’

Prints the last field with a caveat

  • NF = number of field

awk -F’:’ ‘print $1,$7,’ /etc/passwd

meaningful usage when using combined

-this will shows what shell your users are using