Advanced Linux for DevOps Engineer
In this article, we will do some hands-on on advanced Linux, in my previous article I have shown the fundamentals of Linux along with examples.
Users and Groups
File Permission
Search (grep, awk, find)
ACL (Access Control List)
SSH (Secure Shell)
SCP (Secure Copy)
Users and Groups
In this section, we will see, how to create a user and group, how to add users into a group, how to set passwords to a user and then how to check the status of users and groups.
#creating user...users are stored under home directory.
ubuntu@ip-172-31-7-130:/home$ sudo useradd test-user -m
ubuntu@ip-172-31-7-130:/home$ sudo useradd devops-user -m
ubuntu@ip-172-31-7-130:/home$ ls
devops-user test-user ubuntu
ubuntu@ip-172-31-7-130:/home$
# -m is use to make the user to function as a directory.
# Creating Group...
ubuntu@ip-172-31-7-130:/home$ sudo groupadd DevOpsteam
# to check the group status, type the below command and hit enter.
ubuntu@ip-172-31-7-130:/home$ sudo cat /etc/group
# Here we can see all the users are also showing in group section, so when we create any user in ubuntu so by default it create a group id with same name automatically.
mohansahani:x:1001:
test-user:x:1002:
devops-user:x:1003:
DevOpsteam:x:1004
# Password Set..
ubuntu@ip-172-31-7-130:/home$ sudo passwd test-user
New password: # here enter the password which you want to set.
Retype new password:
passwd: password updated successfully
ubuntu@ip-172-31-7-130:/home$
#Adding users into a group
ubuntu@ip-172-31-7-130:/home$ sudo gpasswd -a test-user DevOpsteam
Adding user test-user to group DevOpsteam
ubuntu@ip-172-31-7-130:/home$ sudo cat /etc/group
DevOpsteam:x:1004:test-user # So here we can see that test-user is added into DevOpsteam group ID.
#Add multiple user in group at a time
ubuntu@ip-172-31-7-130:/home$ sudo gpasswd -M test-user,ubuntu,devops-user DevOpsteam
ubuntu@ip-172-31-7-130:/home$ sudo cat /etc/group
DevOpsteam:x:1004:test-user,ubuntu,devops-user
#Switch user..
ubuntu@ip-172-31-7-130:/home$ su test-user
Password:
$ ls
devops-user test-user ubuntu
$ pwd
/home
$ cd test-user
$ pwd
/home/test-user
$ exit # to come out of the user
ubuntu@ip-172-31-7-130:/home$
File Permission
File permission is one of the important part of any operating system.
In Linux we have special feature for file permissions.
#Lets check the permission of one file as shown in below command.
ubuntu@ip-172-31-7-130:~$ ls -la demo-file.txt
-rw-r--r-- 1 root root 20 Apr 6 19:54 demo-file.txt
ubuntu@ip-172-31-7-130:~$
We have three section for permission.
User(---), Group(---),Other(---)
rwx = Read, Write, Execute
So as per above file, this file has read and write permission to user, read only permmission to group and read only permission to other.
#Lets change the permission of a file, We have numeric value to change the file permission.
r (Read)-4
w (Write)-2
x (Execute)-1
ubuntu@ip-172-31-7-130:~$ sudo chmod 777 demo-file.txt
ubuntu@ip-172-31-7-130:~$ ls -la demo-file.txt
-rwxrwxrwx 1 root root 20 Apr 6 19:54 demo-file.txt
ubuntu@ip-172-31-7-130:~$
Be careful while changing the file permission, as per above example now this file is accesible to everyone.
Search (grep, find, awk )
grep
When we need to search any particular file with a name in any location then we can use the grep command.
ubuntu@ip-172-31-7-130:~$ sudo grep -r devops /home/ubuntu/
/home/ubuntu/.bash_history:rm -rf devops-batch-3/
/home/ubuntu/.bash_history:rm -rf devops-batch-3
/home/ubuntu/.bash_history:sudo rm -rf devops-batch-3/
/home/ubuntu/.bash_history:rm -rf mohan-devops yash-tester/
/home/ubuntu/.bash_history:sudo rm -rf mohan-devops yash-tester
/home/ubuntu/.bash_history:sudo userdel -rf LinuxMachine mohan-devops yash-tester
/home/ubuntu/.bash_history:sudo userdel -r LinuxMachine mohan-devops yash-tester
/home/ubuntu/.bash_history:sudo userdel -r mohan-devops
/home/ubuntu/.bash_history:sudo delgroup devops
/home/ubuntu/.bash_history:sudo useradd devops-user -m
/home/ubuntu/.bash_history:sudo gpasswd -M test-user ubuntu devops-user DevOpsteam
/home/ubuntu/.bash_history:sudo gpasswd -M test-user,ubuntu,devops-user DevOpsteam
ubuntu@ip-172-31-7-130:~$
#-r is only requied when we use grep for directory
# Lets say you want to search any file with content and want to stored it into any file then follow the below comamnd.
ubuntu@ip-172-31-7-130:~$ cat demo-file.txt
this is from server
ubuntu@ip-172-31-7-130:~$ sudo grep server demo-file.txt > server.txt
ubuntu@ip-172-31-7-130:~$ cat server.txt
this is from server
ubuntu@ip-172-31-7-130:~$
#if you want to search any data with case insesative then you may use -i with above command.
Find
ubuntu@ip-172-31-7-130:~$ find /home/ubuntu log
/home/ubuntu
/home/ubuntu/.bash_history
/home/ubuntu/.bashrc
/home/ubuntu/.profile
/home/ubuntu/.viminfo
/home/ubuntu/Git_Machine-Key.pem
/home/ubuntu/.sudo_as_admin_successful
/home/ubuntu/server.txt
/home/ubuntu/demo-file.txt
/home/ubuntu/.ssh
/home/ubuntu/.ssh/authorized_keys
/home/ubuntu/.bash_logout
/home/ubuntu/.cache
/home/ubuntu/.cache/motd.legal-displayed
find: ‘log’: No such file or directory
ubuntu@ip-172-31-7-130:~$ find
#To find the file type only the follow the below command.
ubuntu@ip-172-31-7-130:~$ find /home/ubuntu -type f
/home/ubuntu/.bash_history
/home/ubuntu/.bashrc
/home/ubuntu/.profile
/home/ubuntu/.viminfo
/home/ubuntu/Git_Machine-Key.pem
/home/ubuntu/.sudo_as_admin_successful
/home/ubuntu/server.txt
/home/ubuntu/demo-file.txt
/home/ubuntu/.ssh/authorized_keys
/home/ubuntu/.bash_logout
/home/ubuntu/.cache/motd.legal-displayed
#If you want to search directory type then follow the below command.
ubuntu@ip-172-31-7-130:~$ find /home/ubuntu -type d
/home/ubuntu
/home/ubuntu/.ssh
/home/ubuntu/.cache
ubuntu@ip-172-31-7-130:~$
#If you want to filter out more then such file/directory type along with file/directory name then follow the below command.
ubuntu@ip-172-31-7-130:~$ find /home/ubuntu -type f -name Git_Machine-Key.pem
/home/ubuntu/Git_Machine-Key.pem
ubuntu@ip-172-31-7-130:~$
awk
awk command is also work as a grep and find but it has some programatic featur to search data which makes it cool.we can write conditional command in awk to search data. Lets have an example here.
#1- Lets say if you have a long sheet type and you want to print particular rows/column, then follow the below command.
ubuntu@ip-172-31-7-130:~$ sudo awk '/INFO/ {print $1}' demo-file.txt
03/22
03/22
03/22
03/22
03/22
03/22
03/22
03/22
03/22
03/22
03/22
03/22
03/22
03/22
03/22
03/22
03/22
03/22
03/22
03/22
03/22
this will print the first column because we have used $1 to print the first column, likewise if we want to print multiple column then follow the below command.
ubuntu@ip-172-31-7-130:~$ sudo awk '/INFO/ {print $1,$2,$3}' demo-file.txt
03/22 08:51:01 INFO
03/22 08:51:01 INFO
03/22 08:51:01 INFO
03/22 08:51:01 INFO
03/22 08:51:01 INFO
03/22 08:51:02 INFO
03/22 08:51:02 INFO
03/22 08:51:02 INFO
03/22 08:51:06 INFO
03/22 08:51:06 INFO
#If we want to print the data within the range then follow the below command.
ubuntu@ip-172-31-7-130:~$ awk 'NR>=20 && NR<=50 && /INFO/ {print NR,$1,$4}' demo-file.txt
20 03/22 :...read_physical_netif:
21 03/22 :....mailslot_create:
22 03/22 :...mailbox_register:
24 03/22 :.....mailslot_create:
25 03/22 :....mailbox_register:
26 03/22 :.....mailslot_create:
29 03/22 :....mailbox_register:
31 03/22 :.....mailslot_create:
32 03/22 :....mailbox_register:
33 03/22 :.....mailslot_create:
35 03/22 :....mailbox_register:
38 03/22 :.....mailslot_create:
39 03/22 :....mailbox_register:
40 03/22 :.....mailslot_create:
42 03/22 :....mailbox_register:
44 03/22 :.....mailslot_create:
45 03/22 :....mailbox_register:
46 03/22 :.....mailslot_create:
48 03/22 :....mailbox_register:
50 03/22 :.....mailslot_create:
ubuntu@ip-172-31-7-130:~$
ACL- Access Control List
ACL is used to check the access given to any file or directory just like ls -la, to use ACL, first install this in your system.
ubuntu@ip-172-31-7-130:~$ sudo apt-get install acl
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
acl is already the newest version (2.3.1-1).
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
ubuntu@ip-172-31-7-130:~$
ubuntu@ip-172-31-7-130:~$ getfacl demo-file.txt
# file: demo-file.txt
# owner: root
# group: root
user::rwx
group::rwx
other::rwx
ubuntu@ip-172-31-7-130:~$
#to change the access of any files please follow the below command.
ubuntu@ip-172-31-7-130:~$ sudo setfacl -m u:ubuntu:rwx demo-file.txt
ubuntu@ip-172-31-7-130:~$ getfacl demo-file.txt
# file: demo-file.txt
# owner: root
# group: root
user::rwx
user:ubuntu:rwx
group::rwx
mask::rwx
other::rwx
ubuntu@ip-172-31-7-130:~$
ssh- secure shell
SSH is one of the secure methods to connect to your instances, here we will see how we can connect to our instance with the help of the ssh command. Let's say you have launched our EC2 instance and you want to connect to it through the terminal.
So as per the below snap, my EC2 instance is launched and running, and now I will connect to it through the ssh command.
1- Open you gitbash terminal and switch to the directory where you have kept your keypair (.pem) file.
Mohan@LAPTOP-AV55HEFJ MINGW64 ~
$ cd /d/DevopsByBhupiSir/
Mohan@LAPTOP-AV55HEFJ MINGW64 /d/DevopsByBhupiSir
$ ssh -i KubernetesKey.pem ubuntu@3.110.156.23
The authenticity of host '3.110.156.23 (3.110.156.23)' can't be established.
ED25519 key fingerprint is SHA256:vbb66UAN4ri979zXYHC7C+Db9Ir/3yC1zTBLm6qMX9U.
This host key is known by the following other names/addresses:
~/.ssh/known_hosts:197: 43.204.216.155
~/.ssh/known_hosts:200: 13.233.131.131
~/.ssh/known_hosts:201: 65.0.108.230
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '3.110.156.23' (ED25519) to the list of known hosts.
Welcome to Ubuntu 22.04.2 LTS (GNU/Linux 5.19.0-1022-aws x86_64)
Last login: Sat Apr 8 20:56:23 2023 from 13.233.177.4
ubuntu@ip-172-31-7-130:~$ ls
Git_Machine-Key.pem demo-file.txt server.txt
ubuntu@ip-172-31-7-130:~$
#So this is how we can use ssh command to connect to our instance.
SCP- Secure Copy
Let's understand how we can copy files from local to server and server to local with the help of SCP command.
# Copying files from Local to Server
Mohan@LAPTOP-AV55HEFJ MINGW64 /d/DevopsByBhupiSir
$ touch LocaltoServerFile.txt (I have created one file in my local machine)
Mohan@LAPTOP-AV55HEFJ MINGW64 /d/DevopsByBhupiSir
u scp -i KubernetesKey.pem LocaltoServerFile.txt ubuntu@3.110.156.23:/home/ubunt
LocaltoServerFile.txt 100% 0 0.0KB/s 00:00
Mohan@LAPTOP-AV55HEFJ MINGW64 /d/DevopsByBhupiSir
$
#Now the files have been copied, now check at server machine.
ubuntu@ip-172-31-7-130:~$ ls /home/ubuntu/
Git_Machine-Key.pem LocaltoServerFile.txt demo-file.txt server.txt
ubuntu@ip-172-31-7-130:~$
#here we can see that the file LocaltoServerFile.txt has been copied.
#Now lets try from server to local.
ubuntu@ip-172-31-7-130:~$ touch ServerToLocalFile.txt
ubuntu@ip-172-31-7-130:~$ ls
Git_Machine-Key.pem LocaltoServerFile.txt ServerToLocalFile.txt demo-file.txt server.txt
ubuntu@ip-172-31-7-130:~$
#I have created one file ServerToLocalFile.txt in server and same will be copied to local machine.
Mohan@LAPTOP-AV55HEFJ MINGW64 /d/DevopsByBhupiSir
$ scp -i KubernetesKey.pem ubuntu@3.110.156.23:/home/ubuntu/ServerToLocalFile.txt .
Mohan@LAPTOP-AV55HEFJ MINGW64 /d/DevopsByBhupiSir
$ ls
Chef/ apache-ant-1.9.16-bin/ jdk-11.0.16.1_windows-x64_bin.exe*
Git_Machine-Key.pem apache-ant-1.9.16-bin.zip jenkins.msi
KubernetesKey.pem apache-maven-3.8.7-bin.zip jenkins.war
LocaltoServerFile.txt apache-tomcat-9.0.71.exe* terraform/
ServerToLocalFile.txt chef-starter/
'TG devops bhupinder sir notes -.pdf' demo-file.txt
#(.) Single dot is required when you want to copy files to the current directory.
Thank you friends for your time, please follow me on #hashnode to get more updates on DevOps technology