1. Linux History

Lesson Content

Hey rookie! So you decided to dive into this wonderful world known as Linux? Well you better strap in, because it’s gonna be a long and hard road. My name is Penguin Pete and I’m here to guide you through this journey. Let’s get started with a little bit of backstory about Linux.

To learn about how Linux came to be, let’s go back to the beginning to 1969 where Ken Thompson and Dennis Ritchie of Bell Laboratories developed the UNIX operating system. It was later rewritten in C to make it more portable and eventually became a widely used operating system.

A decade or so later, Richard Stallman started working on the GNU (GNU is Not UNIX) project, the GNU kernel called Hurd, which unfortunately never came to completion. The GNU General Public License (GPL), a free software license, was also created as a result of this.

The kernel is the most important piece in the operating system. It allows the hardware to talk to the software. It also does a whole lot of other things, but we’ll dig into that in a different course. For now, just know that the kernel controls pretty much everything that happens on your system.

During this time other efforts such as BSD, MINIX, etc were developed to be UNIX like-systems. However, one thing that all these UNIX like-systems had in common was the lack of a unified kernel.

Then in 1991, a young fellow named Linus Torvalds started developing what we now know today as the Linux kernel.

Exercise

Additional reading:

• GNU
• Ken Thompson
• Richard Stallman
• Linus Torvalds

Quiz Question

2. Choosing a Linux Distribution

Lesson Content

In the previous lesson, we learned about the Linux kernel which powers millions of devices a day. One thing before we move forward, the term Linux is actually quite a misnomer, since it actually refers to the Linux kernel. However, many distributions use the Linux kernel so therefore are commonly known as Linux operating systems.

A Linux system is divided into three main parts:

• Hardware - This includes all the hardware that your system runs on as well as memory, CPU, disks, etc.
• Linux Kernel - As we discussed above, the kernel is the core of the operating system. It manages the hardware and tells it how to interact with the system.
• User Space - This is where users like yourself will be directly interacting with the system.

So the first step we’ll need to take is to install Linux on your machine. You have many options to choose from and this course will help inform you and get you started on choosing a Linux distribution.

There are many Linux distributions to choose from, we’ll just go over the most popular options.

Exercise

No exercises for this lesson.

Quiz Question

No questions, skip ahead!

Debian

Lesson Content

Overview Debian is an operating system composed entirely of free and open-source software. It’s widely known and has been in development for over 20 years. There are three branches that you can use, Stable, Testing and Unstable.

Stable is an overall good branch to be on. Testing and Unstable are rolling releases. This means that any incremental changes in those branches will eventually become Stable. For example, if you wanted to get to the next update from Windows 8 to Windows 10, you’ll have to do a complete Windows 10 installation. However being on the Testing release, you’ll automatically get updates until it becomes the next operating system release without having to do a full installation.

Package Management Debian also uses Debian package management tools. Every Linux distribution installs and manages packages differently and they use different package management tools. We’ll get more into this in a later course.

Configurability Debian may not get the latest updates, but it’s extremely stable. If you want a good “core” operating system, this is the one for you.

Uses Debian is an overall great operating system for any platform.

Exercise

If you’re interested in having Debian as your operating system, head over to the installation section and give it a try: https://www.debian.org/

Quiz Question

Know More about Rolling and Point Release.

Red Hat Enterprise Linux (RHEL)

Lesson Content

Overview Red Hat Enterprise Linux commonly referred to as RHEL is developed by Red Hat. RHEL has strict rules to restrict free re-distribution although it still provides source code for free.

Package Management RHEL uses a different package manager than Debian, RPM package manager, which we will eventually learn about as well.

Configurability RHEL-based operating systems will differ slightly from the Debian-based operating systems, most noticeably in package management. If you decide to go with RHEL it’s probably best if you know you’ll be working with it.

Uses As described by the name it’s mostly used in enterprise, so if you need a solid server OS this would be a good one.

Exercise

If you’re interested in having RHEL as your operating system, head over to the installation section and give it a try: https://www.redhat.com/rhel/

Quiz Questions

Ubuntu

Lesson Content

Overview One of the most popular Linux distributions for personal machines is Ubuntu. Ubuntu also releases its own desktop environment manager Unity by default.

Package Management Ubuntu is a Debian-based operating system developed by Canonical. So it uses a core Debian package management system.

Configurability Ubuntu is a great choice for a beginner who wants to get into Linux. Ubuntu offers ease of use and a great user interface experience that has led to its wide adoption. It’s widely used and supported and is most like other operating systems like OSX and Windows in terms of usability.

Uses Great for any platform, desktop, laptop and server.

Exercise

If you’re interested in having Ubuntu as your operating system, head over to the installation section and give it a try: http://www.ubuntu.com/

Quiz Questions

Fedora

Lesson Content

Overview Backed by Red Hat, the Fedora Project is community driven containing open-source and free software. Red Hat Enterprise Linux branches off Fedora, so think of Fedora as an upstream RHEL operating system. Eventually RHEL will get updates from Fedora after thorough testing and quality assurance. Think of Fedora as an Ubuntu equivalent that uses a Red Hat backend instead of Debian.

Package Management Uses Red Hat package manager.

Configurability If you want to use a Red Hat based operating system, this is a user friendly version.

Uses Fedora is great if you want a Red Hat based operating system without the price tag. Recommended for desktop and laptop.

Exercise

If you’re interested in having Fedora as your operating system, head over to the installation section and give it a try: https://getfedora.org/

Quiz Questions

Linux Mint

Lesson Content

Overview Linux Mint is based off of Ubuntu. It uses Ubuntu’s software repositories so the same packages are available on both distributions. Linux Mint is preferred by others over Ubuntu because it doesn’t come with some of the proprietary software that Ubuntu includes such as Unity.

Package Management Since Linux Mint is Ubuntu based, it uses the Debian package manager.

Configurability Great user interface, great for beginners and less bloated than Ubuntu. In this course, I’ll be using Linux Mint, but any other distribution can be used.

Uses Great for desktop and laptop.

Exercise

If you’re interested in having Linux Mint as your operating system, head over to the installation section and give it a try: http://linuxmint.com/

Quiz Questions

Gentoo

Lesson Content

Overview Gentoo offers ridiculous flexibility with the operating system at a price. It’s made for advanced users who don’t mind getting their hands dirty with the system.

Package Management Gentoo uses its own package management, Portage. The Portage package management is very modular and easy to maintain, which plays a big part in the operating system as a whole being very flexible.

Configurability If you’re just getting started with Linux and want to take a more difficult path, I’d choose Gentoo or Arch Linux as your distribution.

Uses Great for desktop and laptop.

Exercise

If you’re interested in having Gentoo as your operating system, head over to the installation section and give it a try: https://www.gentoo.org/

Quiz Questions

Arch

Lesson Content

Overview Arch is a lightweight and flexible Linux distribution driven 100% by the community. Similar to Debian, Arch uses a rolling release model so incremental updates eventually become the Stable release. You really need to get your hands dirty to understand the system and its functions, but in turn you get complete and total control of your system.

Package Management It uses its own package manager, Pacman, to install, update and manage packages.

Configurability If you want a lightweight operating system and really want to understand Linux use Arch! There’s a bit of a learning curve, but for the hardcore Linux users, this is a great choice.

Uses Great for desktop and laptop. If you also have a small device such as a Raspberry Pi and need to stick a lightweight OS on it, you can’t go wrong with Arch.

Exercise

If you’re interested in having Arch as your operating system, head over to the installation section and give it a try: https://www.archlinux.org/

Quiz Questions

OpenSUSE

Lesson Content

Overview openSUSE Linux is created by the openSUSE Project. A community that promotes the use of Linux everywhere, working together in an open, transparent and friendly manner as part of the worldwide Free and Open Source Software community. openSUSE is the second oldest still running Linux Distributions and shares the base system with SUSE’s award-winning SUSE Linux Enterprise products.

Package Management Uses RPM package manager.

Configurability openSUSE is a great choice for a new Linux user. It offers an easy to use graphical installer/administration application (YaST) and a tiday base system, easy to tinker with. openSUSE includes everything you need to enjoy the Internet worry free of viruses/spy-ware and to live out your creativity, be it with your photos, videos, music or code.

Uses openSUSE Leap is fully capable of being used on a desktop PC and laptop.

Exercise

If you’re interested in having openSUSE as your operating system, head over to the download page and give it a try: https://www.opensuse.org/

Quiz Questions

1. Shell

Lesson Content

The world is your oyster, or really the shell is your oyster. What is the shell? The shell is basically a program that takes your commands from the keyboard and sends them to the operating system to perform. If you’ve ever used a GUI, you’ve probably seen programs such as “Terminal” or “Console” these are just programs that launch a shell for you. Throughout this entire course we will be learning about the wonders of the shell.

In this course we will use the shell program bash (Bourne Again shell), almost all Linux distributions will default to the bash shell. There are other shells available such as ksh, zsh, tsch, but we won’t get into any of those.

Let’s jump right in! Depending on the distribution your shell prompt might change, but for the most part it should adhere to the following format:

username@hostname:current_directory
pete@icebox:/home/pete $

Notice the $ at the end of the prompt? Different shells will have different prompts, in our case the $ is for a normal user using Bash, Bourne or Korn shell, you don’t add the prompt symbol when you type the command, just know that it’s there.

Let’s start with a simple command, echo. The echo command just prints out the text arguments to the display.

$ echo Hello World

Exercise

Try some other Linux commands and see what they output:

$ date
$ whoami

Quiz Question

2. pwd (Print Working Directory)

Lesson Content

Everything in Linux is a file, as you journey deeper into Linux you’ll understand this, but for now just keep that in mind. Every file is organized in a hierarchical directory tree. The first directory in the filesystem is aptly named the root directory. The root directory has many folders and files which you can store more folders and files, etc. Here is an example of what the directory tree looks like:

/
|-- bin
|   |-- file1
|   |-- file2
|-- etc
|   |-- file3
|   `-- directory1
|       |-- file4
|       `-- file5
|-- home
|-- var

The location of these files and directories are referred to as paths. If you had a folder named home with a folder inside of it named pete and another folder in that folder called Movies, that path would look like this: /home/pete/Movies, pretty simple huh?

Navigation of the filesystem, much like real life is helpful if you know where you are and where you are going. To see where you are, you can use the pwd command, this command means “print working directory” and it just shows you which directory you are in, note the path stems from the root directory.

$ pwd

Where are you? Where am I? Give it a try.

Exercise

No exercises for this lesson.

Quiz Question

3. cd (Change Directory)

Lesson Content

Now that you know where you are, let’s see if we can move around the filesystem a bit. Remember we’ll need to navigate our way using paths. There are two different ways to specify a path, with absolute and relative paths.

• Absolute path: This is the path from the root directory. The root is the head honcho. The root directory is commonly shown as a slash. Every time your path starts with / it means you are starting from the root directory. For example, /home/pete/Desktop.

• Relative path: This is the path from where you are currently in filesystem. If I was in location /home/pete/Documents and wanted to get to a directory inside Documents called taxes, I don’t have to specify the whole path from root like /home/pete/Documents/taxes, I can just go to taxes/ instead.

Now that you know how paths work, we just need something to help us change to the directory we want to. Luckily, we have cd or “change directory” to do that.

$ cd /home/pete/Pictures 

So now I’ve changed my directory location to /home/pete/Pictures.

Now from this directory I have a folder inside called Hawaii, I can navigate to that folder with:

$ cd Hawaii 

Notice how I just used the name of the folder? It’s because I was already in /home/pete/Pictures.

It can get pretty tiring navigating with absolute and relative paths all the time, luckily there are some shortcuts to help you out.

• . (current directory). This is the directory you are currently in.
• .. (previous directory). Takes you to the directory above your current.
• ~ (home directory). This directory defaults to your “home directory”. Such as /home/pete.
• - (previous directory). This will take you to the previous directory you were just at.

$ cd .
$ cd ..
$ cd ~
$ cd -

Give them a try!

Exercise

• Run the cd command without any flags, where does it take you?

Quiz Question

4. ls (List directory)

Lesson Content

Now that we know how to move around the system, how do we figure out what is available to us? Right now it’s like we are moving around in the dark. Well, we can use the wonderful ls command to list directory contents. The ls command will list directories and files in the current directory by default, however you can specify which path you want to list the directories of.

$ ls
$ ls /home/pete

ls is a quite useful tool, it also shows you detailed information about the files and directories you are looking at.

Also note that not all files in a directory will be visible. Filenames that start with . are hidden, you can view them however with the ls command and pass the -a flag to it (a for all).

$ ls -a 

There is also one more useful ls flag, -l for long, this shows a detailed list of files in a long format. This will show you detailed information, starting from the left: file permissions, number of links, owner name, owner group, file size, timestamp of last modification, and file/directory name.

$ ls -l 

pete@icebox:~$ ls -l

total 80
drwxr-x--- 7 pete penguingroup   4096 Nov 20 16:37 Desktop
drwxr-x--- 2 pete penguingroup   4096 Oct 19 10:46  Documents
drwxr-x--- 4 pete penguingroup   4096 Nov 20 09:30 Downloads
drwxr-x--- 2 pete penguingroup   4096 Oct  7 13:13   Music
drwxr-x--- 2 pete penguingroup   4096 Sep 21 14:02 Pictures
drwxr-x--- 2 pete penguingroup   4096 Jul 27 12:41   Public
drwxr-x--- 2 pete penguingroup   4096 Jul 27 12:41   Templates
drwxr-x--- 2 pete penguingroup   4096 Jul 27 12:41   Videos 

Commands have things called flags (or arguments or options, whatever you want to call it) to add more functionality. See how we added -a and -l, well you can add them both together with -la. The order of the flags determines which order it goes in, most of the time this doesn’t really matter so you can also do ls -al and it would still work.

$ ls -la 

Exercise

Run ls with different flags and see the output you receive.

Quiz Question

5. touch

Lesson Content

Let’s learn how to make some files. A very simple way is to use the touch command. Touch allows you to the create new empty files.

$ touch mysuperduperfile 

And boom, new file!

Touch is also used to change timestamps on existing files and directories. Give it a try, do an ls -l on a file and note the timestamp, then touch that file and it will update the timestamp.

There are many other ways to create files that involve other things like redirection and text editors, but we’ll get to that in the Text Manipulation course.

Exercise

• Create a new file
• Note the timestamp
• Touch the file and check the timestamp once again

Quiz Question

6. file

Lesson Content

In the previous lesson we learned about touch, let’s go back to that for a bit. Did you notice that the filename didn’t conform to standard naming like you’ve probably seen with other operating systems like Windows? Normally you would expect a file called banana.jpeg and expect a JPEG picture file.

In Linux, filenames aren’t required to represent the contents of the file. You can create a file called funny.gif that isn’t actually a GIF.

To find out what kind of file a file is, you can use the file command. It will show you a description of the file’s contents.

$ file banana.jpg 

Exercise

Run the file command on a few different directories and files and note the output.

Quiz Question

7. cat (Concatenate)

Lesson Content

We’re almost done navigating files, but first let’s learn how to read a file. A simple command to use is the cat command, short for concatenate, it not only displays file contents but it can combine multiple files and show you the output of them.

$ cat dogfile birdfile

It’s not great for viewing large files and it’s only meant for short content. There are many other tools that we use to view larger text files that we’ll discuss in the next lesson.

Exercise

Run cat on different files and directories. Then try to cat multiple files.

Quiz Question

8. less

Lesson Content

If you are viewing text files larger than a simple output, less is more. (There is actually a command called more that does something similar, so this is ironic.) The text is displayed in a paged manner, so you can navigate through a text file page by page.

Go ahead and look at the contents of a file with less. Once you’re in the less command, you can actually use other keyboard commands to navigate in the file.

$ less /home/pete/Documents/text1

Use the following command to navigate through less:

• q - Used to quit out of less and go back to your shell.
• Page up, Page down, Up and Down - Navigate using the arrow keys and page keys.
• g - Moves to beginning of the text file.
• G - Moves to the end of the text file.
• /search - You can search for specific text inside the text document. Prefacing the words you want to search with /
• h - If you need a little help about how to use less while you’re in less, use help.

Exercise

Run less on a file, then page up and around the file. Try searching for a specific word. Quickly navigate to the beginning or the end of the file.

Quiz Question

9. history

Lesson Content

In your shell, there is a history of the commands that you previously entered, you can actually look through these commands. This is quite useful when you want to find and run a command you used previously without actually typing it again.

$ history

Want to run the same command you did before, just hit the up arrow.

Want to run the previous command without typing it again? Use !!. If you typed cat file1 and want to run it again, you can actually just go !! and it will run the last command you ran.

Another history shortcut is ctrl-R, this is the reverse search command, if you hit ctrl-R and you start typing parts of the command you want it will show you matches and you can just navigate through them by hitting the ctrl-R key again. Once you found the command you want to use again, just hit the Enter key.

Our terminal is getting a little cluttered no? Let’s do a little cleanup, use the clear command to clear up your display.

$ clear

There that looks better doesn’t it?

While we are talking about useful things, one of the most useful features in any command-line environment is tab completion. If you start typing the beginning of a command, file, directory, etc and hit the Tab key, it will autocomplete based on what it finds in the directory you are searching as long as you don’t have any other files that start with those letters. For example if you were trying to run the command chrome, you can type chr and press Tab and it will autocomplete chrome.

Exercise

Navigate through your previous command history with the Up and Down keys. Play around with ctrl-R reverse search.

Quiz Question

10. cp (Copy)

Lesson Content

Let’s start making some copies of these files. Much like copy and pasting files in other operating systems, the shell gives us an even simpler way of doing that.

$ cp mycoolfile /home/pete/Documents/cooldocs

mycoolfile is the file you want to copy and /home/pete/Documents/cooldocs is where you are copying the file to.

You can copy multiple files and directories as well as use wildcards. A wildcard is a character that can be substituted for a pattern based selection, giving you more flexibility with searches. You can use wildcards in every command for more flexibility.

• * the wildcard of wildcards, it’s used to represent all single characters or any string.
• ? used to represent one character
• [] used to represent any character within the brackets

$ cp *.jpg /home/pete/Pictures

This will copy all files with the .jpg extension in your current directory to the Pictures directory.

A useful command is to use the -r flag, this will recursively copy the files and directories within a directory.

Try to do a cp on a directory that contains a couple of files to your Documents directory. Didn’t work did it? Well that’s because you’ll need to copy over the files and directories inside as well with -r command.

$ cp -r Pumpkin/ /home/pete/Documents

One thing to note, if you copy a file over to a directory that has the same filename, the file will be overwritten with whatever you are copying over. This is no bueno if you have a file that you don’t want to get accidentally overwritten. You can use the -i flag (interactive) to prompt you before overwriting a file.

$ cp -i mycoolfile /home/pete/Pictures

Exercise

Copy over a couple of files, be careful not to overwrite anything important.

Quiz Question

11. mv (Move)

Lesson Content

Used for moving files and also renaming them. Quite similar to the cp command in terms of flags and functionality.

You can rename files like this:

$ mv oldfile newfile

Or you can actually move a file to a different directory:

$ mv file2 /home/pete/Documents

And move more than one file:

$ mv file_1 file_2 /somedirectory

You can rename directories as well:

$ mv directory1 directory2

Like cp, if you mv a file or directory it will overwrite anything in the same directory. So you can use the -i flag to prompt you before overwriting anything.

mv -i directory1 directory2

Let’s say you did want to mv a file to overwrite the previous one. You can also make a backup of that file and it will just rename the old version with a ~.

$ mv -b directory1 directory2

Exercise

Rename a file, then move that file to a different directory.

Quiz Question

12. mkdir (Make Directory)

Lesson Content

We’re gonna need some directories to store all these files we’ve been working on. The mkdir command (Make Directory) is useful for that, it will create a directory if it doesn’t already exist. You can even make multiple directories at the same time.

$ mkdir books paintings

You can also create subdirectories at the same time with the -p (parent flag).

$ mkdir -p books/hemmingway/favorites

Exercise

Make a couple of directories and move some files into that directory.

Quiz Question

13. rm (Remove)

Lesson Content

Now I think we have too many files, let’s remove some files. To remove files you can use the rm command. The rm (remove) command is used to delete files and directories.

$ rm file1

Take caution when using rm, there is no magical trash can that you can fish out removed files. Once they are gone, they are gone for good, so be careful.

Fortunately there are some safety measures put into place, so the average joe can’t just remove a bunch of important files. Write-protected files will prompt you for confirmation before deleting them. If a directory is write-protected it will also not be easily removed.

Now if you don’t care about any of that, you can absolutely remove a bunch of files.

$ rm -f file1

-f or force option tells rm to remove all files, whether they are write protected or not, without prompting the user (as long as you have the appropriate permissions).

$ rm -i file

Adding the -i flag like many of the other commands, will give you a prompt on whether you want to actually remove the files or directories.

$ rm -r directory

You can’t just rm a directory by default, you’ll need to add the -r flag (recursive) to remove all the files and any subdirectories it may have.

You can remove a directory with the rmdir command.

$ rmdir directory

Exercise

• Create a file called -file (don’t forget the dash!).
• Remove that file.

Quiz Question

14. find

Lesson Content

With all these files we have on the system it can get a little hectic trying to find a specific one. Well there’s a command we can use for that, find!

$ find /home -name puppies.jpg

With find you’ll have to specify the directory you’ll be searching it, what you’re searching for, in this case we are trying to find a file by the name of puppies.jpg.

You can specify what type of file you are trying to find.

$ find /home -type d -name MyFolder

You can see that I set the type of file I’m trying to find as (d) for directory and I’m still searching by the name of MyFolder.

One cool thing to note is that find doesn’t stop at the directory you are searching, it will look inside any subdirectories that directory may have as well.

Exercise

• Find a file from the root directory that has the word net in it.

Quiz Question

15. help

Lesson Content

Linux has some great built-in tools to help you how to use a command or check what flags are available for a command. One tool, help, is a built-in bash command that provides help for other bash commands (echo, logout, pwd, etc).

$ help echo

This will give you a description and the options you can use when you want to run echo. For other executable programs, it’s convention to have an option called –help or something similar.

$ echo --help

Not all developers who ship out executables will conform to this standard, but it’s probably your best shot to find some help on a program.

Exercise

Run help on the echo command, logout command and pwd command.

Quiz Question

16. man (Manual)

Lesson Content

Gee I wish some of these programs had a manual so we can see some more information about them. Well luckily they do! Aptly named man pages, you can see the manuals for a command with the man command.

$ man ls

Man pages are manuals that are by default built into most Linux operating systems. They provide documentation about commands and other aspects of the system.

Try it out on a few commands to get more information about them.

Exercise

Run the man command on the ls command.

Quiz Question

17. whatis

Lesson Content

Whew, we’ve learned quite a bit of commands so far, if you are ever feeling doubtful about what a command does, you can use the whatis command. The whatis command provides a brief description of command line programs.

$ whatis cat

The description gets sourced from the manual page of each command. If you ran whatis cat, you’d see there is a small blurb with a short description.

Exercise

Run the whatis command on the less command.

Quiz Question

18. alias

Lesson Content

Sometimes typing commands can get really repetitive, or if you need to type a long command many times, it’s best to have an alias you can use for that. To create an alias for a command you simply specify an alias name and set it to the command.

$ alias foobar='ls -la'

Now instead of typing ls -la, you can type foobar and it will execute that command, pretty neat stuff. Keep in mind that this command won’t save your alias after reboot, so you’ll need to add a permanent alias in:

~/.bashrc

or similar files if you want to have it persist after reboot.

You can remove aliases with the unalias command:

$ unalias foobar

Exercise

Create a couple of aliases then remove them.

Quiz Question

19. exit

Lesson Content

Well, you sure did a good job getting through the basics. We’ve only scratched the surface, now that you’ve learned to crawl, in the next set of courses, I’m gonna teach how to walk.

For now, you can pat yourself on the back and take a break. To exit from the shell, you can use the exit command

$ exit

Or the logout command:

$ logout

Or if you are working out of a terminal GUI, you can just close the terminal, see you in the next course!

Exercise

Exit out of the shell and see what happens. Make sure you don’t need to do anymore work in that shell.

Quiz Question

1. stdout (Standard Out)

Lesson Content

By now, we’ve become familiar with many commands and their output and that brings us to our next subject I/O (input/output) streams. Let’s run the following command and we’ll discuss how this works.

$ echo Hello World > peanuts.txt

What just happened? Well check the directory where you ran that command and lo and behold you should see a file called peanuts.txt, look inside that file and you should see the text Hello World. Lots of things just happened in one command so let’s break it down.

First let’s break down the first part:

$ echo Hello World

We know this prints out Hello World to the screen, but how? Processes use I/O streams to receive input and return output. By default the echo command takes the input (standard input or stdin) from the keyboard and returns the output (standard output or stdout) to the screen. So that’s why when you type echo Hello World in your shell, you get Hello World on the screen. However, I/O redirection allows us to change this default behavior giving us greater file flexibility.

Let’s proceed to the next part of the command:

 > 

The > is a redirection operator that allows us the change where standard output goes. It allows us to send the output of echo Hello World to a file instead of the screen. If the file does not already exist it will create it for us. However, if it does exist it will overwrite it (you can add a shell flag to prevent this depending on what shell you are using).

And that’s basically how stdout redirection works!

Well let’s say I didn’t want to overwrite my peanuts.txt, luckily there is a redirection operator for that as well, >>

$ echo Hello World >> peanuts.txt

This will append Hello World to the end of the peanuts.txt file, if the file doesn’t already exist it will create it for us like it did with the > redirector!

Exercise

Try a couple of commands:

$ ls -l /var/log > myoutput.txt
$ echo Hello World > rm
$ > somefile.txt 

Quiz Question

2. stdin (Standard In)

Lesson Content

In our previous lesson we learned that we have different stdout streams we can use, such as a file or the screen. Well there are also different standard input (stdin) streams we can use as well. We know that we have stdin from devices like the keyboard, but we can use files, output from other processes and the terminal as well, let’s see an example.

Let’s use the peanuts.txt file in the previous lesson for this example, remember it had the text Hello World in it.

$ cat < peanuts.txt > banana.txt 

Just like we had > for stdout redirection, we can use < for stdin redirection.

Normally in the cat command, you send a file to it and that file becomes the stdin, in this case, we redirected peanuts.txt to be our stdin. Then the output of cat peanuts.txt which would be Hello World gets redirected to another file called banana.txt.

Exercise

Try out a couple of commands:

$ echo < peanuts.txt > banana.txt
$ ls < peanuts.txt > banana.txt
$ pwd < peanuts.txt > banana.txt

Quiz Question

3. stderr (Standard Error)

Lesson Content

Let’s try something a little different now, let’s try to list the contents of a directory that doesn’t exist on your system and redirect the output to the peanuts.txt file again.

$ ls /fake/directory > peanuts.txt 

What you should see is:

ls: cannot access /fake/directory: No such file or directory

Now you’re probably thinking, shouldn’t that message have been sent to the file? There is actually another I/O stream in play here called standard error (stderr). By default, stderr sends its output to the screen as well, it’s a completely different stream than stdout. So you’ll need to redirect its output a different way.

Unfortunately the redirector is not as nice as using < or > but it’s pretty close. We will have to use file descriptors. A file descriptor is a non-negative number that is used to access a file or stream. We will go in depth about this later, but for now know that the file descriptor for stdin, stdout and stderr is 0, 1, and 2 respectively.

So now if we want to redirect our stderr to the file we can do this:

$ ls /fake/directory 2> peanuts.txt

You should see just the stderr messages in peanuts.txt.

Now what if I wanted to see both stderr and stdout in the peanuts.txt file? It’s possible to do this with file descriptors as well:

$ ls /fake/directory > peanuts.txt 2>&1

This sends the results of ls /fake/directory to the peanuts.txt file and then it redirects stderr to the stdout via 2>&1. The order of operations here matters, 2>&1 sends stderr to whatever stdout is pointing to. In this case stdout is pointing to a file, so 2>&1 also sends stderr to a file. So if you open up that peanuts.txt file you should see both stderr and stdout. In our case, the above command only outputs stderr.

There is a shorter way to redirect both stdout and stderr to a file:

$ ls /fake/directory &> peanuts.txt

Now what if I don’t want any of that cruft and want to get rid of stderr messages completely? Well you can also redirect output to a special file call /dev/null and it will discard any input.

$ ls /fake/directory 2> /dev/null

Exercise

What is the following command doing?

$ ls /fake/directory >> /dev/null 2>&1

Quiz Question

4. pipe and tee

Lesson Content

Let’s get into some plumbing now, not really but kinda. Let’s try a command:

$ ls -la /etc

You should see a very long list of items, it’s a little hard to read actually. Instead of redirecting this output to a file, wouldn’t it be nice if we could just see the output in another command like less? Well we can!

$ ls -la /etc | less 

The pipe operator |, represented by a vertical bar, allows us to get the stdout of a command and make that the stdin to another process. In this case, we took the stdout of ls -la /etc and then piped it to the less command. The pipe command is extremely useful and we will continue to use it for all eternity.

Well what if I wanted to write the output of my command to two different streams? That’s possible with the tee command:

$ ls | tee peanuts.txt

You should see the output of ls on your screen and if you open up the peanuts.txt file you should see the same information!

Exercise

Try the following commands:

$ ls | tee peanuts.txt banan.txt

Quiz Question

5. env (Environment)

Lesson Content

Run the following command:

$ echo $HOME

You should see the path to your home directory, mine looks like /home/pete.

What about this command?

$ echo $USER

You should see your username!

Where is this information coming from? It’s coming from your environment variables. You can view these by typing

$ env

This outputs a whole lot of information about the environment variables you currently have set. These variables contain useful information that the shell and other processes can use.

Here is a short example:

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/bin
PWD=/home/user
USER=pete

One particularly important variable is the PATH Variable. You can access these variables by sticking a $ infront of the variable name like so:

$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/bin

This returns a list of paths separated by a colon that your system searches when it runs a command. Let’s say you manually download and install a package from the internet and put it in to a non standard directory and want to run that command, you type $ coolcommand and the prompt says command not found. Well that’s silly you are looking at the binary in a folder and know it exists. What is happening is that $PATH variable doesn’t check that directory for this binary so it’s throwing an error.

Let’s say you had tons of binaries you wanted to run out of that directory, you can just modify the PATH variable to include that directory in your PATH environment variable.

Exercise

What does the following output? Why?

$ echo $HOME

Quiz Question

env

6. cut

Lesson Content

We’re gonna learn a couple of useful commands that you can use to process text. Before we get started, let’s create a file that we’ll be working with. Copy and paste the following command, once you do that add a TAB in between lazy and dog (hold down Ctrl-v + TAB).

$ echo 'The quick brown; fox jumps over the lazy  dog' > sample.txt

First command we’ll be learning about is the cut command. It extracts portions of text from a file.

To extract contents by a list of characters:

$ cut -c 5 sample.txt

This outputs the 5th character in each line of the file. In this case it is “q”, note that the space also counts as a character.

To extract the contents by a field, we’ll need to do a little modification:

$ cut -f 2 sample.txt

The -f or field flag cuts text based off of fields, by default it uses TABs as delimiters, so everything separated by a TAB is considered a field. You should see “dog” as your output.

You can combine the field flag with the delimiter flag to extract the contents by a custom delimiter:

$ cut -f 1 -d ";" sample.txt

This will change the TAB delimiter to a “;” delimiter and since we are cutting the first field, the result should be “The quick brown”.

Exercise

What does the following command do? Why?

$ cut -c 5-10 sample.txt
$ cut -c 5- sample.txt
$ cut -c -5 sample.txt

Quiz Question

7. paste

Lesson Content

The paste command is similar to the cat command, it merges lines together in a file. Let’s create a new file with the following contents:

sample2.txt
The
quick
brown
fox

Let’s combine all these lines into one line:

$ paste -s sample2.txt

The default delimiter for paste is TAB, so now there is one line with TABs separating each word.

Let’s change this delimiter (-d) to something a little more readable:

$ paste -d ' ' -s sample2.txt

Now everything should be on one line delimited by spaces.

Exercise

Try to paste multiple files together, what happens?

Quiz Question

8. head

Lesson Content

Let’s say we have a very long file, in fact we have many to choose from, go ahead and cat /var/log/syslog. You should see pages upon pages of text. What if I just wanted to see the first couple of lines in this text file? Well we can do that with the head command, by default the head command will show you the first 10 lines in a file.

$ head /var/log/syslog

You can also modify the line count to whatever you choose, let’s say I wanted to see the first 15 lines instead.

$ head -n 15 /var/log/syslog

The -n flag stands for number of lines.

Exercise

What does the following command do and why?

$ head -c 15 /var/log/syslog

Quiz Question

9. tail

Lesson Content

Similar to the head command, the tail command lets you see the last 10 lines of a file by default.

$ tail /var/log/syslog

Along with head you can change the number of lines you want to see.

$ tail -n 10 /var/log/syslog

Another great option you can use is the -f (follow) flag, this will follow the file as it grows. Give it a try and see what happens.

$ tail -f /var/log/syslog

Your syslog file will be continually changing while you interact with your system and using tail -f you can see everything that is getting added to that file.

Exercise

Look at the man page of tail and read some of the other commands we didn’t discuss.

$ man tail

Quiz Question

10. expand and unexpand

Lesson Content

In our lesson on the cut command, we had our sample.txt file that contained a tab. Normally TABs would usually show a noticeable difference but some text files don’t show that well enough. Having TABs in a text file may not be the desired spacing you want. To change your TABs to spaces, use the expand command.

$ expand sample.txt

The command above will print output with each TAB converted into a group of spaces. To save this output in a file, use output redirection like below.

$ expand sample.txt > result.txt

Opposite to expand, we can convert back each group of spaces to a TAB with the unexpand command:

$ unexpand -a result.txt

Exercise

What happens if you just type expand with no file input?

Quiz Question

11. join and split

Lesson Content

The join command allows you to join multiple files together by a common field:

Let’s say I had two files that I wanted to join together:

file1.txt
1 John
2 Jane
3 Mary

file2.txt
1 Doe
2 Doe
3 Sue

$ join file1.txt file2.txt
1 John Doe
2 Jane Doe
3 Mary Sue

See how it joined together my files? They are joined together by the first field by default and the fields have to be identical, if they are not you can sort them, so in this case the files are joined via 1, 2, 3.

How would we join the following files?

file1.txt
John 1
Jane 2
Mary 3

file2.txt
1 Doe
2 Doe
3 Sue

To join this file you need to specify which fields you are joining, in this case we want field 2 on file1.txt and field 1 on file2.txt, so the command would look like this:

$ join -1 2 -2 1 file1.txt file2.txt
1 John Doe
2 Jane Doe
3 Mary Sue

-1 refers to file1.txt and -2 refers to file2.txt. Pretty neat. You can also split a file up into different files with the split command:

$ split somefile

This will split it into different files, by default it will split them once they reach a 1000 line limit. The files are named x** by default.

Exercise

Join two files with different number of lines in each file, what happens?

Quiz Question

12. sort

Lesson Content

The sort command is useful for sorting lines.

file1.txt
dog
cow
cat
elephant
bird

$ sort file1.txt
bird
cat
cow
dog
elephant

You can also do a reverse sort:

$ sort -r file1.txt
elephant
dog
cow
cat
bird

And also sort via numerical value:

$ sort -n file1.txt
bird
cat
cow
elephant
dog

Exercise

The real power of sort comes with its ability to be combined with other commands, try the following command and see what happens?

$ ls /etc | sort -rn

Quiz Question

13. tr (Translate)

Lesson Content

The tr (translate) command allows you to translate a set of characters into another set of characters. Let’s try an example of translating all lower case characters to uppercase characters.

$ tr a-z A-Z
hello
HELLO

As you can see we made the ranges of a-z into A-Z and all text we type that is lowercase gets uppercased.

Exercise

Try the following command what happens?

$ tr -d ello
hello

Quiz Question

14. uniq (Unique)

Lesson Content

The uniq (unique) command is another useful tool for parsing text.

Let’s say you had a file with lots of duplicates:

reading.txt
book
book
paper
paper
article
article
magazine

And you wanted to remove the duplicates, well you can use the uniq command:

$ uniq reading.txt
book
paper
article
magazine

Let’s get the count of how many occurrences of a line:

$ uniq -c reading.txt
2 book
2 paper
2 article
1 magazine

Let’s just get unique values:

$ uniq -u reading.txt
magazine

Let’s just get duplicate values:

$ uniq -d reading.txt
book
paper
article

Note : uniq does not detect duplicate lines unless they are adjacent. For eg:

Let’s say you had a file with duplicates which are not adjacent:

reading.txt
book
paper
book
paper
article
magazine
article

$ uniq reading.txt
reading.txt
book
paper
book
paper
article
magazine
article

The result returned by uniq will contain all the entries unlike the very first example.

To overcome this limitation of uniq we can use sort in combination with uniq:

$ sort reading.txt | uniq
article
book
magazine
paper

Exercise

What result would you get if you tried uniq -uc?

Quiz Question

15. wc and nl

Lesson Content

The wc (word count) command shows the total count of words in a file.

$ wc /etc/passwd
 96     265    5925 /etc/passwd

It display the number of lines, number of words and number of bytes, respectively.

To just see just the count of a certain field, use the -l, -w, or -c respectively.

$ wc -l /etc/passwd
96

Another command you can use to check the count of lines on a file is the nl (number lines) command.

file1.txt
i
like
turtles

$ nl file1.txt
1. i
2. like
3. turtles

Exercise

How would you get the total count of lines by using the nl file without searching through the entire output? Hint: Use some of the other commands you learned in this course.

Quiz Question

16. grep

Lesson Content

The grep command is quite possibly the most common text processing command you will use. It allows you to search files for characters that match a certain pattern. What if you wanted to know if a file existed in a certain directory or if you wanted to see if a string was found in a file? You certainly wouldn’t dig through every line of text, you would use grep!

Let’s use our sample.txt file as an example:

$ grep fox sample.txt

You should see that grep found fox in the sample.txt file.

You can also grep patterns that are case insensitive with the -i flag:

$ grep -i somepattern somefile

To get even more flexible with grep you can combine it with other commands with |.

$ env | grep -i User

As you can see grep is pretty versatile. You can even use regular expressions in your pattern:

$ ls /somedir | grep '.txt$'

Should return all files ending with .txt in somedir.

Exercise

You may have heard of egrep or fgrep, these are deprecated grep calls and have since been replaced by grep -E and grep -F. Read the grep manpage to learn more.

Quiz Question

1. regex (Regular Expressions)

Lesson Content

Regular expressions are a powerful tool to do pattern based selection. It uses special notations similar to those we’ve encountered already such as the * wildcard.

We’ll go through a couple of the most common regular expressions, these are almost universal with any programming language.

Well use this phrase as our test string:

sally sells seashells 
by the seashore

1. Beginning of a line with ^

^by
would match the line "by the seashore"

2. End of a line with $

seashore$ 
would match the line "by the seashore"

3. Matching any single character with .

b. 
would match "by" 

4. Bracket notation with [] and ()

This can be a little tricky, brackets allow us to specify characters found within the bracket.

d[iou]g
would match: dig, dog, dug

The previous anchor tag ^ when used in a bracket means anything except the characters within the bracket.

d[^i]g
would match: dog and dug but not dig

Brackets can also use ranges to increase the amount of characters you want to use.

d[a-c]g
will match patterns like dag, dbg, and dcg

Be careful though as brackets are case sensitive:

d[A-C]g
will match dAg, dBg and dCg but not dag, dbg and dcg

And those are some basic regular expressions.

Exercise

Try to combine regular expressions with grep and search through some files.

grep [regular expression here] [file]

Quiz Question

2. Text Editors

Lesson Content

If you get a couple of diehard Linux users in a room and ask them what is the best text editor to use, you’ll hear a never ending banter about the godliness of either vim or emacs. Don’t even try to bring up using a GUI editor if you value your life.

Vim and emacs are popular text editors that are installed by default on most Linux distributions and they both have their pros and cons. If you want to get around your system like a ninja, you’ll need to pick up one of these text editors to use. They are essentially coding, word document processing and basically all in one editors.

Exercise

Take a little tour of vim and emacs:

• Vim
• Emacs

Quiz Question

No questions move along!

3. Vim (Vi Improved)

Lesson Content

Vim stands for vi (Improved) just like its name it stands for an improved version of the vi text editor command.

It’s super lightweight, opening and editing a file with vim is quick and easy. It’s also almost always available, if you booted up a random Linux distribution, chances are vim is installed by default.

To fire up vim just type:

$ vim

Exercise

No exercises for this lesson.

Quiz Question

No questions move along!

4. Vim Search Patterns

Lesson Content

To search for an expression just type the / key and then your search result while you are in a vim session. Once you hit enter, you can press “n” to go forward or “N” to go backward in your search results.

My pretty file is very pretty.

/pretty

will find the pretty words in the text file.

The ? search command will search the text file backwards, so in the previous example, the last pretty would come up first.

My pretty file is very pretty.

?pretty

will find the pretty words in the text file.

Exercise

Play with the search key, open a text file in vim with: vim [textfile] and start searching!

Quiz Question

5. Vim Navigation

Lesson Content

Now you may notice, the mouse is nowhere is use here. To navigate a text document in vim, use the following keys:

- h or the left arrow - will move you left one character 
- j or the up arrow - will move you up one line 
- k or the down arrow - will move you down one line 
- l or the right arrow - will move you right one character 

Exercise

No exercises for this lesson.

Quiz Question

6. Vim Appending Text

Lesson Content

Now you may have noticed if you tried to type something you wouldn’t be able to. That’s because you are in command mode. This can get pretty confusing especially if you just want to open a file and enter text. The command mode is used for when you enter commands like h,j,k.l etc. To insert text you’ll need to enter insert mode first.

i - insert text before the cursor
O - insert text on the previous line
o - insert text on the next line
a - append text after the cursor
A - append text at the end of the line

Notice how when you type any of these insertion modes, you’ll see that vim has entered insert mode at the bottom of the shell. To exit insert mode and go back to command mode, just hit the Esc key.

Exercise

Play around with entering and exiting insertion mode.

Quiz Question

7. Vim Editing

Lesson Content

Now that we have a couple of lines written, let’s edit it a bit more and remove some cruft.

x - used to cut the selected text also used for deleting characters
dd - used to delete the current line
y - yank or copy whatever is selected
yy - yank or copy the current line
p - paste the copied text before the cursor

Exercise

I know this lesson added some oddballs, open up a text editor and play around with these.

Quiz Question

8. Vim Saving and Exiting

Lesson Content

Now that you’ve done your editing it’s time to actually save and quit out of vim:

• :w - writes or saves the file

• :q - quit out of vim

• :wq - write and then quit

• :q! - quit out of vim without saving the file

• ZZ - equivalent of :wq, but one character faster

• u - undo your last action

• Ctrl-r - redo your last action

You may not think ZZ is necessary, but you’ll eventually see that your fingers may tend to lean towards this rather than :wq.

Whew that was a lot of information to take about Vim. Now that you know some basic commands and navigation, you can start editing some text files. There are many more options you can use in vim to increase your ability to master this text editor, head on to Vim’s online guide to take a look.

Exercise

No exercises for this lesson.

Quiz Question

9. Emacs

Lesson Content

Emacs is for users who want an extremely powerful text editor, which may be an understatement because you essentially live in emacs. You can do all your code editing, file manipulation, etc all within emacs. It’s a bit slower to load up and the learning curve is a bit steeper than vim, but if you want a powerful editor that is extremely extensible, this is the one for you. When I say extensible, I literally mean you can write up scripts for emacs that extend its functionality.

To start emacs just use:

emacs

You should be greeted with the default welcome buffer.

Buffers in emacs is what your text resides in. So if you open up a file, a buffer is used to store that file’s content. You can have multiple buffers open at the same time and you can easily switch between buffers.

Exercise

No exercises for this lesson.

Quiz Question

No questions move along!

10. Emacs Manipulate Files

Lesson Content

In a lot (if not all) of Emacs documentation, you will see the syntax C-[letter]. This just means hit the Ctrl-letter, but for shorthand purposes, we’ll call Ctrl with C. If you see syntax such as M-[letter], that means use the Meta key, most commonly the Alt key.

Saving files

C-x C-s - Save a file
C-x C-w - Save file as
C-x s - Save all

The save file options will prompt you if you want to save each file.

Opening a file

C-x C-f

This will prompt you to type a filename to open. If you do not have a file that already exists, it will create a new file. You can load up a directory as well.

Exercise

Play around with opening files and saving files.

Quiz Question

11. Emacs Buffer Navigation

Lesson Content

To move around buffers (or files you’re visiting) use the following commands:

Switch buffers

C-x b - switch buffer
C-x right arrow - right-cycle through buffer
C-x left arrow - left-cycle through buffer

Close the buffer

C-x k

Split the current buffer

C-x 2

This allows you see multiple buffers on one screen. To move between these buffers use: C-x o

Set a single buffer as the current screen

C-x 1

If you ever used a terminal multiplexer like screen and tmux, the buffer commands will feel very familiar.

Exercise

Play around with buffers.

Quiz Question

12. Emacs Editing

Lesson Content

Text Navigation

C-up arrow : move up one paragraph
C-down arrow: move down one paragraph
C-left arrow: move one word left
C-right arrow: move one word right
M-> : move to the end of the buffer

With text navigation, your regular text buttons work as they should, home, end, page up, page down and the arrow keys, etc.

Cutting and Pasting

To cut (kill) or paste (yank) in Emacs you’ll need to be able to select text first. To select text, move your cursor to where you want to cut or paste and hit

C-space key

then you can use the navigation keys to select the text you want. Now you can do the cut and paste like so:

C-w : cut
C-y : yank

Exercise

Play around with text navigation.

Quiz Question

13. Emacs Exiting and Help

Lesson Content

To close out of emacs

C-x C-c

If you have any open buffers, it will ask you to save it before closing out of emacs.

Confused?

C-h C-h : help menu

Undo

C-x u

As you can see Emacs has more moving parts, so the learning curve is a little steeper. In exchange though, you get a very powerful text editor.

Exercise

Visit the Emacs site to learn about more commands. Emacs

Quiz Question

1. Users and Groups

Lesson Content

In any traditional operating system, there are users and groups. They exist solely for access and permissions. When running a process, it will run as the owner of that process whether that is Jane or Bob. File access and ownership is also permission dependent. You wouldn’t want Jane to see Bob’s documents and vice versa.

Each user has their own home directory where their user specific files get stored, this is usually located in /home/username, but can vary in different distributions.

The system uses user ids (UID) to manage users, usernames are the friendly way to associate users with identification, but the system identifies users by their UID. The system also uses groups to manage permissions, groups are just sets of users with permission set by that group, they are identified by the system with their group ID (GID).

In Linux, you’ll have users in addition to the normal humans that use the system. Sometimes these users are system daemons that continuously run processes to keep the system functioning. One of the most important users is root or superuser, root is the most powerful user on the system, root can access any file and start and terminate any process. For that reason, it can be dangerous to operate as root all the time, you could potentially remove system critical files. Luckily, if root access is needed and a user has root access, they can run a command as root instead with the sudo command. The sudo command (superuser do) is used to run a command with root access, we’ll go more in depth on how a user receives root access in a later lesson.

Go ahead and try to view a protected file like /etc/shadow:

$ cat /etc/shadow

Notice how you get a permission denied error, look at the permissions with:

$ ls -la /etc/shadow

-rw-r----- 1 root shadow 1134 Dec 1 11:45 ```/etc/shadow```

We haven’t gone through permissions yet, but what’s happening here is that root is the owner of the file and you’ll need root access or be part of the shadow group to read the contents. Now run the command with sudo:

$ sudo cat /etc/shadow

Now you’ll be able to see the contents of the file!

Exercise

No exercises for this lesson.

Quiz Question

2. root

Lesson Content

We’ve looked at one way to get superuser access using the sudo command. You can also run commands as the superuser with the su command. This command will “substitute users” and open a root shell if no username is specified. You can use this command to substitute to any user as long as you know the password.

$ su

There are some downsides to using this method: it’s much easier to make a critical mistake running everything in root, you won’t have records of the commands you use to change system configurations, etc. Basically, if you need to run commands as the superuser, just stick to sudo.

Now that you know what commands to run as the superuser, the question is how do you know who has access to do that? The system doesn’t let every single Joe Schmoe run commands as the superuser, so how does it know? There is a file called the /etc/sudoers file, this file lists users who can run sudo. You can edit this file with the visudo command.

Exercise

Open up the /etc/sudoers file and see what superuser permissions other users on the machine have.

Quiz Question

3. etc/passwd

Lesson Content

Remember that usernames aren’t really identifications for users. The system uses a user ID (UID) to identify a user. To find out what users are mapped to what ID, look at the /etc/passwd file.

$ cat /etc/passwd

This file shows you a list of users and detailed information about them. For example, the first line in this file most likely looks like this:

root:x:0:0:root:/root:/bin/bash

Each line displays user information for one user, most commonly you’ll see the root user as the first line. There are many fields separated by colons that tell you additional information about the user, let’s look at them all:

• Username
• User’s password - the password is not really stored in this file, it’s usually stored in the /etc/shadow file. We’ll discuss more in the next lesson about /etc/shadow, but for now, know that it contains encrypted user passwords. You can see many different symbols that are in this field, if you see an “x” that means the password is stored in the /etc/shadow file, a “*” means the user doesn’t have login access and if there is a blank field that means the user doesn’t have a password.
• The user ID - as you can see root has the UID of 0
• The group ID
• GECOS field - This is used to generally leave comments about the user or account such as their real name or phone number, it is comma delimited.
• User’s home directory
• User’s shell - you’ll probably see a lot of user’s defaulting to bash for their shell

Normally in a user’s setting page, you would expect you see just human users. However, you’ll notice /etc/passwd contains other users. Remember that users are really only on the system to run processes with different permissions. Sometimes we want to run processes with pre-determined permissions. For example, the daemon user is used for daemon processes.

Also should note that you can edit the /etc/passwd file by hand if you want to add users and modify information with the vipw tool, however things like these are best left to the tools we will discuss in a later lesson such as useradd and userdel.

Exercise

Look at your /etc/passwd file, take a look at some of the users and note the access they have.

Quiz Question

4. etc/shadow

Lesson Content

The /etc/shadow file is used to store information about user authentication. It requires superuser read permissions.

$ sudo cat /etc/shadow

root:MyEPTEa$6Nonsense:15000:0:99999:7:::

You’ll notice that it looks very similar to the contents of /etc/passwd, however in the password field you’ll see an encrypted password. The fields are separated by colons as followed:

• Username
• Encrypted password
• Date of last password changed - expressed as the number of days since Jan 1, 1970. If there is a 0 that means the user should change their password the next time they login
• Minimum password age - Days that a user will have to wait before being able to change their password again
• Maximum password age - Maximum number of days before a user has to change their password
• Password warning period - Number of days before a password is going to expire
• Password inactivity period - Number of days after a password has expired to allow login with their password
• Account expiration date - date that user will not be able to login
• Reserved field for future use

In most distributions today, user authentication doesn’t rely on just the /etc/shadow file, there are other mechanisms in place such as PAM (Pluggable Authentication Modules) that replace authentication.

Exercise

Take a look at the /etc/shadow file

Quiz Question

No questions move along!

5. etc/group

Lesson Content

Another file that is used in user management is the /etc/group file. This file allows for different groups with different permissions.

$ cat /etc/group

root:*:0:pete

Very similar to the /etc/password field, the /etc/group fields are as follows:

• Group name
• Group password - there isn’t a need to set a group password, using an elevated privilege like sudo is standard. A “*” will be put in place as the default value.
• Group ID (GID)
• List of users - you can manually specify users you want in a specific group

Exercise

Run the command groups. What do you see?

Quiz Question

6. User Management Tools

Lesson Content

Most enterprise environments are using management systems to manage users, accounts and passwords. However, on a single machine computer there are useful commands to run to manage users.

Adding Users

You can use the adduser or the useradd command. The adduser command contains more helpful features such as making a home directory and more. There are configuration files for adding new users that can be customized depending on what you want to allocate to a default user.

$ sudo useradd bob 

You’ll see that the above command creates an entry in /etc/passwd for bob, sets up default groups and adds an entry to the /etc/shadow file.

Removing Users

To remove a user, you can use the userdel command.

$ sudo userdel bob 

This basically does its best to undo the file changes by useradd.

Changing Passwords

$ passwd bob 

This will allow you to change the password of yourself or another user (if you are root).

Exercise

Create a new user then change their password and login as the new user.

Quiz Question

1. File Permissions

Lesson Content

As we learned previously, files have different permissions or file modes. Let’s look at an example:

$ ls -l Desktop/
drwxr-xr-x 2 pete penguins 4096 Dec 1 11:45 .

There are four parts to a file’s permissions. The first part is the filetype, which is denoted by the first character in the permissions, in our case since we are looking at a directory it shows d for the filetype. Most commonly you will see a - for a regular file.

The next three parts of the file mode are the actual permissions. The permissions are grouped into 3 bits each. The first 3 bits are user permissions, then group permissions and then other permissions. I’ve added the pipe to make it easier to differentiate.

d | rwx | r-x | r-x 

Each character represent a different permission:

• r: readable
• w: writable
• x: executable (basically an executable program)
• -: empty

So in the above example, we see that the user pete has read, write and execute permissions on the file. The group penguins has read and execute permissions. And finally, the other users (everyone else) has read and execute permissions.

Exercise

Use the ls -l command on multiple files and recite their permissions, user and group.

Quiz Question

2. Modifying Permissions

Lesson Content

Changing permissions can easily be done with the chmod command.

First, pick which permission set you want to change, user, group or other. You can add or remove permissions with a + or -, let’s look at some examples.

Adding permission bit on a file

$ chmod u+x myfile

The above command reads like this: change permission on myfile by adding executable permission bit on the user set. So now the user has executable permission on this file!

Removing permission bit on a file

$ chmod u-x myfile

Adding multiple permission bits on a file

$ chmod ug+w

There is another way to change permissions using numerical format. This method allows you to change permissions all at once. Instead of using r, w, or x to represent permissions, you’ll use a numerical representation for a single permission set. So no need to specify the group with g or the user with u.

The numerical representations are seen below:

• 4: read permission
• 2: write permission
• 1: execute permission

Let’s look at an example:

$ chmod 755 myfile

Can you guess what permissions we are giving this file? Let’s break this down, so now 755 covers the permissions for all sets. The first number (7) represents user permissions, the second number (5) represents group permissions and the last 5 represents other permissions.

Wait a minute, 7 and 5 weren’t listed above, where are we getting these numbers? Remember we are combining all the permissions into one number now, so you’ll have to get some math involved.

7 = 4 + 2 + 1, so 7 is the user permissions and it has read, write and execute permissions

5 = 4 + 1, the group has read and execute permissions

5 = 4 +1, and all other users have read and execute permissions

One thing to note: it’s not a great idea to be changing permissions nilly willy, you could potentially expose a sensitive file for everyone to modify, however many times you legitimately want to change permissions, just take precaution when using the chmod command.

Exercise

Change some basic text file permissions and see the bits changing as you do an ls -l.

Quiz Question

3. Ownership Permissions

Lesson Content

In addition to modifying permissions on files, you can also modify the group and user ownership of the file as well.

Modify user ownership

$ sudo chown patty myfile 

This command will set the owner of myfile to patty.

Modify group ownership

$ sudo chgrp whales myfile 

This command will set the group of myfile to whales.

Modify both user and group ownership at the same time If you add a colon and groupname after the user you can set both the user and group at the same time.

$ sudo chown patty:whales myfile 

Exercise

Modify the group and user of some test files. Afterwards take a look at the permissions with ls -l.

Quiz Question

4. Umask

Lesson Content

Every file that gets created comes with a default set of permissions. If you ever wanted to change that default set of permissions, you can do so with the umask command. This command takes the 3 bit permission set we see in numerical permissions.

Instead of adding these permissions though, umask takes away these permissions.

$ umask 021 

In the above example, we are stating that we want the default permissions of new files to allow users access to everything, but for groups we want to take away their write permission and for others we want to take away their executable permission. The default umask on most distributions is 022, meaning all user access, but no write access for group and other users.

When you run the umask command it will give that default set of permissions on any new file you make. However, if you want it to persist you’ll have to modify your startup file (.profile), but we’ll discuss that in a later lesson.

Exercise

• Create a new file, then note it’s permissions.
• Modify the umask and then create another new file.
• Check the permissions once more on the new file, what do you expect to see?

Quiz Question

5. Setupid

Lesson Content

There are many cases in which normal users need elevated access to do stuff. The system administrator can’t always be there to enter in a root password every time a user needed access to a protected file, so there are special file permission bits to allow this behavior. The Set User ID (SUID) allows a user to run a program as the owner of the program file rather than as themselves.

Let’s look at an example:

Let’s say I want to change my password, simple right? I just use the passwd command:

$ passwd 

What is the password command doing? It’s modifying a couple of files, but most importantly it’s modifying the /etc/shadow file. Let’s look at that file for a second:

$ ls -l /etc/shadow

-rw-r----- 1 root shadow 1134 Dec 1 11:45 /etc/shadow

Oh wait a minute here, this file is owned by root? How is it possible that we are able to modify a file owned by root?

Let’s look at another permission set, this time of the command we ran:

$ ls -l /usr/bin/passwd

-rwsr-xr-x 1 root root 47032 Dec 1 11:45 /usr/bin/passwd

You’ll notice a new permission bit here s. This permission bit is the SUID, when a file has this permission set, it allows the users who launched the program to get the file owner’s permission as well as execution permission, in this case root. So essentially while a user is running the password command, they are running as root.

That’s why we are able to access a protected file like /etc/shadow when we run the passwd command. Now if you removed that bit, you would see that you will not be able to modify /etc/shadow and therefore change your password.

Modifying SUID

Just like regular permissions there are two ways to modify SUID permissions.

Symbolic way:

$ sudo chmod u+s myfile 

Numerical way:

 sudo chmod 4755 myfile 

As you can see the SUID is denoted by a 4 and pre-pended to the permission set. You may see the SUID denoted as a capital S this means that it still does the same thing, but it does not have execute permissions.

Exercise

Look at the permission for /etc/passwd in detail, do you notice anything else? Files with SUID enabled are also easily distinguishable.

Quiz Question

6. Setgid

Lesson Content

Similar to the set user ID permission bit, there is a set group ID (SGID) permission bit. This bit allows a program to run as if it was a member of that group.

Let’s look at one example:

$ ls -l /usr/bin/wall
-rwxr-sr-x 1 root tty 19024 Dec 14 11:45 /usr/bin/wall

We can see now that the permission bit is in the group permission set.

Modifying SGID

$ sudo chmod g+s myfile
$ sudo chmod 2555 myfile

The numerical representation for SGID is 2.

Exercise

No exercises for this lesson.

Quiz Question

7. Process Permissions

Lesson Content

Let’s segway into process permissions for a bit, remember how I told you that when you run the passwd command with the SUID permission bit enabled you will run the program as root? That is true, however does that mean since you are temporarily root you can modify other user’s passwords? Nope fortunately not!

This is because of the many UIDs that Linux implements. There are three UIDS associated with every process:

When you launch a process, it runs with the same permissions as the user or group that ran it, this is known as an effective user ID. This UID is used to grant access rights to a process. So naturally if Bob ran the touch command, the process would run as him and any files he created would be under his ownership.

There is another UID, called the real user ID this is the ID of the user that launched the process. These are used to track down who the user who launched the process is.

One last UID is the saved user ID, this allows a process to switch between the effective UID and real UID, vice versa. This is useful because we don’t want our process to run with elevated privileges all the time, it’s just good practice to use special privileges at specific times.

Now let’s piece these all together by looking at the passwd command once more.

When running the passwd command, your effective UID is your user ID, let’s say its 500 for now. Oh but wait, remember the passwd command has the SUID permission enabled. So when you run it, your effective UID is now 0 (0 is the UID of root). Now this program can access files as root.

Let’s say you get a little taste of power and you want to modify Sally’s password, Sally has a UID of 600. Well you’ll be out of luck, fortunately the process also has your real UID in this case 500. It knows that your UID is 500 and therefore you can’t modify the password of UID of 600. (This of course is always bypassed if you are a superuser on a machine and can control and change everything).

Since you ran passwd, it will start the process off using your real UID, and it will save the UID of the owner of the file (effective UID), so you can switch between the two. No need to modify all files with root access if it’s not required.

Most of the time the real UID and the effective UID are the same, but in such cases as the passwd command they will change.

Exercise

We haven’t discussed processes yet, we can still take a look at this change happening in real time:

• Open one terminal window, and run the command: watch -n 1 “ps aux | grep passwd”. This will watch for the passwd process.-
• Open a second terminal window and run: passwd-
• Look at the first terminal window, you’ll see a process come up for passwd. The first column in the process table is the effective user ID, lo and behold it’s the root user!-

Quiz Question

8. Thee Sticky Bit

Lesson Content

One last special permission bit I want to talk about is the sticky bit.

This permission bit, “sticks a file/directory” this means that only the owner or the root user can delete or modify the file. This is very useful for shared directories. Take a look at the example below:

$ ls -ld /tmp
drwxrwxrwxt 6 root root 4096 Dec 15 11:45 /tmp

You’ll see a special permission bit at the end here **t, this means everyone can add files, write files, modify files in the /tmp directory, but only root can delete the /tmp directory.

Modify sticky bit

$ sudo chmod +t mydir

$ sudo chmod 1755 mydir

The numerical representation for the sticky bit is 1

Exercise

What other files and directories do you think have a sticky bit enabled?

Quiz Question

1. ps (Processes)

Lesson Content

Processes are the programs that are running on your machine. They are managed by the kernel and each process has an ID associated with it called the **process ID (PID). ** This PID is assigned in the order that processes are created.

Go ahead and run the ps command to see a list of running processes:

$ ps

PID       TTY    STAT         TIME        CMD
41230    pts/4    Ss        00:00:00     bash
51224    pts/4    R+        00:00:00     ps

This shows you a quick snapshot of the current processes:

• PID: Process ID
• TTY: Controlling terminal associated with the process (we’ll go in detail about this later)
• STAT: Process status code
• TIME: Total CPU usage time
• CMD: Name of executable/command

If you look at the man page for ps you’ll see that there are lots of command options you can pass, they will vary depending on what options you want to use - BSD, GNU or Unix. In my opinion the BSD style is more popular to use, so we’re gonna go with that. If you are curious the difference between the styles is the amount of dashes you use and the flags.

$ ps aux

The a displays all processes running, including the ones being ran by other users. The u shows more details about the processes. And finally the x lists all processes that don’t have a TTY associated with it, these programs will show a ? in the TTY field, they are most common in daemon processes that launch as part of the system startup.

You’ll notice you’re seeing a lot more fields now, no need to memorize them all, in a later course on advanced processes, we’ll go over some of these again:

• USER: The effective user (the one whose access we are using)
• PID: Process ID
• %CPU: CPU time used divided by the time the process has been running
• %MEM: Ratio of the process’s resident set size to the physical memory on the machine
• VSZ: Virtual memory usage of the entire process
• RSS: Resident set size, the non-swapped physical memory that a task has used
• TTY: Controlling terminal associated with the process
• STAT: Process status code
• START: Start time of the process
• TIME: Total CPU usage time
• COMMAND: Name of executable/command

The ps command can get a little messy to look at, for now the fields we will look at the most are PID, STAT and COMMAND.

Another very useful command is the **top ** command, top gives you real time information about the processes running on your system instead of a snapshot. By default you’ll get a refresh every 10 seconds. Top is an extremely useful tool to see what processes are taking up a lot of your resources.

$ top

Exercise

Use the ps command with different flags and see how the output changes.

Quiz Question

2. Controlling Terminal

Lesson Content

We discussed how there is a TTY field in the ps output. The TTY is the terminal that executed the command.

There are two types of terminals, regular terminal devices and pseudoterminal devices. A regular terminal device is a native terminal device that you can type into and send output to your system, this sounds like the terminal application you’ve been launching to get to your shell, but it’s not.

We’re gonna segue so you can see this action, go ahead and type Ctrl-Alt-F1 to get into TTY1 (the first virtual console), you’ll notice how you don’t have anything except the terminal, no graphics, etc. This is considered a regular terminal device, you can exit this with Ctrl-Alt-F7.

A pseudoterminal is what you’ve been used to working in, they emulate terminals with the shell terminal window and are denoted by PTS . If you look at ps again, you’ll see your shell process under pts/*.

Ok, now circling back to the controlling terminal, processes are usually bound to a controlling terminal. For example, if you were running a program on your shell window such as find and you closed the window, your process would also go with it.

There are processes such as daemon processes, which are special processes that are essentially keeping the system running. They often start at system boot and usually get terminated when the system is shutdown. They run in the background and since we don’t want these special processes to get terminated they are not bound to a controlling terminal. In the ps output, the TTY is listed as a ? meaning it does not have a controlling terminal.

Exercise

Look at your ps output and list all the unique TTY values.

Quiz Question

3. Process Details

Lesson Content

Before we get into more practical applications of processes, we have to first understand what they are and how they work. This part can get confusing since we are diving into the nitty gritty, so feel free to come back to this lesson if you don’t want to learn about it now.

A process like we said before is a running program on the system, more precisely it’s the system allocating memory, CPU, I/O to make the program run. A process is an instance of a running program, go ahead and open 3 terminal windows, in two windows, run the cat command without passing any options (the cat process will stay open as a process because it expects stdin). Now in the third window run: ps aux | grep cat. You’ll see that there are two processes for cat, even though they are calling the same program.

The kernel is in charge of processes, when we run a program the kernel loads up the code of the program in memory, determines and allocates resources and then keeps tabs on each process, it knows:

• The status of the process
• The resources the process is using and receives
• The process owner
• Signal handling (more on that later)
• And basically everything else

All processes are trying to get a taste of that sweet resource pie, it’s the kernel’s job to make sure that processes get the right amount of resources depending on process demands. When a process ends, the resources it used are now freed up for other processes.

Exercise

No exercises for this lesson.

Quiz Question

4. Process Creation

Lesson Content

Again this lesson and the next are purely information to let you see what’s under the hood, feel free to circle back to this once you’ve worked with processes a bit more.

When a new process is created, an existing process basically clones itself using something called the fork system call (system calls will be discussed very far into the future). The fork system call creates a mostly identical child process, this child process takes on a new process ID (PID) and the original process becomes its parent process and has something called a parent process ID PPID. Afterwards, the child process can either continue to use the same program its parent was using before or more often use the execve system call to launch up a new program. This system call destroys the memory management that the kernel put into place for that process and sets up new ones for the new program.

We can see this in action:

$ ps l

The l option gives us a “long format” or even more detailed view of our running processes. You’ll see a column labelled PPID, this is the parent ID. Now look at your terminal, you’ll see a process running that is your shell, so on my system I have a process running bash. Now remember when you ran the ps l command, you were running it from the process that was running bash. Now you’ll see that the PID of the bash shell is the PPID of the ps l command.

So if every process has to have a parent and they are just forks of each other, there must be a mother of all processes right? You are correct, when the system boots up, the kernels creates a process called init, it has a PID of 1. The init process can’t be terminated unless the system shuts down. It runs with root privileges and runs many processes that keep the system running. We will take a closer look at init in the system bootup course, for now just know it is the process that spawns all other processes.

Exercise

Take a look at your running processes, can you see what other processes have parents?

Quiz Question

5. Process Termination

Lesson Content

Now that we know what goes on when a process gets created, what is happening when we don’t need it anymore? Be forewarned, sometimes Linux can get a little dark…

A process can exit using the _exit system call, this will free up the resources that process was using for reallocation. So when a process is ready to terminate, it lets the kernel know why it’s terminating with something called a termination status. Most commonly a status of 0 means that the process succeeded. However, that’s not enough to completely terminate a process. The parent process has to acknowledge the termination of the child process by using the wait system call and what this does is it checks the termination status of the child process. I know it’s gruesome to think about, but the wait call is a necessity, after all what parent wouldn’t want to know how their child died?

There is another way to terminate a process and that involves using signals, which we will discuss soon.

Orphan Processes

When a parent process dies before a child process, the kernel knows that it’s not going to get a wait call, so instead it makes these processes “orphans” and puts them under the care of init (remember mother of all processes). Init will eventually perform the wait system call for these orphans so they can die.

Zombie Processes

What happens when a child terminates and the parent process hasn’t called wait yet? We still want to be able to see how a child process terminated, so even though the child process finished, the kernel turns the child process into a zombie process. The resources the child process used are still freed up for other processes, however there is still an entry in the process table for this zombie. Zombie processes also cannot be killed, since they are technically “dead”, so you can’t use signals to kill them. Eventually if the parent process calls the wait system call, the zombie will disappear, this is known as “reaping”. If the parent doesn’t perform a wait call, init will adopt the zombie and automatically perform wait and remove the zombie. It can be a bad thing to have too many zombie processes, since they take up space on the process table, if it fills up it will prevent other processes from running.

Exercise

No exercises for this lesson.

Quiz Question

6. Signals

Lesson Content

A signal is a notification to a process that something has happened.

Why we have signals

They are software interrupts and they have lots of uses:

• A user can type one of the special terminal characters (Ctrl-C) or (Ctrl-Z) to kill, interrupt or suspend processes
• Hardware issues can occur and the kernel wants to notify the process
• Software issues can occur and the kernel wants to notify the process
• They are basically ways processes can communicate

Signal process

When a signal is generated by some event, it’s then delivered to a process, it’s considered in a pending state until it’s delivered. When the process is ran, the signal will be delivered. However, processes have signal masks and they can set signal delivery to be blocked if specified. When a signal is delivered, a process can do a multitude of things:

• Ignore the signal
• “Catch” the signal and perform a specific handler routine
• Process can be terminated, as opposed to the normal exit system call
• Block the signal, depending on the signal mask

Common signals

Each signal is defined by integers with symbolic names that are in the form of SIGxxx. Some of the most common signals are:

• SIGHUP or HUP or 1: Hangup
• SIGINT or INT or 2: Interrupt
• SIGKILL or KILL or 9: Kill
• SIGSEGV or SEGV or 11: Segmentation fault
• SIGTERM or TERM or 15: Software termination
• SIGSTOP or STOP: Stop

Numbers can vary with signals so they are usually referred by their names.

Some signals are unblockable, one example is the SIGKILL signal. The KILL signal destroys the process.

Exercise

No exercises for this lesson.

Quiz Question

7. Kill (Terminate)

Lesson Content

You can send signals that terminate processes, such a command is aptly named the kill command.

$ kill 12445

The 12445 is the PID of the process you want to kill. By default it sends a TERM signal. The SIGTERM signal is sent to a process to request its termination by allowing it to cleanly release its resources and saving its state.

You can also specify a signal with the kill command:

$ kill -9 12445

This will run the SIGKILL signal and kill the process.

Differences between SIGHUP, SIGINT, SIGTERM, SIGKILL, SIGSTOP?

These signals all sound reasonably similar, but they do have their differences.

• SIGHUP - Hangup, sent to a process when the controlling terminal is closed. For example, if you closed a terminal window that had a process running in it, you would get a SIGHUP signal. So basically you’ve been hung up on
• SIGINT - Is an interrupt signal, so you can use Ctrl-C and the system will try to gracefully kill the process
• SIGTERM - Kill the process, but allow it to do some cleanup first
• SIGKILL - Kill the process, kill it with fire, doesn’t do any cleanup
• SIGSTOP - Stop/suspend a process

Exercise

Kill some processes using different signals.

Quiz Question

8. niceness

Lesson Content

When you run multiple things on your computer, like perhaps Chrome, Microsoft Word or Photoshop at the same time, it may seem like these processes are running at the same time, but that isn’t quite true.

Processes use the CPU for a small amount of time called a time slice. Then they pause for milliseconds and another process gets a little time slice. By default, process scheduling happens in this round-robin fashion. Every process gets enough time slices until it’s finished processing. The kernel handles all of these switching of processes and it does a pretty good job at it most of the time.

Processes aren’t able to decide when and how long they get CPU time, if all processes behaved normally they would each (roughly) get an equal amount of CPU time. However, there is a way to influence the kernel’s process scheduling algorithm with a nice value. Niceness is a pretty weird name, but what it means is that processes have a number to determine their priority for the CPU. A high number means the process is nice and has a lower priority for the CPU and a low or negative number means the process is not very nice and it wants to get as much of the CPU as possible.

$ top

You can see a column for NI right now, that is the niceness level of a process.

To change the niceness level you can use the nice and renice commands:

$ nice -n 5 apt upgrade

The nice command is used to set priority for a new process. The renice command is used to set priority on an existing process.

$ renice 10 -p 3245

Exercise

What processes aren’t very nice and why?

Quiz Question

9. Process States

Lesson Content

Let’s take a look at the ps aux command again:

$ ps aux

In the STAT column, you’ll see lots of values. A linux process can be in a number of different states. The most common state codes you’ll see are described below:

• R: running or runnable, it is just waiting for the CPU to process it
• S: Interruptible sleep, waiting for an event to complete, such as input from the terminal
• D: Uninterruptible sleep, processes that cannot be killed or interrupted with a signal, usually to make them go away you have to reboot or fix the issue
• Z: Zombie, we discussed in a previous lesson that zombies are terminated processes that are waiting to have their statuses collected
• T: Stopped, a process that has been suspended/stopped

Exercise

Take a look at the running processes on your system and check out their process states.

Quiz Question

10. /proc Filesystem

Lesson Content

Remember everything in Linux is a file, even processes. Process information is stored in a special filesystem known as the /proc filesystem.

$ ls /proc

You should see multiple values in here, there are sub-directories for every PID. If you looked at a PID in the ps output, you would be able to find it in the /proc directory.

Go ahead and enter one of the processes and look at that file:

$ cat /proc/12345/status

You should see process state information and well as more detailed information. The /proc directory is how the kernel is views the system, so there is a lot more information here than what you would see in ps.

Exercise

No exercises for this lesson.

Quiz Question

11. Job Control

Lesson Content

Let’s say you’re working on a single terminal window and you’re running a command that is taking forever. You can’t interact with the shell until it is complete, however we want to keep working on our machines, so we need that shell open. Fortunately we can control how our processes run with jobs:

Sending a job to the background

Appending an ampersand (&) to the command will run it in the background so you can still use your shell. Let’s see an example:

$ sleep 1000 &
$ sleep 1001 &
$ sleep 1002 &

View all background jobs

Now you can view the jobs you just sent to the background.

$ jobs

[1]    Running     sleep 1000 &
[2]-   Running     sleep 1001 &
[3]+   Running     sleep 1002 &

This will show you the job id in the first column, then the status and the command that was run. The + next to the job ID means that it is the most recent background job that started. The job with the - is the second most recent command.

Sending a job to the background on existing job

If you already ran a job and want to send it to the background, you don’t have to terminate it and start over again. First suspend the job with Ctrl-Z, then run the bg command to send it to the background.

pete@icebox ~ $ sleep 1003
^Z
[4]+    Stopped     sleep 1003

pete@icebox ~ $ bg
[4]+    sleep 1003 &

pete@icebox ~ $ jobs

[1]    Running     sleep 1000 &
[2]    Running     sleep 1001 &
[3]-   Running     sleep 1002 &
[4]+   Running     sleep 1003 &

Moving a job from the background to the foreground

To move a job out of the background just specify the job ID you want. If you run fg without any options, it will bring back the most recent background job (the job with the + sign next to it)

$ fg %1

Kill background jobs

Similar to moving jobs out of the background, you can use the same form to kill the processes by using their Job ID.

kill %1

Exercise

Move some jobs between the background and the foreground

Quiz Question

1. Software Distribution

Lesson Content

Your system is comprised of many packages such as internet browsers, text editors, media players, etc. These packages are managed via package managers, which install and maintain the software on your system. Not all packages are installed through package managers though, you can commonly install packages directly from their source code (we’ll get to that soon). However the majority of the time you will use a package manager to install software, the most common variety of packages are Debian (.deb) and Red Hat (.rpm). Debian style packages are used in distributions such as Debian, Ubuntu, LinuxMint, etc. Red Hat style packages are seen in Red Hat Enterprise Linux, Fedora, CentOS, etc.

What are packages? You may know them as Chrome, Photoshop, etc and they are, but what they really are just lots and lots of files that have been compiled into one. The people (or sometimes a single person) that write this software are known as upstream providers, they compile their code and write up how to get it installed. These upstream providers work on getting out new software and update existing software. When they are ready to release it to the world, they send their package to package maintainers, who handle getting this piece of software in the hands of the users. These package maintainers review, manage and distribute this software in the form of packages.

Exercise

No exercises for this lesson.

Quiz Question

No questions, move along!

2. Package Repositories

Lesson Content

How do packages that get uploaded to the internet somehow end up on our computers? Do you go to the download page of each package you want and click download and install? Well, actually you can do that, but there is something better called package repositories. Repositories are just a central storage location for packages. There are tons of repositories that hold lots of packages and best of all they are all found on the internet, no silly installation disks. Your machine doesn’t know where to look for these repositories unless you explicitly tell it where to look.

For example, let’s say I want WackyWidgets Software on my machine. Well WackyWidgets manages their own repositories for their widget packages, inside this repository are 10 packages, the CoolWidget package, the SuperWidget package, etc. WackyWidgets hosts this repository at a source link called: WackyWidgets

Now instead of going to their website to download the package directly, you can tell your machine to find WackyWidgets software from the source link.

Your distribution already comes with pre-approved sources to get packages from and this is how it installs all the base packages you see on your system. On a Debian system, this sources file is the /etc/apt/sources.list file. Your machine will know to look there and check for any source repositories you added.

Exercise

No exercises for this lesson.

Quiz Question

3. tar and gzip

Lesson Content

Before we get into package installation and the different managers, we need to discuss archiving and compressing files, because you will most likely encounter these when you hunt for software on the internet.

You probably already know what a file archive is, you’ve most likely encountered file types such as .rar and .zip. These are an archive of files, they contain many files inside of them, but they come in this very neat single file known as an archive.

Compressing files with gzip

gzip is program used to compress files in Linux, they end in a .gz extension.

To compress a file down:

$ gzip mycoolfile

To decompress the file:

$ gunzip mycoolfile.gz

Creating archives with tar Unfortunately, gzip can’t add multiple files into one archive for us. Luckily we have the tar program which does. When you create an archive using tar, it will have a .tar extension.

$ tar cvf mytarfile.tar mycoolfile1 mycoolfile 2

• c - create
• v - tell the program to be verbose and let us see what it’s doing
• f - the filename of the tar file has to come after this option, if you are creating a tar file you’ll have to come up with a name

Unpacking archives with tar

To extract the contents of a tar file, use:

$ tar xvf mytarfile.tar

• x - extract
• v - tell the program to be verbose and let us see what it’s doing
• f - the file you want to extract

Compressing/uncompressing archives with tar and gzip

Many times you’ll see a tar file that has been compressed such as: mycompressedarchive.tar.gz, all you need to do is work outside in, so first remove the compression with gunzip and then you can unpack the tar file. Or you can alternatively use the z option with tar, which just tells it to use the gzip or gunzip utility.

Create a compressed tar file:

$ tar czf myfile.tar.gz

Uncompress and unpack:

$ tar xzf file.tar

If you need help remember this: eXtract all Zee Files!

tar is one of those commands that is so important and yet you never really remember it, relevant xkcd: xkcd

Other Utilities

Throughout your journey of Linux, you’ll encounter other archive and compression types such as: bzip2, compress, zip, unzip, etc. They are a little less common, but just keep in mind that different utilities will call for different commands.

Exercise

Familiarize yourself with the tar documentation and look at the other options available in the manpage.

Quiz Question

4. Package Dependencies

Lesson Content

Packages very rarely work by themselves, they are most often accompanied by dependencies to help them run. For example, let’s say we have a group of restaurants, these restaurants all make different cuisine, however they all get their ingredients from the same farm. Their food is dependent on the farm’s supplies, if the farm were to suddenly stop supplying food, well then the restaurants would be in a pretty bad state.

In Linux, these dependencies are often other packages or shared libraries. Shared libraries are libraries of code that other programs want to use and don’t want to have to rewrite for themselves. Think of the restaurant again, how much work would it be if every restaurant also farmed their own food? Too much.

We will dig more into shared libraries in the filesystem course, so for now just remember that packages have dependencies to help them run, whether those dependencies are other packages or libraries, if the dependencies aren’t there the package will end up in a broken state and most of the time not even install.

Exercise

No exercises for this lesson.

Quiz Question

No questions, move along!

5. rpm and dpkg

Lesson Content

Although most of this course is about package management systems (the Batmans of package management), we mustn’t forget about the Robins. Although very useful and reliable, they don’t come with that sweet batmobile and utility belt.

Just like .exe is a single executable file, so is .deb and .rpm. You normally wouldn’t see these if you use package repositories, but if you directly download packages, you will most likely get them in these popular format. Obviously, they are exclusive to their distributions, .deb for Debian based and .rpm for Red Hat based.

To install these direct packages, you can use the package management commands: rpm and dpkg. These tools are used to install package files, however they will not install the package dependencies, so if your package had 10 dependencies, you would have to install those packages separately and then their dependencies and so on and so forth. As you can see, that was one of the reasons that brought forth the full blown management systems that we will discuss this later.

Keep in mind that there will be countless times when you need to install, query or verify a package with one of these tools, so remember these commands.

Install a package

Debian: $ dpkg -i some_deb_package.deb
RPM: $ rpm -i some_rpm_package.rpm

The i stands for install. You can also use the longer format of –install.

Remove a package

Debian: $ dpkg -r some_deb_package.deb
RPM: $ rpm -e some_rpm_package.rpm

Debian: r for remove RPM: e for erase

List installed packages

Debian: $ dpkg -l
RPM: $ rpm -qa

Debian: l for list RPM: q for query and a for all

Exercise

Find a program that you want to install on your system like Google Chrome and install it using one of these commands.

Quiz Question

6. Yum and Apt

Lesson Content

Ah, the Batmans of package management, these systems come with all the fixins to make package installation, removal and changes easier, including installing package dependencies. Two of the most popular management systems is yum and apt. Yum is exclusive to the Red Hat family and apt is exclusively to the Debian family.

Install a package from a repository

Debian: $ apt install package_name
RPM: $ yum install package_name

Remove a package

Debian: $ apt remove package_name
RPM: $ yum erase package_name

Updating packages for a repository

It’s always best practice to update your package repositories so they are up to date before you install and update a package.

Debian: apt update; apt upgrade
RPM: yum update

Get information about an installed package

Debian: apt show package_name
RPM: yum info package_name

Exercise

Run through each of these package commands and see the output you receive.

Quiz Question

7. Compile Source Code

Lesson Content

Often times you will encounter an obscure package that only comes in the form of pure source code. You’ll need to use a few commands to get that source code package compiled and installed on your system.

First thing is first, you’ll need to have software to install the tools that will allow you to compile source code.

$ sudo apt install build-essential

Once you do that, extract the contents of the package file, most likely a .tar.gz file.

$ tar -xzvf package.tar.gz

Before you do anything, take a look at the README or INSTALL file inside the package. Sometimes there will be specific installation instructions.

Depending on what compile method that the developer used, you’ll have to use different commands, such as cmake or something else.

However, most commonly you’ll see basic make compilation, so we’ll discuss that:

Inside the package contents will be a configure script, this script checks for dependencies on your system and if you are missing anything, you’ll see an error and you’ll need to fix those dependencies.

$ ./configure

The ./ allows you to execute a script in the current directory.

$ make

Inside of the package contents, there is a file called Makefile that contains rules to building the software. When you run the make command, it looks at this file to build the software.

$ sudo make install

This command actually installs the package, it will copy the correct files to the correct locations on your computer.

If you want to uninstall the package, use:

$ sudo make uninstall

Be wary when using make install, you may not realize how much is actually going on in the background. If you decide to remove this package, you may not actually remove everything because you didn’t realize what was added to your system. Instead forget everything about make install that I just explained to you and use the checkinstall command. This command will make a .deb file for you that you can easily install and uninstall.

$ sudo checkinstall

This command will essentially “make install” and build a .deb package and install it. This makes it easier to remove the package later on.

Exercise

Find a source code program (from a trusted site) and install from source.

Quiz Question