world leader in high performance signal processing
Trace: » operating_systems

Operating Systems

Introduction to Linux

Linux is an operating system that was initially created as a hobby by a young student, Linus Torvalds, at the University of Helsinki in Finland. Linus had an interest in Minix, a small UNIX system, and decided to develop a system that exceeded the Minix standards. He began his work in 1991 when he released version 0.02 and worked steadily until 1994 when version 1.0 of the Linux Kernel was released. The kernel, at the heart of all Linux systems, is developed and released under the GNU General Public License (GNU GPL) and its source code is freely available to everyone. It is this kernel that forms the base around which a Linux operating system is developed. There are now literally hundreds of companies and organizations and an equal number of individuals that have released their own versions of operating systems based on the Linux kernel.

Because of Linux's success on the x86 desktop, and the proliferation and porting to various processor architectures over the past two years, Linux has become very popular on embedded devices, especially consumer products, telecom routers and switches, Internet appliances, and industrial and automotive devices.

Using Linux in embedded devices which require some intelligence is attractive for many reasons:

  • it is a mature, robust operating system
  • it supports a large number of devices, file systems, and networking protocols
  • upgrades and new features are constantly being added, tested and refined by a large community of programmers and users
  • it gives everyone from developers to end users complete visibility of the source code
  • a large number of applications (such as GNU software) exist which require little to no porting effort
  • the low cost of Linux

Some examples of Linux in embedded systems can be found at Linux Devices.

What is Linux?

Linux is a monolithic, multitasking, multiuser operating system (OS) which is a UNIX workalike. It is modularly designed with well understood interdependencies, which allows one to tailor Linux for a variety of purposes. In this course, Linux exists on the development workstation as a full blown OS with an incredibly large variety of features. However, it also exists on the target system as a very small, focused OS appropriate for an embedded system.

An exciting attribute of Linux is that it can be run on a wide variety of microprocessors and microcontrollers. Once you are comfortable with Linux, you will find this to be a great advantage in the embedded systems arena. In particular, your development platform is a Linux box which can be used for many different embedded targets spanning a range of processor architectures, but all running Linux.

It is not the purpose of this introductory chapter to explore Linux in either great depth or breadth. The assumption is that the course participants are quite computer literate, but may have little previous experience with this particular OS. Hence our goal is to get up and running on a generic Linux box. To accomplish this the following sections will look at:

  • logging on to a Linux system
  • the file system
  • a selection of commands
  • editing text files

Then we'll close out the chapter with some activities/exercises followed by salient references.

Loggin On

If you are a new Linux user and have your own machine, then you will be able to log in as root or as a user. Our assumption here is that you are logging in as a user. Working as root (also called super user or system administrator) level should be reserved for tasks that really require it e.g. many system administration tasks. When Linux boots up, it either boots into the X Windows System (providing a GUI) or into console mode (no GUI). In either case, you will be prompted to log on. The logon is done in two steps in response to two successive prompts requiring that you:

  • enter your username
  • enter your password

So the system administrator must provide you a username and password ahead of time. Upon successful logon, the OS starts a shell which responds to your input. There are many possible shells with the bash shell being the default shell for most Linux distributions. However, an embedded system is likely to use a more compact and less powerful shell. The shell interprets commands you enter at the command line, but also is a powerful scripting language. In particular, the shell can interpret text files which are written using the appropriate shell's grammar. It is beyond the scope of this course to investigate shell scripting. At any rate, once successfully logged in, you can enter commands and, in the X Windows environment, point and click on various icons.

To log off, type exit at the command line. If you have your own personal Linux machine and need to power down, this must be done carefully. In particular, you need to be logged on with root privileges (don't exit) and then enter at the command line shutdown -h now There are variations on this (see man shutdown).

The File System

Linux, like most operating systems, has a hierarchical file structure. Application developers would like sufficient standardization in that hierarchy (and in other aspects of Linux) that they need not tailor their application's installation and operation for each style of Linux distribution. There is an evolving Linux Standard Base and perhaps the most mature component is the file system hierarchy. However, the space and performance constraints of an embedded system will force an embedded Linux to stray from the standard. Nor is that particularly serious; an embedded system is a special purpose system, less likely to cause an application developer pain. In this section we'll look at the file system hierarchy standard and then at the hierarchy found in uClinux, as an example of an embedded system. Note that our embedded system development will typically be on the Linux workstation where the standard is hopefully followed. At the same time an understanding of the uClinux hierarchy will give us insight into the embedded Linux world.

The File System Hierarchy Standard

The File System Hierarchy Standard, which at this writing is at Version 2.3 can be found here. We'll give an overview of what we need for this course, but you are encouraged to look at the standard for the underlying rationale.

The root of the file hierarchy is symbolized simply by /. There is some confusion possible because one of the top level subdirectories under / is called root, but the latter is essentially a home directory for the root user (system administrator). The immediate subdirectories of / are described in the following table. The File System Hierarchy Standard discusses each of these directories in some detail, not only the contents, but accepted conventions for their usage. Of course, the subdirectories contain files and further subdirectories and so on.

Note that a directory is actually one of several kinds of files from the perspective of the OS, but it is convenient to make the distinction between directories and other files here. All common distributions allow one additional directory at the same level as those of Table 1, the proc directory. It is not part of the standard, but deserves mention. It is often called a pseudo directory. It contains information about the executing kernel. The files hold data on such things as the resource usage of current processes, what CPU is present, interrupt allocation, I/O port allocations, pci bus components, devices, modules currently installed, and so on. Much of the information is generated dynamically when it is accessed.

Directory Name Purpose/Contents
binessential command binaries which may be used by the system administrator and by ordinary users, required for system
bootboot kernel image and configuration files used by the boot loader
devdevice files
etchost-specific configuration files
homeuser home directories
libessential shared libraries and kernel modules
mnt mount point for temporarily mounting file systems such as those on a CDROM or floppy disk
opt add-on application software packages
rootthe root user's home directory
sbinsystem binaries, essential for system administration, but not for system boot
tmplocation of temporary files
usrsecondary hierarchy, intended as shareable, read- only data. Here you'll find games, include files used by C programs, the linux source code hierarchy, and so on
varvariable data such as spool directories and log files.

The file system hierarchy in uClinux

The directories at the top level of the unmodified uClinux hierarchy are tabulated next. As would be expected, these 'standard' directories are much sparser than a workstation resident Linux. Keep in mind that we do our development on our workstation rather than on the target. For example, if we write a C program to incorporate into our STAMP/uClinux target, it is written and compiled on the workstation - so there is no need for the C program include files to be in the uClinux hierarchy.

Pathnames

If your username is, say, jsmythe, then when you log on you will be placed in jsmythe/ as your current working directory. Where is this located with respect to the root of the file hierarchy? The top level under root contains the subdirectories of Table 1, which includes the home subdirectory.

user home directories including jsmythe/ (the trailing slash indicates a directory, but may be omitted at the end of a pathname). Designating the root of the file system by a leading, solitary slash (/), we find the jsmythe log on directory by navigating like so / → home → jsmythe. This is compacted as a pathname

/home/jsmythe If you used an editor to create the file vita.txt in this home directory, it would be found at /home/jsmythe/vita.txt

/ is called the parent directory of home, home/ is the parent directory of jsmythe, and jsmythe/ is the parent directory of vita.txt.

A pathname that starts at the root of the file hierarchy starts with / and is called an absolute pathname. Clearly if you are 42 levels deep in the hierarchy, absolute pathnames become unwieldy. Hence, Linux also provides relative pathnames. Let's look at some examples of relative pathnames. If you have logged on to your home directory at /home/jsmythe, where you have created a file vita.txt, then its absolute pathname is, as before, /home/jsmythe/vita.txt but working from within the home directory /home/jsmythe/, you can refer to this file as vita.txt or as ./vita.txt The dot (.) indicates that the path starts at the current working directory; so, in our example, it effectively replaces /home/jsmythe. Hence, when working within /home/jsmythe, all three of the following are equivalent: /home/jsmythe/vita.txt ./vita.txt vita.txt. Similarly the double dot (..) represents the parent directory of the current working directory.

File Permissions

File permissions are part of the security system. They help

  • protect the OS from users, whether malicious or clumsy
  • protect each user from other users

Permissions cover

  • reading a file
  • writing a file
  • executing a file

Further, a file has three user categories

  • the file's owner
  • the file's group
  • all other users

File permissions are written as three concatenated triads. The first triad refers to the file owner, and the second to the file group, and the third to all other user's. Here are some example permissions and what they mean:

rwxrwxrwx - file owner, file group, and all others have read/write/execute permission.

rw-rw—- - file owner and file group have read/write access, but all others have no access.

rwxr-xr-x - file owner has read/write/execute access, while group and all others have read/execute access.

Setuid and Getgid Programs

The three concatenated permission triads (for owner, group, and others) are preceded by another field whose purpose it is to give the user of the program special permissions. In particular, the user's effective uid (user ID) or effective gid (group ID) can be set to the file owner's uid or gid. As a somewhat dated example consider a user changing his/her password in the days before shadow passwords became prevalent. The passwords were kept in /etc/passwd with permissions rw-r--r-and root as owner. So the average user does not have write access to change a personal password. However, the /usr/bin/passwd command has permissions r-s--x--x where the s means that the user's effective uid is set to that of root and will be allowed to update /etc/passwd. The following commands can be used to set the user's effective id and effective gid to that of the file's owner

chmod u+s program … we now say the program is a setuid program

chmod g+s program … we now say the program is a setgid program

We'll see the chmod command in the next section.

Commands

Here we'll look at some useful commands which perform such tasks as file system manipulation and navigation, getting information, and modifying file permissions. Many of the commands care about permissions and the command will fail if you don't have the proper permissions. Linux, although relatively new, profits from the historical evolution of UNIX in that it comes close to achieving the following goals:

  • there is a command or combination of commands to perform any task you can dream up
  • the securely configured system prevents you from harming the system or other users

Part of the philosophy of Linux (again, inherited from UNIX) is that commands tend to be very simple, but can be combined very compactly to do complex tasks. Note: Linux is case sensitive so Cat, CAT, and cat would be distinct - our typical commands will be lower case. In this chapter, we'll consider some useful commands and then look at what might be called plumbing - standard input, standard output, standard error, redirection, and pipes. These plumbing operators are not commands, but rather operators provided by the shell. They are used in concert with the commands, allowing the commands to be used in combination.

Some useful commands

From the many commands provided by Linux, we'll consider those tabulated next. We'll discuss these only briefly, and in most cases they have many further options which can be explored via the man pages.

Command Purpose
lslists directory contents
cdchanges from the current working directory to another
pwdshow pathname of present working directory
mkdirmake a directory
rmdirremove a directory (directory must be empty)
rmremoves a file, but has a recursive option that can be used to remove non empty directories as well
cpcopies a file
mvmoves its argument (a file name) to a new name
morelists its argument a page at a time
catconcatenates its arguments and sends the result to standard out
grepprint lines matching a pattern (regular expression)
tararchives its argument (with compression if specified)
mandisplays the manual pages of its argument
suswitch to another user
chmodchanges file/directory permissions
chownchanges file/directory owner
chgrpchanges file/directory group.

The ls command

If you want to list the contents of your current working directory, you can simply type ls and if you want lots of information (e.g., permissions, sizes), use the long form ls -l As you would expect, you can list contents of other directories (as allowed by the permissions) e.g. ls -l /usr/src Other options allow sorting on such information as date etc.

The cd command

This allows you to change your working directory to another, if permissions allow. Here are some examples:

cd / change to the root directory of the file system

cd /usr/src change to /usr/src

cd .. change to the parent of your current directory, i.e. one level up

cd ../.. change two levels up

cd - change back to the directory you were in before this one

The pwd command

This is the 'present working directory' command. It displays the absolute pathname of where in the directory hierarchy you currently are.

The mkdir and rmdir command

These can make and remove (empty) directories. Assuming our current working directory is /home/jsmythe, consider these sequential examples:

mkdir emails - creates the directory /home/jsmythe/emails

mkdir emails/outgoing - creates directory /home/jsmythe/emails/outgoing

rmdir emails/outgoing - removes directory /home/jsmythe/emails/outgoing.

The rm command

This command allows you to delete files. For example, if you are in directory /home/jsmythe and want to delete a file at that level named vita.txt, you could enter rm vita.txt This is a very powerful command in that you can delete a whole chunk of a hierarchy. For example, if /home/jsmythe/project1/ was the top of a significant hierarchy of subdirectories and files, you can delete it via rm -rf /home/jsmythe/project1

The cp and mv command

If you want a copy of a program, the format is cp existing new for example, cp vita.txt vita1.txt Now there are two files, the original (vita.txt) and a new copy (vita1.txt). On the other hand, if you want to rename a file, the syntax is mv existing new for example mv vita.txt resume.txt In this case, the file vita.txt has been replaced by the identical file, except that it has a new name, resume.txt. The mv command can also be used to rename any node in the file hierarchy e.g. mv project/ project1

The more command

This command can display a file at the console, one screen full at a time. For example, more vita.txt Once invoked, more provides some helpful key strokes:

  • space key - display next screen
  • q key - quit

A similar, but more powerful command is less.

The cat command

The cat command concatenates its arguments and displays the entire result at the console, so a multi-screen result will quickly scroll off the end. For example, cat file1 file2

The grep command

The grep command can search a file or set of files for a pattern. For example, let's say we're searching for mention of sys_write in any of the C files in a directory. This could be done via grep 'sys_write' *.c Then grep would print all lines in all files mentioning the sought for pattern, sys_write. There is also a recursive grep (rgrep) available which can search recursively through a directory hierarchy. While we're on the subject of searching, there is a wonderful tool for searching through the source hierarchy which can be used over the web or downloaded. See http://lxr.linux.no

The tar command

We'll just give two examples of this convenient and powerful command. Let's say you have a project at work which you also pursue at home and are faced with moving the project back and forth as you work on it in both places. Let's say the project hierarchy starts at the node /home/jsmythe/project/ and consists of many subdirectories and files. To compress and archive the hierarchy into a single file called proj.tgz, use the command tar cvfz proj.tgz project/ where we have assumed that the command is issued from directory /home/jsmythe. To subsequently expand and unarchive the file on your other machine from directory /home/jsmythe, use tar xvfz proj.tgz and it will produce the /home/jsmythe/project hierarchy if it does not currently exist and append to it if it does.

Linux provides a powerful tool called cvs for help in managing projects which have multiple developers.

The man command

If a manual page on some entity, name, exists, then man name will give the information. For example, if you want to learn what options are supported for the ls command, enter man ls or to learn about the vi editor, enter man vi. You can even learn more about man by entering man man.

The su command

The switch user command (sometimes misnamed the super user command) allows you to switch to a different user. If no argument is supplied, you will switch to super user if you successfully respond to the subsequent prompt for the super user password. If an argument (a valid username) is provided then you will switch to that user if you successfully respond to the subsequent prompt for that user's password. You return to your previous identity with the exit command. As you can see, it is essentially a nested log on scenario.

In this document there are several command line instructions given that must be run as root. To switch to the root user, without having to logout from a user session, the switch user command may be used as follows:

bash$ su -

You will then be prompted to enter the password for root access, if this is successful a root session will be started. You can exit back to the user session by typing the command exit. If you wish to run graphical programs which use the XWindows interface then use the sux - command instead. This transfers your XWindows credentials.

Alternately, to execute a single command as root use the sudo command as follows:

bash$ sudo <command to execute as root>

If this is your first time running sudo in this session you will be prompted for a password. After successfully entering the password your command will be run as root.

The chmod, chown, and chgrp command

These allow you to change permissions (chmod), ownership (chown), and group (chgrp) of files to which you have appropriate access. There are typically used for system administration. For example, as root I could change a file's owner and group to fred_smith (assuming there is such a user and group) via chown fred_smith.fred_smith filename Similarly, if an owner wants to change a file's permissions to rw-rw-r--, the appropriate command would be chmod 664 filename For those uncomfortable with octal permissions (like 664 representing rw-rw-r--), there is the option of using letter based syntax. Investigate these commands further via their man pages.

Standard Input, Output, Error

Commands and other programs often expect input to come from the keyboard, which we call standard input. Similarly they often send output to the console, which we call standard output. Error information is also typically sent to the console, but the destination is then called standard error. We'll see in the next section that the distinction between standard output and standard error is not as trivial as it first seems. A command such as more vita.txt sends its output to standard output, the console.

Redirection and Pipes

The simple concepts of standard input, output, and error become significantly more powerful when used with redirection and pipes. Some examples are in order.

Redirection

Recall that the command cat file1 file2 will concatenate file1 and file2, sending the output to standard out, the console. However, standard output can be redirected to a file other than the console e.g. cat file1 file2 > file3 where the right arrow (>) is the redirection operator for standard output. In this case file3 will hold the concatenated result of file1 and file2. In the case just shown file3 is created and any existing file by that name would have been destroyed. If file3 already existed and you wanted to append the new information, you would enter cat file1 file2 >> file3 where the double right arrow is called the nondestructive redirection operator. Similarly, the left arrow (<) is the redirection operator for standard input. There is also a syntax (somewhat shell dependent) for redirecting standard error. For example, in the bash shell we might perform a compilation with this syntax:

gcc file5.c -o file5.c 2> file5.err

where the last piece 2> file5.err redirects standard error (but not standard output) to the file called file5.err. This is quite useful when your compilation triggers many errors which scroll down the screen. Typically, the early errors are significant, but may scroll off the screen. However, they will be available in a redirected error file such as the example's file5.err.

To capture all the output of a command into a log file use the following syntax:

gcc file5.c -o file5.c > file5.log 2>&1

This directs the output of the compile to file5.log and also sends the error output to the same output file.

These redirection operators appear very handy, but when combined with the pipe concept the power is much enhanced.

The Pipe

The pipe connects commands so the output of one becomes input for the second. Let's look at its use without redirection, first. Consider using ls -l on a very large directory. It would scroll off the screen. However you can pipe it to more, which you'll recall, displays one screen at a time. The syntax is ls -l | more where the vertical bar (|) is the pipe operator. You can pipe several commands together and combine them with redirection. Here is a slightly more involved example,

cat file1 file2 | sort > file3

which does these tasks in order

  • concatenates file1 and file2
  • sends the result to the sort command, which sorts the lines in alphabetical order
  • stores the alphabetized, concatenated result as a new file called file3

The above set of tasks might be given to beginning programming students who would then write some high level language program to get the result. Here we see how the UNIX philosophy of simple commands used in combination can do complex tasks.

If you require more information on the Linux operating system a good introductory guide is available here.

Activities/Excercises

The exercises assume you have read through the entire chapter. These excercises should be completed on the host or development PC.

Logging on

Log on your workstation. Try the command

ls -l

and then

ls -al

What does the additional a switch do? [see man ls]

Try the command

sh -version

What does this tell you?

Standard Directories

Using the command ls -F, check out the contents of each of the standard directories from Table 1 of this chapter. For example, try ls -F /dev and so on. What does the F switch do?

Standard Directories

Look at the contents of the /proc directory. Examine these files in /proc with cat:

  • cpuinfo
  • meminfo
  • interrupts
  • ioports
  • pci

If any of the cat displays are more than a screen full, pipe the display through more e.g. cat /proc/pci | more

More about ls

Try ls with the f and t options:

ls -f or ls -lf

ls -t or ls -lt

What do these options do?

mkdir and tree

Using mkdir, make a directory hierarchy in your home directory e.g.

mkdir projects

mkdir projects/proj0

mkdir projects/proj1

and so on, adding breadth and depth. Then try the tree command, if it's present on your system, e.g. tree projects

tar

Archive and compress your directory hierarchy from the prior problem i.e.

tar cvfz proj.tgz projects/

Now try to remove the directory e.g.

rmdir projects/

Why didn't that work?

Now try

rm -rf projects/

Was it removed this time?

Now restore it via

tar xvfz proj.tgz

In the two instances where we used tar, the switches used were c, v, f, z, and x.

What does each mean?

Use vi

Create two different multi line files with vi, file1.txt and file2.txt. We'll use them subsequently.

The cp and mv commands

Try the cp command:

cp file1.txt file3.txt

cp file2.txt file4.txt

Use ls -l to verify that the new files were made. Then try

mv file3.txt file5.txt

mv file4.txt file6.txt

Now what does ls -l tell you?

Using chmod, chown, chgrp

See the man pages for these commands. Make some files and try them out.

Using pipes

Try something like

cat file1.txt file2.txt | sort > file12.txt.

Describe what this command has done. Now try this

ls -l | wc -w

What does this do? Does the result make sense? [see man wc]

Complete Table of Contents/Topics