This blog is about the System Administration in Linux, this may help in working with Linux as a SYSTEM ADMINISTRATOR and the topics include duties of them and how to manage also.
Duties of the Administrator:
Computer Systems Administrator Duties:
The qualified individual will administer research computer systems and networks, conducting hardware installation, operating system maintenance, and ensuring smooth operations for a network of computers, running under UNIX, MS Windows, and Macintosh operating systems, used in research and teaching. The administrator will install new hardware as needed, network client machines and servers, maintain secure user accounts, maintain system backups, promptly upgrade software to current releases, install application software, coordinate software and hardware maintenance agreements, maintain network protocols in keeping with University specifications, supervise and train personnel, engineer software tools and attend regular campus system administrator meetings. The expanding role of computers in biological research and teaching, plus the commitment of the Department of Biology and its research components to the support of state-of-the-art computer resources, creates opportunities for professional development in this position.
Administration Tools: Linux old-timers revel in reminding newcomers that they used to have to do everything by hand, at the command line, uphill, both ways, with duct tape for shoes. What really gets some of these folks sputtering is today's collection of system administration tools that introduce quite a bit of automation. There's good reason for this, actually; if you don't know how to administer your system by hand then you are sunk if something goes wrong. However, this factor doesn't mean you shouldn't take advantage of available helping hands.
There are four major players in the world of Linux system administration tools: COAS, Linuxconf, Webmin and YaST. One of these, YaST, is tied specifically to SuSE Linux. The other three, COAS, Linuxconf and Webmin, come by default with some distributions but are independently available for download and installation.
System Administration
Administration tools:
There are four major players in the world of Linux system administration tools: COAS, Linuxconf, Webmin and YaST. One of these, YaST, is tied specifically to SuSE Linux. The other three, COAS, Linuxconf and Webmin, come by default with some distributions but are independently available for download and installation.
Linuxconf
Linuxconf (Figure 1) comes with Mandrake Linux and Red Hat Linux, but is also available for most modern Linux distributions. You've probably encountered this tool before if you use one of these distributions, either as the whole package or in one of its modular components. Multiple interfaces for Linuxconf have been available for years, but now we're up to four: GUI, Web, command-line and ncurses.
Figure 1. The GNOME Version of Linuxconf
Linuxconf has actually been around for years, which means its bugs have had longer to shake out than the other distribution-neutral tools. You can download and find out more about this tool at www.solucorp.qc.ca/linuxconf. Be sure to read through what each portion of the Linuxconf package is used for. There is a base package with the non-GUI components, and then there are various GUI front-end pieces from a more general X Window System version to one specifically built for GNOME.
Whether you stick with the command-line version or add a GUI front end, you run the tool by typing linuxconf. From here you navigate text or point and click menus to access a wide variety of system settings, everything from basic networking details to GRUB configuration. Linuxconf also plays well with people who refuse to use the root account for anything but the most vital tasks. If you try to run it as root, the tool simply asks for the root password--if this fact makes you nervous, then you may want to consider not using this tool, but this practice is fairly standard in modern administration tools. When you consider that anyone could just try to su to the root account at whim, you start to see why it is so important to have a secure root password in place.
Webmin
Webmin (Figure 2) comes with, and was recently acquired by, Caldera Linux. This tool is not only available for most modern Linux distributions, it also runs on most major flavors of UNIX and is available in around twenty languages (though some modules are not available in all of the languages). As you might guess, Webmin is purely a web-based application and a heavily modular one at that.
Figure 2. Webmin in Red Hat Linux 7.2, under GNOME
There is a set of core modules that handle the usual system administration functionality, and then there are the third-party modules available for administering a variety of packages and services. To download and learn more about Webmin, point your web browser to www.webmin.com/webmin. Once again, this package is available in a number of formats specific to different distributions.
Where any user can install Linuxconf, Webmin must be installed by root. After that you can access this tool from any user account as long as you know the root password.
There are three separate rows of icons on this tool's front page. On the upper right, you have a pair of administrative links, one to log you out of the Webmin tool and another that allows you to fill out a feedback form that sends your comments back to the Webmin team. In the same top row on the upper left you can click on the word Webmin and go to the product home page. On the upper bar directly beneath those links, there are a series of menu icons, which are, from left to right:
· Webmin: takes you back to the main Webmin screen.
· System: a collection of configuration issues, such as user and group manipulation, disk quotas and cron jobs.
· Servers: configuration routines for a number of servers you may have installed on your system, such as Apache, WU-FTPD and sendmail.
· Hardware: configuration utilities for hardware issues such as RAID, printers and disk partitions.
· Cluster: a collection of cluster maintenance tools.
· Others: a set of tools that system administrators typically need, such as a command prompt, an alias manager and a file manager.
Finally, there is the Webmin tab, which has a series of Webmin management tools:
· Webmin Actions Log: if you've enabled Webmin logging, this function allows you to search through the logfiles for what you've utilized this tool to do to your system.
· Webmin Configuration: takes you to the amazing number of configuration options available for Webmin, everything from strengthening your Webmin authentication requirements to upgrading either the main package or individual modules.
Yet Another System Tool (YaST)
YaST (Figure 3) and its cousin YaST2 (Figure 4) come with SuSE Linux; these items are specific only to a single distribution.
Figure 3. SuSE's YaST
Figure 4. SuSE's YaST2 in KDE
Generally speaking, you'll want to use YaST2 when at all possible; YaST2 is the graphical version, and the older YaST is a great fallback if you aren't able to get into the GUI or have not installed a GUI. YaST2 is laid out in a standard file-manager format, with the menu of categories on the left and icons for the various configuration routines on the right, which change to correspond with your category choice. The categories are:
· Software: the selection of SuSE software-management utilities, such as the ability to update your system over the Internet or add and remove packages from the SuSE CD-ROM or DVD-ROM.
· Hardware: a selection of hardware configuration routines, including printers, sound cards and scanners.
· Network/Basics: the selection of configuration tools formodem and other connectivity devices, Ethernet cards and more.
· Network/Advanced: the section where you can configure many of your network services, such as sendmail, routing and NIS+.
· Security&Users: the selection of user and group-management tools, as well as a couple of useful security tools.
· System: the selection of overall system-configuration tools, including a boot script (rc-config) file editor, boot loader configuration editor and routines for changing the language, keyboard and so on.
· Misc: a collection of tools that couldn't be otherwise grouped into the other categories. Some of the items represented here involve alternate print setup tools, some for working with log files, and some for communicating with SuSE.
The Caldera Open Administration System (COAS)
COAS comes in a rudimentary form with Caldera OpenLinux but is available for most modern Linux distributions, and is open source and covered by the GPL. This tool comes in four different formats, so you can use the interface that's most comfortable: command-line, ncurses (command-line but menu driven), GUI and Web for remote use. This tool is still under development but will be included in its entirety in later versions of Caldera OpenLinux--though there are rumors that because Caldera now has a stake in Webmin, COAS may be on its way out. This concern is born out by the fact that the COAS web site doesn't seem to have been updated since September of 1999.
This tool is modular, meaning that rather than loading its full set of features into memory when you start it, only the portions you're using come into play. Adding and removing modules is typically transparent as you work with the tool unless you require an external module, one made by a third party or one that for some reason is not included with the core release.
Right now the system is in development, specifically using KDE's Qt library set, but rumor has it that a GTK (GNOME) version is in the works. You can find out more about this tool by visiting http://www.coas.org/, including the list of the tool's anticipated capabilities.
Dee-Ann LeBlanc is a Linux writer, trainer, course developer and consultant who never seems to be able to stick with doing just one thing.
UNIX vendors all offer their own GUI administrative tools that are generally useful, provided that you do not have to do something that the vendor did not anticipate. These tools vary widely in how they work and how they are implemented.
AIX
IBM's UNIX operating system, AIX, comes with a sophisticated tool called SMIT. Administrators can use SMIT to configure the system, add and remove users, and upgrade software among other things. SMIT is widely considered to be the best and most mature of the system administration systems, because it can be customized and run in either X-windows or a terminal session. It also allows the user to view the command line equivalent of each task before it is performed. The downside (in at least some system administrator's opinions) is that use of SMIT is just about mandatory for some basic administration tasks.
HP-UX
Hewlett Packard's HP/UX has a similar tool called SAM, which provides much of the functionality offered by SMIT but is not quite as powerful or sophisticated. Its use is not required to administer the system, however.
Solaris
Sun's Solaris does not come with a tool comparable to SMIT or SAM. However, Sun's individual tools for upgrading and installing software and administering NIS+ are functional and intuitive. Unfortunately, the tool supplied with Solaris for administering users, printers, and NIS/NIS+ requires X-windows. It is not, however required to administer the system at all.
Overview of permissions:
File and Directory Permissions
Earlier in this chapter, you saw that the ls command with the option -al displayed the permissions associated with a file or a directory. The permissions associated with a file or a directory tell who can or cannot access the file or directory, and what the user can or cannot do.
In UNIX, each user is identified with a unique login id. Additionally, multiple users can be grouped and associated with a group. A user can belong to one or more of these groups. However, a user belongs to one primary group. All other groups to which a user belongs are called secondary groups. The user login id is defined in the /etc/passwd file, and the user group is defined in /usr/group file. The file and directory permissions in UNIX are based on the user and group.
All the permissions associated with a file or a directory have three types of permissions:
- Permissions for the owner: This identifies the operations the owner of the file or the directory can perform on the file or the directory
- Permissions for the group: This identifies the operations that can be performed by any user belonging to the same group as the owner of the file or the directory.
- Permissions for world: This identifies the operations everybody else (other than the owner and members of the group to which the owner belongs) can do.
Using the permission attributes of a file or directory, a user can selectively provide access to users belonging to a particular group and users not belonging to a particular group. UNIX checks on the permissions in the order of owner, group, and other (world)--and the first permission that is applicable to the current user is used.
Here is an example of a file called testfile in the current directory, created by a user called guhas belonging to a group called staff. The file is set up so that only the user guhas can read, modify, or delete the file; users belonging to the group can read it, but nobody outside the group can access it. Executing the following command from current directory
ls -al testfile
displays the permissions of the file testfile:
-rw-r----- 1 guhas staff 2031 Nov 04 06:14 testfile
You should be careful when setting up permissions for a directory. If a directory has read permissions only, you might be able to obtain a list of the files in the directory, but you will be prevented from doing any operations on the files in that directory.
For example, if you have a directory called testdir in the current directory, which contains a file called testfile, and the group permissions for testdir is read-only, executing the following command
ls testdir
will display the result
testfile
However, if you want to see the content of the file testfile using the following command:
cat testdir/testfile
you will get the following error message:
cat: testdir/testfile permission denied
To perform any operation on testfile in testdir, you must have the execute permission for testdir.
If you want all the members in your group to know the names of the files in a particular directory but do not want to provide any access to those files, you should set up the directory using only read permission.
The owner of a file is determined by the user who creates the file. The group to which the file belongs is dependent on which UNIX system you are working on. In some cases, the group is determined by the current directory. In other cases, you might be able to change to one of you secondary groups (by using the newgrp command) and then create a file or directory belonging to that group.
Similarly, if you set up a directory with just execute permission for the group, all members of the group can access the directory. However, without read permission, the members of the group cannot obtain a list of directories or files in it. However, if someone knows the name of a particular file within the directory, he or she can access the file with the file's absolute pathname.
For example, let us assume that we have a sub-directory testdir under /u/guhas that has a file called testfile. Let us assume the sub-directory testdir has been set up with 710 permission (that is execute permission for the group). In such a case, if a member of the group executes the ls command on testdir, the following will be the result
ls -l testdir
testdir unreadable
total 0
while if someone is aware of the file testfile and executes the following command
ls -l testdir/testfile
-rw-r--r-- 1 guhas staff 23 Jul 8 01:48 testdir/testfile
then he or she will get all the information about the file testfile.
In UNIX, there is a special user who has blanket permission to read, write and execute all files in the system regardless of the owner of the files and directories. This user is known as root.
The Permission Bits
You know that files and directories have owners and groups associated with them. The following are three set of permissions associated with a file or directory:
- Owner permission
- Group permission
- World (other) permission
For each of these three types for permissions there are three permission bits associated. The following is a list of these permission bits and their meanings for files:
- Read (r): The file can be read.
- Write (w): The file can be modified, deleted, and renamed.
- Execute (x): The file can be executed.
The following is a list of these permissions and their meanings for directories:
- Read (r): The directory can be read.
- Write (w): The directory can be updated, deleted, and renamed.
- Execute (x): Operations may be performed on the files in the directory. This bit is also called the search bit, because execute permission in a directory is not used to indicate whether a directory can be executed or not but to indicate whether you are permitted to search files under the directory.
Let us examine the directory permissions more closely. Suppose there is a sub-directory called testdir under the directory /u/guhas with the following permissions:
drwxrws--- 3 guhas staff 1536 Nov 4 06:00 testdir
Also a file called testfile is in the directory testdir with the following permission:
-rwxr----- 1 guhas staff 2000 Nov 4 06:10 testfile
This means that the user guhas can read, modify, and rename the directory and files within the directory. Any member of the group staff also has access to the directory. The file testfile is set up with read permissions only for all members of group staff. However, because all members of staff have read, write, and execute permissions on testdir, anyone belonging to group staff may modify, delete, and rename the file testfile.
CAUTION: If a user has write permissions to a directory containing a file, the permissions of the files in that directory are overridden by permissions of the directory.
Permissions (for owners, groups, and others) are stored in the UNIX system in octal numbers. An octal number is stored in UNIX system using three bits so that each number can vary from 0 through 7. Following is how a octal number is stored:
- Bit 1, value 0 or 1 (defines read permission)
- Bit 2, value 0 or 1 (defines write permission)
- Bit 3, value 0 or 1 (defines execute permission)
The first bit (read) has a weight of 4, the second bit (write) has a weight of 2, and the third bit (execute) has a weight of 1. For example, a value of 101 will be 5. (The value of binary 101 is (4 * 1) + (0 * 1) + (1 * 1) = 5.)
Let us now examine how to use the octal number to define the permissions. For example, you might want to define the following permissions for the file testfile in the current directory:
- Owner read, write, and execute
- Group read and execute
- Others--no access at all
This can be defined as (using binary arithmetic):
- Owner 111 = 7
- Group 101 = 5
- Others 000 = 0
Thus, the permission of the file testfile is 750.
Some versions of UNIX provide an additional bit called the sticky bit as part of a directory permission. The purpose of the sticky bit is to allow only the owner of the directory, owner of the file, or the root user to delete and rename files.
The following is a convention for setting up permissions to directories and files. For private information, the permission should be set to 700. Only you will have read, write, and execute permissions on the directory or file.
If you want to make information public but you want to be the only one who can publish the information, set the permission to 755. Nobody else will have write access, and nobody else will be able to update the file or directory.
If you do not want the information to be accessed by anybody other than you or your group, set the permission for other 0. The permission may be 770 or 750.
The following is an example of where you can set up permissions to deny permissions to a particular group. Assume that there is a directory called testdir in the current directory owned by a group called outsider. If you execute the following command in the current directory, the group outsider will not be able to perform any function on the directory testdir:
chmod 705 testdir
Default Permissions: umask
When a user logs into a UNIX system, she is provided with a default permission. All the files and directories the user creates will have the permissions defined in umask.
You can find out what the default permissions you have by executing the following command:
umask
It might display the following result:
022
umask is stored and displayed as a number to be subtracted from 777. 022 means that the default permissions are
777 - 022 = 755
That is, the owner can read, write, and execute; the group can read and execute; and all others can also read and execute.
The default umask, usually set for all users by the system administrator, may be modified to suit your needs. You can do that by executing the umask command with an argument, which is the mask you want. For example, if you want the default permissions to be owner with read, write, and execute (7); group with read and write (5); and others with only execute (1), umask must be set to 777 - 751 = 026. You would execute the command as follows:
umask 026
Changing Permissions: chmod
You have just seen how the default permissions can be set for files and directories. There might be times when you will want to modify the existing permissions of a file or directory to suit your needs. The reason for changing permissions might be that you want to grant or deny access to one or more individuals. This can be done by using the chmod command.
With the chmod command, you specify the new permissions you want on the file or directory. The new permissions can be specified using one the following two ways:
- In a three-digit, numeric octal code
- In symbolic mode
You are already familiar with the octal mode. If you wanted the file testfile to allow the owner to read, write, and execute; the group to read; and others to execute, you would need to execute the following command:
chmod 741 testfile
When using symbolic mode, specify the following:
- Whose (owner, group, or others) permissions you want to change
- What (+ to add, - to subtract, = to equal) operation you want to perform on the permission
- The permission (r, w, x)
Assuming that the current permission of testfile is 740 (the group has read-only permission), you can execute the following command to modify the permissions of testfile so that the group has write permissions also:
chmod g+w testfile
Another example of symbolic mode is when you want others to have the same permissions as the group for a file called testfile. You can execute the following command:
chmod o=g testfile
Another example of symbolic mode is when you want to modify the permissions of the group as well as the world. You can execute the following command to add write permission for the group and eliminate write permission for the world:
chmod g+w, o-w testfile
Changing Owner and Group: chown and chgrp
If you wanted to change the owner of a file or directory, you could use the chown command.
CAUTION: On UNIX systems with disk quotas, only the root user may change the owner of a file or directory.
If the file testfile is owned by user guhas, to change the ownership of the file to a user friend, you would need to execute the following command:
chown friend testfile
If you wanted to change the group to which file belongs, you may use the chgrp command. The group must be one of the groups to which the owner belongs. That is, the group must be either the primary group or one of the secondary groups of the owner. Let us assume that user guhas owns the file testfile and the group of the file is staff. Also assume that guhas belongs to the groups staff and devt. To change the owner of testfile from staff to devt, execute the following command:
chgrp devt testfile
Setuid and Setgid
When you execute some programs, it becomes necessary to assume the identity of a different user or group. It is possible in UNIX to set the SET USER ID(setuid) bit of an executable so that when you execute it, you will assume the identity of the user who owns the executable. For example, if you are executing a file called testpgm, which is owned by special user, for the duration of the execution of the program you will assume the identity of special user. In a similar manner, if SET GROUP ID(setgid) of a executable file is set, executing that file will result in you assuming the identity of the group that owns the file during the duration of execution of the program.
Here is an example of how the SET USER ID bit is used. Suppose you wanted a backup of all the files in the system to be done by a nightshift operator. This usually is done by the root user. Create a copy of the backup program with the SET USER ID bit set. Then, the nightshift operator can execute this program and assume the identity of the root user during the duration of the backup.
Processes:
Process status, Killing processes, and process priority:
A process is a running program to which the Linux kernel has assigned a share of your system's resources or computing power. A process that starts when the system starts and runs in the background until needed is a daemon. Because Linux is a multi-tasking operating system, dozens of processes of all sorts can be running at any one time.
Some of the reasons that you may want to check the processes that are running include:
- To Troubleshoot: Your system is not responding properly. The processes, the combination of processes, and the resources used by processes should all be examined as a routine part of troubleshooting.
- To check system security: If you suspect an intruder on the system, the processes used may provide a clue. You should also check what user or group id a process is running under. In particular, check programs running by the root user and see if anything is out of the ordinary.
- To close a frozen or strangely behaving program: While Linux itself is stable, the desktop has developed rapidly, and some programs that run under the X Window System may behave erratically. You may also need to close a program while testing it.
Understanding Processes
Linux uses three basic types of processes:
- Interactive: An ordinary program started by a user. Many commands entered from the command line are interactive processes. So is each new window opened in the X Window system.
- Batch: Any process run without a command line terminal.
- Daemon: A process that runs continuously. Usually, they are started when the system starts, but they can also be started by the root user. Daemons generally run in the background.
You should also know:
- A major characteristic of each process is its priority. If system resources are short, the priority determines what share of the resources the process uses. In extreme cases of resource shortages, the priority determines the order in which processes can use system resources.
- Processes can be hierarchal. For example, the printing daemon is always running, but a new copy of it is started for an individual print job, then closed after the job is printed. The same thing happens when you open a command line, then enter a command. In both cases, the original process is called the parent, and the secondary process is called the child. Because of this type of relation, you may sometimes hear about process trees.
- The opening of a secondary process is also called forking.
Linux includes a number of tools for working with processes.
Be careful about killing a process or changing its priority. Unless you know what you are doing, you can easily crash your system. Be especially careful of daemons, which often manage important system services and resources.
Understanding Process Statistics
Statistics What It Means
command The command used to start the process
%cpu The percentage of processing power used by the process.
% mem The percentage of system memory used by the process.
The process's nice scheduling priority. The higher the
ni number, the lower the priority.
Negative numbers are highest of all.
pid Process id . Useful for killing a process.
The process's scheduling priority. The higher the
pri number, the lower the priority.
Negative numbers are highest of all.
rss The amount of ram memory in kilobytes.
shared The amount of shared memory used.
size The size of the virtual image.
start The date that the process was started.
The state that the process is in. Common states are:
* S: Sleep.
stat * T: Suspended, but running.
* R: Running
* D: Asleep and not interruptible.
* Z: Zombie
swap The amount of swap memory used, in kilobytes.
time How much cpu time the process is using.
tty Controlling Terminal. Daemons do not have a terminal.
user The user running the process
wchan The kernel function which controls the process.
Viewing Active Processes From the Desktop
- 1 Type: gnome-system-monitor . The gnome System Monitor opens.
>
- 2 To view the details of a process in the lower window, click on the more info button.
- 3 To select the characteristics to display, select from the menu, Edit > Preferences > Process Fields.
Viewing Active Processes From the Command Line
Type: ps options .
Unlike most commands, ps does not require one or two hyphens before the options.
For a complete view, you must be root user.
Once the display is running, you can use the O+ or O- options to add or subtract statistics from the display. The display you set by the keyboard is used in the terminal in which you are running the command as long as the terminal is open, or until you change it.
The ps Command (Process Screen)
+-------------------------------------------------------------------+
Option Action Comment
-------+--------------------+--------------------------------------
l Displays in long
format
-------+--------------------+--------------------------------------
u Displays user name
and start time
-------+--------------------+--------------------------------------
Does not display of
r processes that are
not running
-------+--------------------+--------------------------------------
A plus sign adds a column the next
* O + time you use the command in the same
key terminal.
or Adjusts displays A minus sign makes the column not
display the next time that the command
* O - is used in the same terminal.
key
See Display Options for the ps
Command below.
+-------------------------------------------------------------------+
Display Options for the ps Command
+-------------------------------------------------------------------+
Option To display Comment
---------+-------------------------------+-------------------------
* k The cpu time taken by the
* stimeprocess
---------+-------------------------------+-------------------------
* U The id of the process's user
* uid
---------+-------------------------------+-------------------------
* c The name of the processes'
* cmd binary file
---------+-------------------------------+-------------------------
* g The process's group
* pgrp
---------+-------------------------------+-------------------------
* p The process's id
* if
---------+-------------------------------+-------------------------
* p The process's parent's id
* ppid
---------+-------------------------------+-------------------------
* s The process's size The size is given in
* size kilobytes
+-------------------------------------------------------------------+
Viewing Intensive Processes
The top command shows a constantly updated summary of the processes that are using the most system resources. The figures given are relative to each other, rather than representing the actual amount of ram used.
Type: top options .
Once the display is running, you can use keyboard commands to:
- Change the details of the display, including the time between updates and the statistics shown.
- Change the process or number of processes shown.
- Sort the display according to different criteria.
- Set a priority, as with the nice command. Stop a process as with the kill command. The top Command (Views Intensive Processes)
+---------------------------------------------------------------+
Option Action
------------+--------------------------------------------------
-d seconds To set the interval between updatings of the
display.
------------+--------------------------------------------------
-p To display only processes with a particular id .
process_id
+---------------------------------------------------------------+
Keyboard Shortcuts for the top Command
+---------------------------------------------------------------+
Press. To. . .
. .
------+--------------------------------------------------------
c Toggle the display of the command name or full command
line.
------+--------------------------------------------------------
f Select fields to display.
------+--------------------------------------------------------
Open a prompt to enter a process id to kill and send a
k signal to kill it. Unless you are the root user, you can
only kill processes you started.
------+--------------------------------------------------------
l Toggle the display of load average and uptime
information.
------+--------------------------------------------------------
m Toggle the display of memory information.
------+--------------------------------------------------------
M Sort by the use of resident memory.
------+--------------------------------------------------------
N Sort by process id .
------+--------------------------------------------------------
n Set the number of processes to show. Selecting 0
numberdisplays a screenful.
------+--------------------------------------------------------
q Exit the display.
------+--------------------------------------------------------
Set priority, as with the nice command.
r
See Setting the Priority for a Process From the Command
Line. (***)
------+--------------------------------------------------------
space Update the display.
bar
------+--------------------------------------------------------
t Toggle display of processes and cpu states.
+---------------------------------------------------------------+
Stopping a Process From the Desktop
- 1 Type: gnome-system-monitor. The gnome System Monitor opens.
>
- 2 Select a process. A process is highlighted when selected.
- 3 Right-click on the process and select Kill Process from the pop-up menu. A dialog box will appear warning that any unsaved data will be lost. Hit the Kill Process button to confirm.
[Warning]
Closing some proceses can crash your system or cause other problems.
Stopping a Process From the Command Line
You can stop a process from the command line in three ways:
- Identifying it by id .
- Identifying it by the command that starts it.
- Identifying it by a file that uses it.
Stopping a Process Identified by ID
Type: kill process_ids .
Unless you are root user, you can only kill processes that you started.
[Warning]
Closing a process can crash your system or cause other problems.
Stopping a Process Identified by Name or Files
Type: killall options command .
Unless you are root user, you can only kill processes that you started.
[Warning]
Closing some processes can crash your system or cause other problems.
Stopping a Process Identified By a File Using It
Type: killall options file .
A name with a forward slash in it is read as a file name. All processes that use that file are closed.
Unless you are root user, you can only kill processes that you started.
[Warning]
Closing some processes can crash your system or cause other problems.
The killall Command (Stops Processes)
+-------------------------------------------------------------------+
Option Action Comment
------+-------------------------+----------------------------------
Stops only processes thatIf this option is not used, all
-e are an exact match of processes are killed that match
what you enter. the first 15 characters.
------+-------------------------+----------------------------------
-g Stops all parent
processes as well.
------+-------------------------+----------------------------------
Enters a confirmation
-i before each process is
stopped.
------+-------------------------+----------------------------------
-v Displays progress
on-screen.
+-------------------------------------------------------------------+
Setting the Priority for a Process From the Desktop
Linux assigns priorities for processes. However, you can set your own priorities:
- 1 Type: gnome-system-monitor . The gnome System Monitor opens.
- 2 Select a process. The process is highlighted when selected.
- 3 Right-click on the process and select Change Priority from the pop-up menu. The Renice window opens.
- 4 Drag the slider bar with the mouse cursor to set the priority. The priority is a number with a range from -20 (highest priority) to 19 (lowest priority). Only the root user can use a negative priority (-1 to -20).
- 5 Select the Change Priority button to complete the task and close the window.
Setting the Priority for a Process From the Command Line
Linux assigns priorities for processes. However, you can set your own priorities.
Type: nice -n adjustment command .
In this structure:
- The adjustment is a number with a range from -20 (highest priority) to 19 (lowest priority). Only the root user can use a negative priority (-1 to -20).
- The command includes any options or arguments.
[Warning]
Changing a process's priority may cause problems on your system, especially on systems with limited hardware resources or a large number of users.
Starting up and Shut down:
Shutting Down
To shut down Red Hat Linux, issue the shutdown command. You can read the shutdown man page for complete details, but the two most common uses are:
/sbin/shutdown -h now /sbin/shutdown -r now |
You must run shutdown as root. After shutting everything down, the -h option will halt the machine, and the -r option will reboot.
Although the reboot and halt commands are now able to invoke shutdown if run while the system is in runlevels 1-5, it is a bad habit to get into, as not all Linux-like operating systems have this feature.
| Warning |
If your computer does not power itself down, you should not turn off the computer until you see a message indicating that the system is halted or finished shutting down. Failure to wait for this message will mean that you may be turning off the machine before your hard drive partitions are unmounted. This can cause filesystem corruption, even to the point where your system may not boot the next time it attempts to start up. Be patient when halting your system |
Peripherals, Kernel loading, Console, The scheduler,
Sysconfig Information
The following information outlines some of the various files in /etc/sysconfig, their function, and their contents. This information is not intended to be complete, as many of these files have a variety of options that are only used in very specific or rare circumstances.
Files in /etc/sysconfig
The following files are normally found in /etc/sysconfig:
· amd
· apmd
· authconfig
· cipe
· clock
· desktop
· firewall
· harddisks
· hwconf
· i18n
· init
· ipchains
· iptables
· irda
· keyboard
· kudzu
· mouse
· network
· pcmcia
· rawdevices
· sendmail
· soundcard
· ups
· vncservers
It is possible that your system may be missing a few of them if the corresponding program that would need that file is not installed.
Next, we will take a look at each one.
/etc/sysconfig/amd
The /etc/sysconfig/amd file contains various parameters used by amd allowing for the automounting and automatic unmounting of filesystems.
/etc/sysconfig/apmd
The /etc/sysconfig/apmd file is used by apmd as a configuration for what things to start/stop/change on suspend or resume. It is set up to turn on or off apmd during startup, depending on whether your hardware supports Advanced Power Management (APM) or if you choose not to use it. apm is a monitoring daemon that works with power management code within the Linux kernel. It can alert you to a low battery if you are using Red Hat Linux on a laptop, among other things.
/etc/sysconfig/authconfig
The /etc/sysconfig/authconfig file sets the kind of authorization to be used on the host. It contains one or more of the following lines:
· USEMD5=<value>, where <value> is one of the following:
o yes — MD5 is used for authentication.
o no — MD5 is not used for authentication.
· USEKERBEROS=<value>, where <value> is one of the following:
o yes — Kerberos is used for authentication.
o no — Kerberos is not used for authentication.
· USELDAPAUTH=<value>, where <value> is one of the following:
o yes — LDAP is used for authentication.
o no — LDAP is not used for authentication.
/etc/sysconfig/clock
The /etc/sysconfig/clock file controls the interpretation of values read from the system clock. Earlier releases of Red Hat Linux used the following values (which are deprecated):
· CLOCKMODE=<value>, where <value> is one of the following:
o GMT — Indicates that the clock is set to Universal Time (Greenwich Mean Time).
o ARC — Indicates the ARC console's 42-year time offset is in effect (for Alpha-based systems only).
Currently, the correct values are:
· UTC=<value>, where <value> is one of the following boolean values:
o true — Indicates that the clock is set to Universal Time. Any other value indicates that it is set to local time.
· ARC=<value>, where <value> is the following:
o true — Indicates the ARC console's 42-year time offset is in effect. Any other value indicates that the normal UNIX epoch is assumed (for Alpha-based systems only).
· ZONE=<filename> — Indicates the timezone file under /usr/share/zoneinfo that /etc/localtime is a copy of, such as:
ZONE="America/New York" |
/etc/sysconfig/desktop
The /etc/sysconfig/desktop file specifies the desktop manager to be run, such as:
DESKTOP="GNOME" |
/etc/sysconfig/firewall
The /etc/sysconfig/firewall file contains various firewall settings. By default, this file (if created) is empty.
/etc/sysconfig/harddisks
The /etc/sysconfig/harddisks file allows you to tune your hard drive(s). You can also use /etc/sysconfig/hardiskhd[a-h], to configure parameters for specific drives.
| Warning |
Do not make changes to this file lightly. If you change the default values stored here, you could corrupt all of the data on your hard drive(s). |
![clip_image015[1]](http://lh3.ggpht.com/_eR0kVz99BZw/SewSoarBQOI/AAAAAAAAAaI/-Qgr5u9Fyko/s1600-h/clip_image015%5B1%5D%5B2%5D.gif "clip_image015[1]"){kind=small}
The /etc/sysconfig/harddisks file may contain the following:
· USE_DMA=1, where setting this to 1 enables DMA. However, with some chipsets and hard drive combinations, DMA can cause data corruption. Check with your hard drive documentation or manufacturer before enabling this.
· Multiple_IO=16, where a setting of 16 allows for multiple sectors per I/O interrupt. When enabled, this feature reduces operating system overhead by 30-50%. Use with caution.
· EIDE_32BIT=3 enables (E)IDE 32-bit I/O support to an interface card.
· LOOKAHEAD=1 enables drive read-lookahead.
· EXTRA_PARAMS= specifies where extra parameters can be added.
/etc/sysconfig/hwconf
The /etc/sysconfig/hwconf file lists all the hardware that kudzu detected on your system, as well as the drivers used, vendor ID and device ID information. The kudzu program detects and configures new and/or changed hardware on a system. The /etc/sysconfig/hwconf file is not meant to be manually edited. If you do edit it, devices could suddenly show up as being added or removed.
/etc/sysconfig/i18n
The /etc/sysconfig/i18n file sets the default language, such as:
LANG="en_US" |
/etc/sysconfig/init
The /etc/sysconfig/init file controls how the system will appear and function during bootup.
The following values may be used:
· BOOTUP=<value>, where <value> is one of the following:
o BOOTUP=color means the standard color boot display, where the success or failure of devices and services starting up is shown in different colors.
o BOOTUP=verbose means an old style display, which provides more information than purely a message of success or failure.
o Anything else means a new display, but without ANSI-formatting.
· RES_COL=<value>, where <value> is the number of the column of the screen to start status labels. Defaults to 60.
· MOVE_TO_COL=<value>, where <value> moves the cursor to the value in the RES_COL line. Defaults to ANSI sequences output by echo -e.
· SETCOLOR_SUCCESS=<value>, where <value> sets the color to a color indicating success. Defaults to ANSI sequences output by echo -e, setting the color to green.
· SETCOLOR_FAILURE=<value>, where <value> sets the color to a color indicating failure. Defaults to ANSI sequences output by echo -e, setting the color to red.
· SETCOLOR_WARNING=<value>, where <value> sets the color to a color indicating warning. Defaults to ANSI sequences output by echo -e, setting the color to yellow.
· SETCOLOR_NORMAL=<value>, where <value> sets the color to 'normal'. Defaults to ANSI sequences output by echo -e.
· LOGLEVEL=<value>, where <value> sets the initial console logging level for the kernel. The default is 7; 8 means everything (including debugging); 1 means nothing except kernel panics. syslogd will override this once it starts.
· PROMPT=<value>, where <value> is one of the following boolean values:
o yes — Enables the key check for interactive mode.
o no — Disables the key check for interactive mode.
/etc/sysconfig/ipchains
The /etc/sysconfig/ipchains file contains information used by the kernel to set up ipchains rules regarding packet filtering.
This file is modified by running the service ipchains save command when valid ipchains rules are in place. You should not manually edit this file. Instead, use the ipchains command to configure the necessary packet filtering rules and then save the rules to this file.
/etc/sysconfig/iptables
Like /etc/sysconfig/ipchains, the /etc/sysconfig/iptables file stores information used by the kernel to provide specialized packet filtering services. However, this file is used by iptables rather than ipchains.
You should not modify this file by hand unless you are familiar with methods used to construct iptables rules. These rules are written to /etc/sysconfig/iptables by the service iptables save command, which stores the current iptables rules by running the /sbin/iptables-save program. Then, when iptables is restarted, such as is the case when the system is booted, the /sbin/iptables-restore program reads the file and reinstitutes the packet filtering rules.
/etc/sysconfig/irda
The /etc/sysconfig/irda file controls how infrared devices on your system are configured at startup.
The following values may be used:
· IRDA=<value>, where <value> is one of the following boolean values:
o yes — irattach will be run, which periodically checks to see if anything is trying to connect to the infrared port, such as another notebook computer trying to make a network connection. For infrared devices to work on your system, this line must be set to yes.
o no — irattach will not be run, preventing infrared device communication.
· DEVICE=<value>, where <value> is the device (usually a serial port) that handles infrared connections.
· DONGLE=<value>, where <value> specifies the type of dongle being used for infrared communication. This setting exists for people who use serial dongles rather than real infrared ports. A dongle is a device that is attached to a traditional serial port to communicate via infrared. This line is commented out by default because notebooks with real infrared ports are far more common than computers with add-on dongles.
· DISCOVERY=<value>, where <value> is one of the following boolean values:
o yes — Starts irattach in discovery mode, meaning it actively checks for other infrared devices. This needs to be turned on for the machine to be actively looking for an infrared connection (meaning the peer that does not initiate the connection).
o no — Does not start irattach in discovery mode.
/etc/sysconfig/keyboard
The /etc/sysconfig/keyboard file controls the behavior of the keyboard. The following values may be used:
· KEYBOARDTYPE=sunpc, which is used on SPARCs only. sun means a Sun keyboard is attached on /dev/kbd, and pc means a PS/2 keyboard connected to a PS/2 port.
· KEYTABLE=<file>, where <file> is the name of a keytable file. For example: KEYTABLE="us". The files that can be used as keytables start in /usr/lib/kbd/keymaps/i386 and branch into different keyboard layouts from there, all labeled <file>.kmap.gz. The first file found beneath /usr/lib/kbd/keymaps/i386that matches the KEYTABLE setting is used.
/etc/sysconfig/kudzu
The /etc/sysconfig/kuzdu allows you to specify a safe probe of your system's hardware by kudzu at boot time. A safe probe is one that disables serial port probing.
· SAFE=<value>, where <value> is one of the following:
o yes — kuzdu does a safe probe.
o no — kuzdu does a normal probe.
/etc/sysconfig/mouse
The /etc/sysconfig/mouse file is used to specify information about the available mouse. The following values may be used:
· FULLNAME=<value>, where <value> refers to the full name of the kind of mouse being used.
· MOUSETYPE=<value>, where <value> is one of the following:
o microsoft — A Microsoft™ mouse.
o mouseman — A MouseMan™ mouse.
o mousesystems — A Mouse Systems™ mouse.
o ps/2 — A PS/2 mouse.
o msbm — A Microsoft™ bus mouse.
o logibm — A Logitech™ bus mouse.
o atibm — An ATI™ bus mouse.
o logitech — A Logitech™ mouse.
o mmseries — An older MouseMan™ mouse.
o mmhittab — An mmhittab mouse.
· XEMU3=<value>, where <value> is one of the following boolean values:
o yes — The mouse only has two buttons, but three mouse buttons should be emulated.
o no — The mouse already has three buttons.
· XMOUSETYPE=<value>, where <value> refers to the kind of mouse used when X is running. The options here are the same as the MOUSETYPE setting in this same file.
· DEVICE=<value>, where <value> is the mouse device.
In addition, /dev/mouse is a symbolic link that points to the actual mouse device.
/etc/sysconfig/network
The /etc/sysconfig/network file is used to specify information about the desired network configuration. The following values may be used:
· NETWORKING=<value>, where <value> is one of the following boolean values:
o yes — Networking should be configured.
o no — Networking should not be configured.
· HOSTNAME=<value>, where <value> should be the Fully Qualified Domain Name (FQDN), such as hostname.domain.com, but can be whatever hostname you want.
| Note |
For compatibility with older software that people might install (such as trn), the /etc/HOSTNAME file should contain the same value as here. |
· GATEWAY=<value>, where <value> is the IP address of the network's gateway.
· GATEWAYDEV=<value>, where <value> is the gateway device, such as eth0.
· NISDOMAIN=<value>, where <value> is the NIS domain name.
/etc/sysconfig/pcmcia
The /etc/sysconfig/pcmcia file is used to specify PCMCIA configuration information. The following values may be used:
· PCMCIA=<value>, where <value> is one of the following:
o yes — PCMCIA support should be enabled.
o no — PCMCIA support should not be enabled.
· PCIC=<value>, where <value> is one of the following:
o i82365 — The computer has an i82365-style PCMCIA socket chipset.
o tcic — The computer has a tcic-style PCMCIA socket chipset.
· PCIC_OPTS=<value>, where <value> is the socket driver (i82365 or tcic) timing parameters.
· CORE_OPTS=<value>, where <value> is the list of pcmcia_core options.
· CARDMGR_OPTS=<value>, where <value> is the list of options for the PCMCIA cardmgr (such as -q for quiet mode; -m to look for loadable kernel modules in the specified directory, and so on). Read the cardmgr man page for more information.
/etc/sysconfig/rawdevices
The /etc/sysconfig/rawdevices file is used to configure raw device bindings, such as:
/dev/raw/raw1 /dev/sda1 /dev/raw/raw2 8 5 |
/etc/sysconfig/sendmail
The /etc/sysconfig/sendmail file allows messages to be sent to one or more recipients, routing the message over whatever networks are necessary. The file sets the default values for the Sendmail application to run. Its default values are to run as a background daemon, and to check its queue once an hour in case something has backed up.
The following values may be used:
· DAEMON=<value>, where <value> is one of the following boolean values:
o yes — Sendmail should be configured to listen to port 25 for incoming mail. yes implies the use of Sendmail's -bd options.
o no — Sendmail should not be configured to listen to port 25 for incoming mail.
· QUEUE=1h which is given to Sendmail as -q$QUEUE. The -q option is not given to Sendmail if /etc/sysconfig/sendmail exists and QUEUE is empty or undefined.
/etc/sysconfig/soundcard
The /etc/sysconfig/soundcard file is generated by sndconfig and should not be modified. The sole use of this file is to determine what card entry in the menu to pop up by default the next time sndconfig is run. Soundcard configuration information is located in the /etc/modules.conf file.
It may contain the following:
· CARDTYPE=<value>, where <value> is set to, for example, SB16 for a Soundblaster 16 sound card.
/etc/sysconfig/ups
The /etc/sysconfig/ups file is used to specify information about any Uninterruptible Power Supplies (UPS) connected to your system. A UPS can be very valuable for a Red Hat Linux system because it gives you time to correctly shut down the system in the case of power interruption. The following values may be used:
· SERVER=<value>, where <value> is one of the following:
o yes — A UPS device is connected to your system.
o no — A UPS device is not connected to your system.
· MODEL=<value>, where <value> must be one of the following or set to NONE if no UPS is connected to the system:
o apcsmart — For a APC SmartUPS™ or similar device.
o fentonups — For a Fenton UPS™.
o optiups — For an OPTI-UPS™ device.
o bestups — For a Best Power™ UPS.
o genericups — For a generic brand UPS.
o ups-trust425+625 — For a Trust™ UPS.
· DEVICE=<value>, where <value> specifies where the UPS is connected, such as /dev/ttyS0.
· OPTIONS=<value>, where <value> is a special command that needs to be passed to the UPS.
/etc/sysconfig/vncservers
The /etc/sysconfig/vncservers file configures how the Virtual Network Computing (VNC) server starts up. VNC is a remote display system which allows you to view a desktop environment not only on the machine where it is running but across different networks (from a LAN to the Internet) and using a wide variety of machine architectures.
It may contain the following:
· VNCSERVERS=<value>, where <value> is set to something like "1:fred", to indicate that a VNC server should be started for user fred on display :1. User fred must have set a VNC password using vncpasswd before attempting to connect to the remote VNC server.
Note that when you use a VNC server, your communication with it is unencrypted, and so it should not be used on an untrusted network. For specific instructions concerning the use of SSH to secure the VNC communication, please read the information found at http://www.uk.research.att.com/vnc/sshvnc.html. To find out more about SSH, see Chapter 10 or Official Red Hat Linux Customization Guide.
init and the inittab file,
Init
The kernel, once it is loaded, finds init in sbin and executes it.
When init starts, it becomes the parent or grandparent of all of the processes that start up automatically on your Linux system. The first thing init does, is reading its initialization file, /etc/inittab. This instructs init to read an initial configuration script for the environment, which sets the path, starts swapping, checks the file systems, and so on. Basically, this step takes care of everything that your system needs to have done at system initialization: setting the clock, initializing serial ports and so forth.
Then init continues to read the /etc/inittab file, which describes how the system should be set up in each run level and sets the default run level. A run level is a configuration of processes. All UNIX-like systems can be run in different process configurations, such as the single user mode, which is referred to as run level 1 or run level S (or s). In this mode, only the system administrator can connect to the system. It is used to perform maintenance tasks without risks of damaging the system or user data. Naturally, in this configuration we don't need to offer user services, so they will all be disabled. Another run level is the reboot run level, or run level 6, which shuts down all running services according to the appropriate procedures and then restarts the system.
Commonly, run level 3 is configured to be text mode on a Linux machine, and run level 5 initializes the graphical login and environment. More about run levels in the next section, see Section 4.2.5.
After having determined the default run level for your system, init starts all of the background processes necessary for the system to run by looking in the appropriate rc directory for that run level. init runs each of the kill scripts (their file names start with a K) with a stop parameter. It then runs all of the start scripts (their file names start with an S) in the appropriate run level directory so that all services and applications are started correctly. In fact, you can execute these same scripts manually after the system is finished booting with a command like /etc/rc.d/init.d/httpd stop or service httpd stop logged in as root, in this case stopping the web server.
None of the scripts that actually start and stop the services are located in /etc/rc.d/rc<x>.d. Rather, all of the files in /etc/rc.d/rc<x>.d are symbolic links that point to the actual scripts located in /etc/rc.d/init.d. A symbolic link is nothing more than a file that points to another file, and is used in this case because it can be created and deleted without affecting the actual scripts that kill or start the services. The symbolic links to the various scripts are numbered in a particular order so that they start in that order. You can change the order in which the services start up or are killed by changing the name of the symbolic link that refers to the script that actually controls the service. You can use the same number multiple times if you want a particular service started or stopped right before or after another service.
Below a sample of /etc/rc5.d content, which lists the services started up in run level 5:
[jean@blub /etc/rc5.d] ls K15httpd@ K45named@ S08ipchains@ S25netfs@ S85gpm@ K16rarpd@ K46radvd@ S08iptables@ S26apmd@ S90crond@ K20nfs@ K61ldap@ S09isdn@ S28autofs@ S90xfs@ K20rstatd@ K65identd@ S10network@ S30nscd@ S95anacron@ K20rusersd@ K74ntpd@ S12syslog@ S55sshd@ S95atd@ K20rwalld@ K74ypserv@ S13portmap@ S56rawdevices@ S97rhnsd@ K20rwhod@ K74ypxfrd@ S14nfslock@ S56xinetd@ S99local@ K25squid@ K89bcm5820@ S17keytable@ S60lpd@ K34yppasswdd@ S05kudzu@ S20random@ S80sendmail@ |
After init has progressed through the run levels to get to the default run level, the /etc/inittab script forks a getty process for each virtual console (login prompt in text mode). getty opens tty lines, sets their modes, prints the login prompt, gets the user's name, and then initiates a login process for that user. This allows users to authenticate themselves to the system and use it. By default, most systems offer 6 virtual consoles, but as you can see from the inittab file, this is configurable.
/etc/inittab can also tell init how it should handle a user pressing Ctrl+Alt+Delete at the console. As the system should be properly shut down and restarted rather than immediately power-cycled, init is told to execute the command /sbin/shutdown -t3 -r now, for instance, when a user hits those keys. In addition, /etc/inittab states what init should do in case of power failures, if your system has a UPS unit attached to it.
In run level 5, /etc/inittab runs a script called /etc/X11/prefdm. The prefdm script runs the preferred X display manager, based on the contents of the /etc/sysconfig/desktop directory. This is typically gdm if you run GNOME or kdm if you run KDE, but they can be mixed, and there's also the xdm that comes with a standard X installation.
The /etc/sysconfig directory contains entries for a range of functions and services, these are all read at boot time. This last part, however, might be somewhat different depending on your Linux distribution.
Besides the graphical user environment, a lot of other services may be started as well. But if all goes well, you should be looking at a login prompt or login screen when the boot process has finished.
Running Programs at Boot Time
The file /etc/rc.d/rc.local script is run by init at boot time, after all other initialization is complete, and whenever you change runlevels. You can add additional initialization commands here. For instance, you may want to start up additional daemons or initialize a printer.
In addition, if you require serial port setup, you can create and edit /etc/rc.serial, and it will be executed automatically at boot time. This script can run a number of setserial commands to specially configure the system's serial ports. See the setserial man page for more information.
Run-levels, Run level scripts.
Init run levels
The idea behind operating different services at different run levels essentially revolves around the fact that different systems can be used in different ways. Some services cannot be used until the system is in a particular state, or mode, such as being ready for more than one user or having networking available.
There are times in which you may want to operate the system in a lower mode. Examples are fixing disk corruption problems in run level 1 so no other users can possibly be on the system, or leaving a server in run level 3 without an X session running. In these cases, running services that depend upon a higher system mode to function does not make sense because they will not work correctly anyway. By already having each service assigned to start when its particular run level is reached, you ensure an orderly start up process, and you can quickly change the mode of the machine without worrying about which services to manually start or stop.
Available run levels are generally described in /etc/inittab, which is partially shown below:
# # inittab This file describes how the INIT process should set up # the system in a certain run-level. # Default runlevel. The runlevels are: # 0 - halt (Do NOT set initdefault to this) # 1 - Single user mode # 2 - Multiuser, without NFS # (The same as 3, if you do not have networking) # 3 - Full multiuser mode # 4 - unused # 5 - X11 # 6 - reboot (Do NOT set initdefault to this) # id:5:initdefault: <--cut--> |
Feel free to configure runlevels 2 and 4 as you see fit. Many users configure those runlevels in a way that makes the most sense for them while leaving the standard runlevels 3 and 5 alone. This allows them to quickly move in and out of their custom configuration without disturbing the normal set of features at the standard runlevels.
If your machine gets into a state where it will not boot due to a bad /etc/inittab or will not let you log in because you have a corrupted /etc/passwd file (or if you have simply forgotten your password), boot into single-user mode.
Init Runlevels
The idea behind operating different services at different runlevels essentially revolves around the fact that different systems can be used in a different ways. Some services cannot be used until the system is in a particular state, or mode, such as ready for more than one user or has networking available.
There are times in which you may want to operate the system at a lower mode, such as fixing disk corruption problems in runlevel 1 so no other users can possibly be on the system or leaving a server in runlevel 3 without an X session running. In these cases, running services that depend upon a higher system mode to function does not make sense because they will not work correctly anyway. By already having each service assigned to start when its particular runlevel is reached, you ensure an orderly start up process and can quickly change the mode of the machine without worrying about which services to manually start or stop.
Generally, Red Hat Linux operates in runlevel 3 — full multi-user mode. The following runlevels are defined in Red Hat Linux:
· 0 — Halt
· 1 — Single-user mode
· 2 — Not used (user-definable)
· 3 — Full multi-user mode
· 4 — Not used (user-definable)
· 5 — Full multi-user mode (with an X-based login screen)
· 6 — Reboot
The default runlevel for a system to boot to and stop is configured in /etc/inittab. For more information on /etc/inittab, see the section called SysV Init.
Feel free to configure runlevels 2 and 4 as you see fit. Many users configure those runlevels in a way that makes the most sense for them while leaving the standard runlevels 3 and 5 alone. This allows them to quickly move in and out of their custom configuration without disturbing the normal set of features at the standard runlevels.
If your machine gets into a state where it will not boot due to a bad /etc/inittab or will not let you log in because you have a corrupted /etc/passwd (or if you have simply forgotten your password), boot into single-user mode.
If you are using LILO, you can enter single-user mode by typing linux single at the LILO boot: prompt.
If you are using GRUB as your boot loader, you can enter single-user mode using the following steps.
· In the graphical GRUB boot loader screen, select the Red Hat Linux boot label and press [e] to edit it.
· Arrow down to the kernel line and press [e] to edit it.
· At the prompt, type single and press [Enter].
· You will be returned to the GRUB screen with the kernel information. Press the [b] key to boot the system into single user mode.
A very bare system will boot, and you will have a command shell from which you can fix things.
If this does not work, you will need to boot by typing linux init=/bin/bash at the LILO boot: prompt. This places you at a shell prompt; note that no filesystems other than the root filesystem are mounted, and the root filesystem is mounted in read-only mode. To mount it in read-write mode (to allow editing of a broken /etc/inittab, for example) do:
mount -n /proc mount -o rw,remount / |
Initscript Utilities
The chkconfig utility in /sbin provides a simple command-line tool for maintaining the /etc/rc.d/init.d directory hierarchy. It relieves system administrators from having to directly manipulate the numerous symbolic links in the directories under /etc/rc.d.
In addition, there is ntsysv that provides a text-based interface, which you may find easier to use than chkconfig's command-line interface.
If you prefer a graphical interface, use the serviceconf program.
All of these utilities must be run as root.
Linux Run level scripts
The runlevel scripts are used to bring up many system and networking functions. Since some functions are interdependent on other functions there is some required order in which these scripts must be run in order to bring the system up and to bring it gracefully down. Each runlevel has its own set of start(S) and kill(K) scripts but all these scripts are supported in the directory /etc/rc.d/init.d. This is because the start and kill scripts are soft links to the files in the /etc/rc.d/init.d directory.
The rc script Program
The script file /etc/rc.d/rc is run for the appropriate runlevel (typically 3 or 5) This file does the following:
1. It gets the previous and current system runlevels.
2. If the word confirm is in the file "/proc/cmdline" if sets up to run the scripts below in user confirmation mode.
3. All kill files (files whose first letter is 'K') in the subdirectory "/etc/rc.d/rc3.d" (assuming the previous runlevel was 3) are run. The parameter stop is usually passed on the command line to the kill script.
4. All startup files (files whose first letter is 'S") in the subdirectory "/etc/rc.d/rc5.d" (assuming the current runlevel is 5) are run. The parameter start is usually passed on the command line to the kill script.
These runlevel scripts are used to bring up (or down) various system services such as cron and gpm along with networking services from the network cards through Samba, and servers like DNS, DHCP, and NFS. A directory listing of the files in the /etc/rc.d/init.d will reveal the many possible services that the system can support.
These services are can be functionally categorized as a system service or a network service. They are described in more detail in the section on Daemons and Services. For more information on how some of the script files for these services run, read the Linux startup Manual. Normally any of these services may be stopped, started, restarted, or status be checked by typing the name of one of these services (with the correct path) followed by the word stop, start, restart, or status respectively. For example the line:
/etc/rc.d/init.d/nfs restart
will restart network file sharing assuming it was running. To see the status type:
/etc/rc.d/init.d/nfs status
The rc.local Script Program
The file "/etc/rc.d/rc3.d/S99.local" is a link file to the file "/etc/rc.d/rc.local". This file doesn't do much except for setting up the "/etc/issue" and "/etc/issue.net" files to reflect the system version when a user begins a terminal or telnet session. This is where most administrators will put any system customizations they want to make
The basic steps of Booting in Linux
1. Of course the first step is the display of the BIOS info, but this too can be changed.
Instead of the Energy Star logo you are able to place a nice penguin logo here.
More information about changing the BIOS logo can be found at here.
2. Next, the kernel is loaded into the memory and executed. This can be done by different programs like LOADLIN that reads a kernel file you specify and writes it into the memory (it overwrites DOS loaded into the memory).
Another way can be LILO, the LInux LOader. It loads the kernel directly from the physical sector on a hard disk. Therefore, LILO writes itself into the Master Boot Record (MBR) together with the sectors where the kernels are placed. After choosing the appropriate kernel, it writes it into memory and executes it.
3. The kernel is responsible for recognizing and including all the hardware into the system. Every discovered (and not discovered) component is printed on the screen. If you want to read some of the scrolling lines a little bit longer, the key PAUSE, Scroll Back, or Roll are useful. Of course the scroll back buffer has already been activated by Linux (Ctrl+PgUp or Ctrl+PgDn).
4. The last task of the kernel is to mount the root partition. The root partition is specified in LILO- or LOADLIN- as a parameter. Then it starts the first process, which is usually called INIT (the program is placed at /sbin/init).
The kernel finished its job and the complete system of all discovered and mounted hardware is available.
The messages that show the point where the kernel ends and the programs begin looks like:
5. VFS: mounted root (ext2fs) filesystem read-only. (still Kernel)
6. INIT: Version X.XX booting (already INIT)
7. INIT checks different items. First, it looks for the file /etc/inittab where it finds the runlevel that has to be started (id:X:initdefault:, where `X' is the number of the runlevel).
Then INIT checks which program has to be started next, because not a server nor a shell has been started, nor has a network connection been made nor any partition been made available. All this happens in the line si:I:wait:PROGRAM).
In general PROGRAM stands for /etc/rc.d/init.boot or /sbin/init.d/boot and this one is executed now.
8. The BOOTscript usually does the following (from here everything can be configured and tailored to your wishes):
o Turn on swap
o check the root partition and remount it read/write-able
o check for other partitions and mount them
and if one partition has a problem, it would stop the process and requires a root login
o start the module manager that is responsible for "auto loading" the required modules
o if needed, basic network programs are started (e.g. loopback device)
Please Remember: Every system can be different.
Often, you find the file boot.local in the same directory which is executed by boot at the end. In it, you can place additional commands.
Of course, you could put these commands into the boot script, but be careful! You should not change anything there as long as you do not know what you are doing.
9. Then, INIT checks which runlevel is declared to be the next (/etc/inittab) and now all scripts of this runlevel are /etc/inittab: lN:N:wait:SCRIPT (N=runlevel). In general, this means that all scripts in /etc/init.d/rcN.d/ (or similar) are started (N=runlevel).
Here, all the services and daemons are started which are run in the background. At the same time, lock- and log-files are created if necessary.
10. Next, INIT starts a getty for all terminals. It controls the terminal and starts the /bin/login first.
This is specified in the /etc/inittab,too.
"respawn" means here that the program would be restarted at once if it was finished. Without this option you would be able to login only once ...
11. If you chose a graphical login (at SuSE it is the runlevel 3, at RedHat the runlevel 5), 'xdm' or 'kdm' is started and you get a graphical login window.
This works over a network like so many other things, too. E.g. a simple startx --query another_computer:0 places a login window of another computer in the network on your screen.
12. After you logged in the script /etc/profile (identical for all users) is called and then the files ~/.profile, ~/.login or for the bash shell ~/.bashrc ~/.bash_login and ~/.bash_profile would be "executed" if they existed.
13. After the login, the shell is started and you can use the system.
Scheduling processes
A Linux system can have a lot to suffer from, but it usually suffers only during office hours. Whether in an office environment, a server room or at home, most Linux systems are just idling away during the morning, the evening, the nights and weekends. Using this idle time can be a lot cheaper than buying those machines you'd absolutely need if you want everything done at the same time.
There are three types of delayed execution:
· Waiting a little while and then resuming job execution, using the sleep command. Execution time depends on the system time at the moment of submission.
· Running a command at a specified time, using the at command. Execution of the job(s) depends on system time, not the time of submission.
· Regularly running a command on a monthly, weekly, daily or hourly basis, using the cron facilities.
The sleep command
The Info page on sleep is probably one of the shortest there is. All sleep does is wait. By default the time to wait is expressed in seconds.
So why does it exist? Some practical examples:
Somebody calls you on the phone, you say "Yes I'll be with you in half an hour" but you're about drowned in work as it is and bound to forget your lunch:
(sleep 1800; echo "Lunch time..") &
When you can't use the at command for some reason, it's five o'clock, you want to go home but there's still work to do and right now somebody is eating system resources:
(sleep 10000; myprogram) &
Make sure there's an auto-logout on your system, and that you log out or lock your desktop/office when submitting this kind of job, or run it in a screen session.
The at command
The at command executes commands at a given time, using your default shell unless you tell the command otherwise (see the man page).
The options to at are rather user-friendly, which is demonstrated in the examples below:
steven@home:~>at tomorrow + 2 days warning: commands will be executed using (in order) a) $SHELL b) login shell c) /bin/sh at> cat reports mail myboss@mycompany at> <EOT> job 1 at 2001-06-16 12:36 |
User steven does a strange thing here combining two commands; we will study this sort of practice in Chapter 5, Redirecting Input and Output.
steven@home:~>at 0237 warning: commands will be executed using (in order) a) $SHELL b) login shell c) /bin/sh at> cd new-programs at> ./configure; make at> <EOT> job 2 at 2001-06-14 02:00 |
The -m option sends mail to the user when the job is done, or explains when a job can't be done. The command atq lists jobs; perform this command before submitting jobs in order prevent them from starting at the same time as others. With the atrm command you can remove scheduled jobs if you change your mind.
Cron and crontab
The cron system is managed by the cron daemon. It gets information about which programs and when they should run from the system's and users' crontab entries. Only the root user has access to the system crontabs, while each user should only have access to his own crontabs. On some systems (some) users may not have access to the cron facility.
At system startup the cron daemon searches /var/spool/cron/ for crontab entries which are named after accounts in /etc/passwd, it searches /etc/cron.d/ and it searches /etc/crontab, then uses this information every minute to check if there is something to be done. It executes commands as the user who owns the crontab file and mails any output of commands to the owner.
On systems using Vixie cron, jobs that occur hourly, daily, weekly and monthly are kept in separate directories in /etc to keep an overview, as opposed to the standard UNIX cron function, where all tasks are entered into one big file.
Example of a Vixie crontab file:
[root@blob /etc]# more crontab SHELL=/bin/bash PATH=/sbin:/bin:/usr/sbin:/usr/bin MAILTO=root HOME=/ # run-parts # commands to execute every hour 01 * * * * root run-parts /etc/cron.hourly # commands to execute every day 02 4 * * * root run-parts /etc/cron.daily # commands to execute every week 22 4 * * 0 root run-parts /etc/cron.weekly commands to execute every month 42 4 1 * * root run-parts /etc/cron.monthly |
Some variables are set, and after that there's the actual scheduling, one line per job, starting with 5 time and date fields. The first field contains the minutes (from 0 to 59), the second defines the hour of execution (0-23), the third is day of the month (1-31), then the number of the month (1-12), the last is day of the week (0-7, both 0 and 7 are Sunday). An asterisk in these fields represents the total acceptable range for the field. Lists are allowed; to execute a job from Monday to Friday enter 1-5 in the last field, to execute a job on Monday, Wednesday and Friday enter 1,3,5.
Then comes the user who should run the processes which are listed in the last column. The example above is from a Vixie cron configuration where root runs the program run-parts on regular intervals, with the appropriate directories as options. In these directories, the actual jobs to be executed at the scheduled time are stored as shell scripts, like this little script that is run daily to update the database used by the locate command:
billy@ahost cron.daily]$ cat slocate.cron #!/bin/sh renice +19 -p $$ >/dev/null 2>&1 /usr/bin/updatedb -f "nfs,smbfs,ncpfs,proc,devpts" -e \ "/tmp,/var/tmp, /usr/tmp,/afs,/net" |
Users are supposed to edit their crontabs in a safe way using the crontab -e command. This will prevent a user from accidentally opening more than one copy of his/her crontab file. The default editor is vi (see Chapter 6. When you quit, the system will tell you that a new crontab is installed.
This crontab entry reminds billy to go to his sports club every Thursday night:
billy:~>crontab -l # DO NOT EDIT THIS FILE - edit the master and reinstall. # (/tmp/crontab.18185 installed on Wed Sep 19 16:37:08 2001) # (Cron version--$Id: chap4.xml,v 1.7 2003/09/17 19:58:26 tille Exp $) 38 16 * * 3 mail -s "sports evening" billy |
Cron will also report about the executed tasks via E-mail. Note that normal users don't have to specify the user who should run the commands. They are executed with the user's own permissions by default.
Linux Startup Scripts
or
What happens before the login prompt!
Boot sequence summary
- BIOS
- Master Boot Record (MBR)
- Kernel
- init
BIOS
Load boot sector from one of:
- Floppy
- CDROM
- SCSI drive
- IDE drive
Master Boot Record
- MBR (loaded from /dev/hda or /dev/sda) contains:
- lilo
- load kernel (image=), or
- load partition boot sector (other=)
- DOS
- load "bootable" partition boot sector (set with fdisk)
- partition boot sector (eg /dev/hda2) contains:
- DOS
- loadlin
- lilo
- kernel
LILO
One minute guide to installing a new kernel
- edit /etc/lilo.conf
- duplicate image= section, eg:
o image=/bzImage-2.2.12
o label=12
o read-only
- man lilo.conf for details
- run /sbin/lilo
- (copy modules)
- reboot to test
Kernel
- initialise devices
- (optionally loads initrd, see below)
- mount root FS
- specified by lilo or loadin
- kernel prints:
- VFS: Mounted root (ext2 filesystem) readonly.
- run /sbin/init, PID 1
- can be changed with boot=
- init prints:
- INIT: version 2.76 booting
initrd
Allows setup to be performed before root FS is mounted
- lilo or loadlin loads ram disk image
- kernel runs /linuxrc
- load modules
- initialise devices
- /linuxrc exits
- "real" root is mounted
- kernel runs /sbin/init
Details in /usr/src/linux/Documentation/initrd.txt
/sbin/init
- reads /etc/inittab
- runs script defined by this line:
- si::sysinit:/etc/init.d/rcS
- switches to runlevel defined by
- id:3:initdefault:
sysinit
- debian: /etc/init.d/rcS which runs
- /etc/rcS.d/S* scripts
- symlinks to /etc/init.d/*
- /etc/rc.boot/* (depreciated)
- redhat: /etc/rc.d/rc.sysinit script which
- load modules
- check root FS and mount RW
- mount local FS
- setup network
- mount remote FS
Example Debian /etc/rcS.d/ directory
README
S05keymaps-lct.sh -> ../init.d/keymaps-lct.sh
S10checkroot.sh -> ../init.d/checkroot.sh
S20modutils -> ../init.d/modutils
S30checkfs.sh -> ../init.d/checkfs.sh
S35devpts.sh -> ../init.d/devpts.sh
S35mountall.sh -> ../init.d/mountall.sh
S35umsdos -> ../init.d/umsdos
S40hostname.sh -> ../init.d/hostname.sh
S40network -> ../init.d/network
S41ipmasq -> ../init.d/ipmasq
S45mountnfs.sh -> ../init.d/mountnfs.sh
S48console-screen.sh -> ../init.d/console-screen.sh
S50hwclock.sh -> ../init.d/hwclock.sh
S55bootmisc.sh -> ../init.d/bootmisc.sh
S55urandom -> ../init.d/urandom
Run Levels
- 0 halt
- 1 single user
- 2-4 user defined
- 5 X11
- 6 Reboot
- Default in /etc/inittab, eg
- id:3:initdefault:
- Change using /sbin/telinit
Run Level programs
- Run programs for specified run level
- /etc/inittab lines:
- 1:2345:respawn:/sbin/getty 9600 tty1
- Always running in runlevels 2, 3, 4, or 5
- Displays login on console (tty1)
- 2:234:respawn:/sbin/getty 9600 tty2
- Always running in runlevels 2, 3, or 4
- Displays login on console (tty2)
- l3:3:wait:/etc/init.d/rc 3
- Run once when switching to runlevel 3.
- Uses scripts stored in /etc/rc3.d/
- ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now
- Run when control-alt-delete is pressed
Typical /etc/rc3.d/ directory
When changing runlevels /etc/init.d/rc 3:
- Kills K##scripts
- Starts S##scripts
K25nfs-server -< ../init.d/nfs-server
K99xdm -< ../init.d/xdm
S10sysklogd -< ../init.d/sysklogd
S12kerneld -< ../init.d/kerneld
S15netstd_init -< ../init.d/netstd_init
S18netbase -< ../init.d/netbase
S20acct -< ../init.d/acct
S20anacron -< ../init.d/anacron
S20gpm -< ../init.d/gpm
S20postfix -< ../init.d/postfix
S20ppp -< ../init.d/ppp
S20ssh -< ../init.d/ssh
S20xfs -< ../init.d/xfs
S20xfstt -< ../init.d/xfstt
S20xntp3 -< ../init.d/xntp3
S89atd -< ../init.d/atd
S89cron -< ../init.d/cron
S99rmnologin -< ../init.d/rmnologin
Managing User Accounts: [2L]
Principles, password file, Password security, Shadow file, Groups and the group file, Shells, restricted shells, user management commands, homes and permissions, default files, profiles, locking accounts, setting passwords, Switching user, Switching group, Removing users.
Managing User Accounts
Even if you're the only actual human being who uses your Linux system, understanding how to manage user accounts is important--even more so if your system hosts multiple users.
User accounts serve a number of purposes on Unix systems. Most prominently, they give the system a way to distinguish between different people who use the system for reasons of identification and security. Each user has a personal account with a separate username and password. As discussed in the section "Section 4.13, "File Ownership and Permissions"" in Chapter 4, "Basic Unix Commands and Concepts", users may set permissions on their files, allowing or restricting access to them by other users. Each file on the system is "owned" by a particular user, who may set the permissions for that file. User accounts are used to authenticate access to the system; only those people with accounts may access the machine. Also, accounts are used to identify users, keep system logs, tag electronic mail messages with the name of the sender, and so forth.
Apart from personal accounts, there are users on the system that provide administrative functions. As we've seen, the system administrator uses the root account to perform maintenance--but usually not for personal system use. Such accounts are accessed using the su command, allowing another account to be accessed after logging in through a personal account.
Other accounts on the system may not be set aside for human interaction at all. These accounts are generally used by system daemons, which must access files on the system through a specific user ID other than root or one of the personal user accounts. For example, if you configure your system to receive a newsfeed from another site, the news daemon must store news articles in a spool directory that anyone can access, but only one user (the news daemon) can write to. No human being is associated with the news account; it is an "imaginary" user set aside for the news daemon only.
One of the permissions bits that can be set on executables is the setuid bit, which causes the program to be executed with the permissions of the owner of that file. For example, if the news daemon were owned by the user news, and the setuid bit set on the executable, it would run as if by the user news. news would have write access to the news spool directory, and all other users would have read access to the articles stored there. This is a security feature. News programs can give users just the right amount of access to the news spool directory, but no one can just play around there.
As the system administrator, it is your job to create and manage accounts for all users (real and virtual) on your machine. This is actually a painless, hands-off task in most cases, but it's important to understand how it works.
5.7.1. The passwd File
Every account on the system has an entry in the file /etc/passwd. This file contains entries, one line per user, that specify several attributes for each account, such as the username, real name, and so forth.
Each entry in this file is of the format:
username:password:uid:gid:gecos:homedir:shell
The following list explains each of these fields:
username
A unique character string, identifying the account. For personal accounts, this is the name the user logs in with. On most systems it is limited to eight alphanumeric characters--for example, larry or kirsten.
password
An encrypted representation of the user's password. This field is set using the passwd program to set the account's password; it uses a one-way encryption scheme that is difficult (but not impossible) to break. You don't set this by hand; the passwd program does it for you. Note, however, that if the first character of the passwd field is * (an asterisk), the account is "disabled"; the system will not allow logins as this user. See the section "Section 5.7.5, "Creating Accounts"" later in this chapter.
uid
The user ID, a unique integer the system uses to identify the account. The system uses the uid field internally when dealing with process and file permissions; it's easier and more compact to deal with integers than byte strings. Therefore, both the uid and the username identify a particular account: the uid is more important to the system, while username is more convenient for humans.
gid
The group ID, an integer referring to the user's default group, found in the file /etc/group. See the section "Section 5.7.4, "The Group File"" that follows.
gecos
Miscellaneous information about the user, such as the user's real name, and optional "location information" such as the user's office address or phone number. Such programs as mail and finger use this information to identify users on the system; we'll talk more about it later. By the way, gecos is a historical name dating back to the 1970s; it stands for General Electric Comprehensive Operating System. GECOS has nothing to do with Unix, except that this field was originally added to /etc/passwd to provide compatibility with some of its services.
homedir
The user's home directory, for his personal use; more on this later. When the user first logs in, her shell finds its current working directory in the named homedir.
shell
The name of the program to run when the user logs in; in most cases, this is the full pathname of a shell, such as /bin/bash or /bin/tcsh.
Many of these fields are optional; the only required fields are username, uid, gid, and homedir. Most user accounts have all fields filled in, but "imaginary" or administrative accounts may use only a few.
Here are two sample entries you might find in /etc/passwd:
root:ZxPsI9ZjiVd9Y:0:0:The root of all evil:/root:/bin/bash
aclark:BjDf5hBysDsii:104:50:Anna Clark:/home/aclark:/bin/bash
The first entry is for the root account. First of all, notice that the uid of root is zero. This is what makes root root: the system knows that uid 0 is "special" and that it does not have the usual security restrictions. The gid of root is also zip, which is mostly a convention. Many of the files on the system are owned by root and the root group, which have a uid and gid of zero, respectively. More on groups in a minute.
On many systems, root uses the home directory /root, or just /. This is not usually relevant, because you most often use su to access root from your own account. Also, it is tradition to use a Bourne-shell variant (in this case /bin/bash) for the root account, although you can use C shell if you like. (Shells are discussed in the section "Section 4.5, "Shells"" in Chapter 4, "Basic Unix Commands and Concepts".) Be careful, though: Bourne shells and C shells have differing syntax, and switching between them when using root can be confusing and lead to mistakes.
The second entry is for an actual human being, username aclark. In this case, the uid is 104. The uid field can technically be any unique integer; on many systems, it's customary to have user accounts numbered 100 and above and administrative accounts in the sub-100 range. The gid is 50, which just means that aclark is in whatever group is numbered 50 in the /etc/group file. Hang on to your horses; groups are covered in section "Section 5.7.4, "The Group File"" later in this chapter.
Home directories are often found in /home, and named for the username of their owner. This is, for the most part, a useful convention that avoids confusion when finding a particular user's home directory, but you can technically place a home directory anywhere. You should, however, observe the directory layout used on your system.
Note that as the system administrator, it's not usually necessary to modify the /etc/passwd file directly. There are several programs available that can help you create and maintain user accounts; see the section "Section 5.7.5, "Creating Accounts"" that follows.
5.7.2. Shadow Passwords
To some extent, it is a security risk to let everybody with access to the system view the encrypted passwords in /etc/passwd. Special crack programs are available that try a huge number of possible passwords and check whether the encrypted version of those passwords is equal to a specified one.
To overcome this potential security risk, shadow passwords have been invented. When shadow passwords are used, the password field in /etc/passwd contains only an x or a *, which can never occur in the encrypted version of a password. Instead, a second file called /etc/shadow is used. This file contains entries that look very similar to those in /etc/passwd, but contain the real encrypted password in the password field. /etc/shadow is readable only by root, so that normal users do not have access to the encrypted passwords. The other fields in /etc/shadow, except the username and the password are present as well, but normally contain bogus values or are empty.
Note that in order to use shadow passwords, you need special versions of the programs that access or modify user information like passwd or login. Nowadays, most distributions come with shadow passwords already set up so that this should not be a problem for you.
There are two tools for converting "normal" user entries to shadow entries and back. pwconv takes the /etc/passwd file, looks for entries that are not yet present in /etc/shadow, generates shadow entries for those and merges them with the entries already present in /etc/shadow.
Debian users should use "shadowconfig on" instead to ensure that shadow passwords are enabled on their systems.
pwunconv is rarely used, because it gives you less security instead of more. It works like pwconv, but generates traditional /etc/passwd entries that work without /etc/shadow counterparts.
5.7.3. PAM and Other Authentication Methods
You might think that having two means of user authentication, /etc/passwd and /etc/shadow, is already enough choice, but you are wrong in this case. There are a number of other authentification methods with strange names like Kerberos authentication (so named after the dog from Greek mythology that guards the entrance to Hell). While we think that shadow passwords provide enough security for almost all cases, it all depends on how much security you really need and how paranoid you want to be.
The problem with all those authentication methods is that you cannot simply switch from one to another, because you always need a set of programs like login and passwd that go with those tools. To overcome this problem, the Pluggable Authentification Methods (PAM) system has been invented. Once you have a PAM-enabled set of tools, you can change the authentification method of your system by reconfiguring PAM. The tools will automatically get the code necessary to perform the required authentication procedures from dynamically loaded shared libraries.
Setting up and using PAM is beyond the scope of this book, but you can get all the information you need from http://www.de.kernel.org/pub/linux/libs/pam/.
5.7.4. The Group File
User groups are a convenient way to logically organize sets of user accounts and allow users to share files within their group or groups. Each file on the system has both a user and a group owner associated with it. Using ls -l, you can see the owner and group for a particular file, as in:
rutabaga% ls -l boiler.tex
-rwxrw-r-- 1 mdw megabozo 10316 Oct 6 20:19 boiler.tex
rutabaga%
This file is owned by the user mdw and belongs to the megabozo group. We can see from the file permissions that mdw has read, write, and execute access to the file; that anyone in the megabozo group has read and write access; and that all other users have read access only.
This doesn't mean that mdw is in the megabozo group; it simply means the file may be accessed, as shown by the permissions bits, by anyone in the megabozo group (which may or may not include mdw).
This way files can be shared among groups of users, and permissions can be specified separately for the owner of the file, the group to which the file belongs, and everyone else. An introduction to permissions appears in the section "Section 4.13, "File Ownership and Permissions"" in Chapter 4, "Basic Unix Commands and Concepts".
Every user is assigned to at least one group, which you specify in the gid field of the /etc/passwd file. However, a user can be a member of multiple groups. The file /etc/group contains a one-line entry for each group on the system, very similar in nature to /etc/passwd. The format of this file is:
groupname:password:gid:members
Here, groupname is a character string identifying the group; it is the group name printed when using commands such as ls -l.
password is an optional password associated with the group, which allows users not in this group to access the group with the newgrp command. Read on for information on this.
gid is the group ID used by the system to refer to the group; it is the number used in the gid field of /etc/passwd to specify a user's default group.
members is a comma-separated list of usernames (with no whitespace in between), identifying those users who are members of this group, but who have a different gid in /etc/passwd. That is, this list need not contain those users who have this group set as their "default" group in /etc/passwd; it's only for users who are additional members of the group.
For example, /etc/group might contain the following entries:
root:*:0:
bin:*:1:root,daemon
users:*:50:
bozo:*:51:linus,mdw
megabozo:*:52:kibo
The first entries, for the groups root and bin, are administrative groups, similar in nature to the "imaginary" accounts used on the system. Many files are owned by groups such as root and bin. The other groups are for user accounts. Like user IDs, the group ID values for user groups are often placed in ranges above 50 or 100.
The password field of the group file is something of a curiosity. It isn't used much, but in conjunction with the newgrp program it allows users who aren't members of a particular group to assume that group ID if they have the password. For example, using the command:
rutabaga% newgrp bozo
Password: password for group bozo
rutabaga%
starts a new shell with the group ID of bozo. If the password field is blank, or the first character is an asterisk, you receive a permission denied error if you attempt to newgrp to that group.
However, the password field of the group file is seldom used and is really not necessary. (In fact, most systems don't provide tools to set the password for a group; you could use passwd to set the password for a dummy user with the same name as the group in /etc/passwd and copy the encrypted password field to /etc/group.) Instead, you can make a user a member of multiple groups simply by including the username in the members field for each additional group. In the previous example, the users linus and mdw are members of the bozo group, as well as whatever group they are assigned to in the /etc/passwd file. If we wanted to add linus to the megabozo group as well, we'd change the last line of the previous example to:
megabozo:*:52:kibo,linus
The command groups tells you which groups you belong to, as in:
rutabaga% groups
users bozo
Giving a list of usernames to groups lists the groups each user in the list belongs to.
When you log in, you are automatically assigned to the group ID given in /etc/passwd, as well as any additional groups for which you're listed in /etc/group. This means you have "group access" to any files on the system with a group ID contained in your list of groups. In this case, the group permission bits (set with chmod g+…) for those files apply to you. (Unless you're the owner, in which case the owner permission bits apply, instead.)
Now that you know the ins and outs of groups, how should you assign groups on your system? This is really a matter of style and depends on how your system will be used. For systems with just one or a handful of users, it's easiest to have a single group (called, say, users) to which all personal user accounts belong. Note that all the system groups--those groups contained within /etc/group when the system is first installed--should probably be left alone. Various daemons and programs may depend upon them.
If you have a number of users on your machine, there are several ways to organize groups. For example, an educational institution may have separate groups for students, faculty, and staff. A software company might have different groups for each design team. On other systems, each user is placed into a separate group, named identically to the username. This keeps each pigeon in its own hole, so to speak, and allows users to share files with a particular group. However, adding a user to an additional group usually requires the system administrator to intervene (by editing /etc/group ; Debian has the utility gpasswd ). It's really up to you.
Another situation where groups are often used is special hardware groups. Let's say that you have a scanner that is accessed via /dev/scanner. If you do not want to give everybody access to the scanner, you could create a special group called scanner, assign /dev/scanner to this group, make this special file readable for the group and nonreadable for everybody else, and add everybody who is allowed to use the scanner to the scanner group in the /etc/groups file.
5.7.5. Creating Accounts
Creating a user account requires several steps: adding an entry to /etc/passwd, creating the user's home directory, and setting up the user's default configuration files (such as .bashrc) in her home directory. Luckily, you don't have to perform these steps manually; nearly all Linux systems include a program called adduser to do this for you.
To create a new account manually, follow these steps:
1. Edit /etc/passwd with vipw and add a new line for the new account. Be careful with the syntax. Do not edit directly with an editor! vipw locks the file, so that other commands won’t try to update it at the same time. You should make the password field be “*”, so that it is impossible to log in.
2. Similarly, edit /etc/group with vigr, if you need to create a new group as well.
3. Create the home directory of the user with mkdir.
4. Copy the files from /etc/skel to the new home directory.
5. Fix ownerships and permissions with chown and chmod. The -R option is most useful. The correct permissions vary a little from one site to another, but usually the following commands do the right thing:
cd /home/newusername
chown -R username.group .
chmod -R go=u,go-w .
chmod go= .
Set the password with passwd.
After you set the password in the last step, the account will work. You shouldn’t set it until everything else has been done, otherwise the user may inadvertently log in while you’re still copying the files.
It is sometimes necessary to create dummy accounts that are not used by people. For example, to set up an anonymous FTP server (so that anyone can download files from it, without having to get an account first), you need to create an account called ftp. In such cases, it is usually not necessary to set the password (last step above). Indeed, it is better not to, so that no one can use the account, unless they first become root, since root can become any user.
Changing User Properties
There are a few commands for changing various properties of an account (i.e., the relevant field in /etc/passwd):
• chfn—changes the full name field
• chsh—changes the login shell
• passwd—changes the password
The super-user may use these commands to change the properties of any account. Normal users can only change the properties of their own account. It may sometimes be necessary to disable these commands (with chmod) for normal users, for example in an environment with many novice users.
Other tasks need to be done by hand. For example, to change the username, you need to edit /etc/passwd directly (with vipw, remember). Likewise, to add or remove the user to more groups, you need to edit /etc/group (with vigr). Such tasks tend to be rare, however, and should be done with caution: For example, if you change the username, email will no longer reach the user, unless you also create a mail alias. (The user’s name might change due to marriage, for example, and he might want to have his username reflect his new name.)
5.7.6. Deleting and Disabling Accounts
Deleting a user account is much easier than creating one; this is the well-known concept of entropy at work. To delete an account, you must remove the user's entry in /etc/passwd, remove any references to the user in /etc/group, and delete the user's home directory, as well as any additional files created or owned by the user. For example, if the user has an incoming mailbox in /var/spool/mail, it must be deleted as well.
The command userdel (the yin to useradd's yang) deletes an account and the account's home directory. For example:
userdel -r norbert
will remove the recently created account for norbert. The -r option forces the home directory to be removed as well. Other files associated with the user--for example, the incoming mailbox, crontab files, and so forth--must be removed by hand. Usually these are quite insignificant and can be left around. By the end of this chapter, you should know where these files are, if they exist. A quick way to find the files associated with a particular user is through the command:
find / -user username -ls
This will give an ls -l listing of each file owned by username. Of course, to use this, the account associated with username must still have an entry in /etc/passwd. If you deleted the account, use the -uid num argument instead, where num is the numeric user ID of the dearly departed user.
Temporarily (or not-so-temporarily) disabling a user account, for whatever reason, is even simpler. You can either remove the user's entry in /etc/passwd (leaving the home directory and other files intact), or add an asterisk to the first character of the password field of the /etc/passwd entry, as so:
aclark:*BjDf5hBysDsii:104:50:Anna Clark:/home/aclark:/bin/bash
This will disallow logins to the account in question.
5.7.7. Modifying User Accounts
Modifying attributes of user accounts and groups is usually a simple matter of editing /etc/passwd and /etc/group. Many systems provide commands such as usermod and groupmod to do just this; it's often easier to edit the files by hand.
To change a user's password, use the passwd command, which will prompt for a password, encrypt it, and store the encrypted password in the /etc/passwd file.
If you need to change the user ID of an existing account, you can do this by editing the uid field of /etc/passwd directly. However, you should also chown the files owned by the user to that of the new uid. For example:
chown -R aclark /home/aclark
will set the ownership for all files in the home directory used by aclark back to aclark, if you changed the uid for this account. If ls -l prints a numeric user ID, instead of a username, this means there is no username associated with the uid owning the files. Use chown to fix this.
Restricted Shells
Disabled commands in restricted shells
Running a script or portion of a script in restricted mode disables certain commands that would otherwise be available. This is a security measure intended to limit the privileges of the script user and to minimize possible damage from running the script.
Using cd to change the working directory.
Changing the values of the $PATH, $SHELL, $BASH_ENV, or $ENV environmental variables.
Reading or changing the $SHELLOPTS, shell environmental options.
Output redirection.
Invoking commands containing one or more /'s.
Invoking exec to substitute a different process for the shell.
Various other commands that would enable monkeying with or attempting to subvert the script for an unintended purpose.
Getting out of restricted mode within the script.
Example 21-1. Running a script in restricted mode
#!/bin/bash # Starting the script with "#!/bin/bash -r" # runs entire script in restricted mode. echo echo "Changing directory." cd /usr/local echo "Now in `pwd`" echo "Coming back home." cd echo "Now in `pwd`" echo # Everything up to here in normal, unrestricted mode. set -r # set --restricted has same effect. echo "==> Now in restricted mode. <==" echo echo echo "Attempting directory change in restricted mode." cd .. echo "Still in `pwd`" echo echo echo "\$SHELL = $SHELL" echo "Attempting to change shell in restricted mode." SHELL="/bin/ash" echo echo "\$SHELL= $SHELL" echo echo echo "Attempting to redirect output in restricted mode." ls -l /usr/bin > bin.files ls -l bin.files # Try to list attempted file creation effort. echo exit 0 |
Linux User Management command
ac | Print statistics about users' connect time. | |
accton | Turn on accounting of processes. To turn it on type "accton /var/log/pacct". | |
adduser | Ex: adduser mark - Effect: Adds a user to the system named mark | |
chage | Used to change the time the user's password will expire. | |
chfn | Change the user full name field finger information | |
chgrp | Changes the group ownership of files. | |
chown | Change the owner of file(s ) to another user. | |
chpasswd | Update password file in batch. | |
chroot | Run command or interactive shell with special root directory. | |
chsh | Change the login shell. | |
edquota | Used to edit user or group quotas. This program uses the vi editor to edit the quota.user and quota.group files. If the environment variable EDITOR is set to emacs, the emacs editor will be used. Type "export EDITOR=emacs" to set that variable. | |
faillog | Examine faillog and set login failure limits. | |
finger | See what users are running on a system. | |
gpasswd | Administer the /etc/group file. | |
groupadd | Create a new group. | |
grpck | Verify the integrity of group files. | |
grpconv | Creates /etc/gshadow from the file /etc/group which converts to shadow passwords. | |
grpunconv | Uses the files /etc/passwd and /etc/shadow to create /etc/passwd, then deletes /etc/shadow which converts from shadow passwords. | |
groupdel | Delete a group. | |
groupmod | Modify a group. | |
groups | Print the groups a user is in | |
id | Print real and effective user id and group ids. | |
last | Display the last users logged on and how long. | |
lastb | Shows failed login attempts. This command requires the file /var/log/btmp to exist in order to work. Type "touch /var/log/btmp" to begin logging to this file. | |
lastcomm | Display information about previous commands in reverse order. Works only if process accounting is on. | |
lastlog | Formats and prints the contents of the last login. | |
logname | Print user's login name. | |
newgrp | Lets a suer log in to a new group. | |
newusers | Update and create newusers in batch. | |
passwd | Set a user's pass word. | |
pwck | Verify integrity of password files. | |
pwconv | Convert to and from shadow passwords and groups. | |
quota | Display users' limits and current disk usage. | |
quotaoff | Turns system quotas off. | |
quotaon | Turns system quotas on. | |
quotacheck | Used to check a filesystem for usage, and update the quota.user file. | |
repquota | Lists a summary of quota information on filesystems. | |
sa | Generates a summary of information about users' processes that are stored in the /var/log/pacct file. | |
smbclient | Works similar to an ftp client enabling the user to transfer files to and from a windows based computer. | |
smbmount | Allows a shared directory on a windows machine to be mounted on the Linux machine. | |
smbpasswd | Program to change users passwords for samba. | |
su | Ex: su mark - Effect: changes the user to mark, If not root will need marks password. | |
sulogin | Single user login. | |
ulimit | A bash builtin command for setting the processes a user can run. | |
useradd | Create a new user or update default new user information. | |
userdel | Delete a user account and related files. | |
usermod | Modify a user account. | |
users | Print the user names of users currently logged in. | |
utmpdump | Used for debugging. | |
vigr | Edit the password or group files. | |
vipw | Edit the password or group files. | |
w | Display users logged in and what they are doing. | |
wall | Send a message to everybody's terminal. | |
who | Display the users logged in. | |
whoami | Print effective user id. |
Switch user account to root: Shell/command mode:
su –
Managing Unix File Systems: [2L]
Partitions, Swap space, Device files, Raw and Block files, Formatting disks, Making filesystems, Superblock, I-nodes, Filesystem checker, Mounting filesystems, Logical Volumes, Network Filesystems, Boot disks
Partitions
A hard disk can be divided into several partitions. Each partition functions as if it were a separate hard disk. The idea is that if you have one hard disk, and want to have, say, two operating systems on it, you can divide the disk into two partitions. Each operating system uses its partition as it wishes and doesn't touch the other one's. This way the two operating systems can co-exist peacefully on the same hard disk. Without partitions one would have to buy a hard disk for each operating system.
Floppies are not partitioned. There is no technical reason against this, but since they're so small, partitions would be useful only very rarely. CD-ROM's are usually also not partitioned, since it's easier to use them as one big disk, and there is seldom a need to have several operating systems on one.
The MBR, boot sectors and partition table
The information about how a hard disk has been partitioned is stored in its first sector (that is, the first sector of the first track on the first disk surface). The first sector is the master boot record (MBR) of the disk; this is the sector that the BIOS reads in and starts when the machine is first booted. The master boot record contains a small program that reads the partition table, checks which partition is active (that is, marked bootable), and reads the first sector of that partition, the partition's boot sector (the MBR is also a boot sector, but it has a special status and therefore a special name). This boot sector contains another small program that reads the first part of the operating system stored on that partition (assuming it is bootable), and then starts it.
The partitioning scheme is not built into the hardware, or even into the BIOS. It is only a convention that many operating systems follow. Not all operating systems do follow it, but they are the exceptions. Some operating systems support partitions, but they occupy one partition on the hard disk, and use their internal partitioning method within that partition. The latter type exists peacefully with other operating systems (including Linux), and does not require any special measures, but an operating system that doesn't support partitions cannot co-exist on the same disk with any other operating system.
As a safety precaution, it is a good idea to write down the partition table on a piece of paper, so that if it ever corrupts you don't have to lose all your files. (A bad partition table can be fixed with fdisk). The relevant information is given by the fdisk -l command:
$ fdisk -l /dev/hda
Disk /dev/hda: 15 heads, 57 sectors, 790 cylinders
Units = cylinders of 855 * 512 bytes
Device Boot Begin Start End Blocks Id System
/dev/hda1 1 1 24 10231+ 82 Linux swap
/dev/hda2 25 25 48 10260 83 Linux native
/dev/hda3 49 49 408 153900 83 Linux native
/dev/hda4 409 409 790 163305 5 Extended
/dev/hda5 409 409 744 143611+ 83 Linux native
/dev/hda6 745 745 790 19636+ 83 Linux native
$
Extended and logical partitions
The original partitioning scheme for PC hard disks allowed only four partitions. This quickly turned out to be too little in real life, partly because some people want more than four operating systems (Linux, MS-DOS, OS/2, Minix, FreeBSD, NetBSD, or Windows/NT, to name a few), but primarily because sometimes it is a good idea to have several partitions for one operating system. For example, swap space is usually best put in its own partition for Linux instead of in the main Linux partition for reasons of speed (see below).
To overcome this design problem, extended partitions were invented. This trick allows partitioning a primary partition into sub-partitions. The primary partition thus subdivided is the extended partition; the subpartitions are logical partitions. They behave like primary [1] partitions, but are created differently. There is no speed difference between them.
The partition structure of a hard disk might look like that in Figure 4-2. The disk is divided into three primary partitions, the second of which is divided into two logical partitions. Part of the disk is not partitioned at all. The disk as a whole and each primary partition has a boot sector.
Figure 4-2. A sample hard disk partitioning.
Partition types
The partition tables (the one in the MBR, and the ones for extended partitions) contain one byte per partition that identifies the type of that partition. This attempts to identify the operating system that uses the partition, or what it uses it for. The purpose is to make it possible to avoid having two operating systems accidentally using the same partition. However, in reality, operating systems do not really care about the partition type byte; e.g., Linux doesn't care at all what it is. Worse, some of them use it incorrectly; e.g., at least some versions of DR-DOS ignore the most significant bit of the byte, while others don't.
There is no standardization agency to specify what each byte value means, but some commonly accepted ones are included in in Table 4-1. The same list is available in the Linux fdisk program.
Table 4-1. Partition types (from the Linux fdisk program).
0 | Empty | 40 | Venix 80286 | 94 | Amoeba BBT |
1 | DOS 12-bit FAT | 51 | Novell? | a5 | BSD/386 |
2 | XENIX root | 52 | Microport | b7 | BSDI fs |
3 | XENIX usr | 63 | GNU HURD | b8 | BSDI swap |
4 | DOS 16-bitf <32M | 64 | Novell | c7 | Syrinx |
5 | Extended | 75 | PC/IX | db | CP/M |
6 | DOS 16-bit >=32M | 80 | Old MINIX | e1 | DOS access |
7 | OS/2 HPFS | 81 | Linux/MINIX | e3 | DOS R/O |
8 | AIX | 82 | Linux swap | f2 | DOS secondary |
9 | AIX bootable | 83 | Linux native | ff | BBT |
a | OS/2 Boot Manag | 93 | Amoeba |
Partitioning a hard disk
There are many programs for creating and removing partitions. Most operating systems have their own, and it can be a good idea to use each operating system's own, just in case it does something unusual that the others can't. Many of the programs are called fdisk, including the Linux one, or variations thereof. Details on using the Linux fdisk are given on its man page. The cfdisk command is similar to fdisk, but has a nicer (full screen) user interface.
When using IDE disks, the boot partition (the partition with the bootable kernel image files) must be completely within the first 1024 cylinders. This is because the disk is used via the BIOS during boot (before the system goes into protected mode), and BIOS can't handle more than 1024 cylinders. It is sometimes possible to use a boot partition that is only partly within the first 1024 cylinders. This works as long as all the files that are read with the BIOS are within the first 1024 cylinders. Since this is difficult to arrange, it is a very bad idea to do it; you never know when a kernel update or disk defragmentation will result in an unbootable system. Therefore, make sure your boot partition is completely within the first 1024 cylinders.
Some newer versions of the BIOS and IDE disks can, in fact, handle disks with more than 1024 cylinders. If you have such a system, you can forget about the problem; if you aren't quite sure of it, put it within the first 1024 cylinders.
Each partition should have an even number of sectors, since the Linux filesystems use a 1 kilobyte block size, i.e., two sectors. An odd number of sectors will result in the last sector being unused. This won't result in any problems, but it is ugly, and some versions of fdisk will warn about it.
Changing a partition's size usually requires first backing up everything you want to save from that partition (preferably the whole disk, just in case), deleting the partition, creating new partition, then restoring everything to the new partition. If the partition is growing, you may need to adjust the sizes (and backup and restore) of the adjoining partitions as well.
Since changing partition sizes is painful, it is preferable to get the partitions right the first time, or have an effective and easy to use backup system. If you're installing from a media that does not require much human intervention (say, from CD-ROM, as opposed to floppies), it is often easy to play with different configuration at first. Since you don't already have data to back up, it is not so painful to modify partition sizes several times.
There is a program for MS-DOS, called fips, which resizes an MS-DOS partition without requiring the backup and restore, but for other filesystems it is still necessary.
Device files and partitions
Each partition and extended partition has its own device file. The naming convention for these files is that a partition's number is appended after the name of the whole disk, with the convention that 1-4 are primary partitions (regardless of how many primary partitions there are) and 5-8 are logical partitions (regardless of within which primary partition they reside). For example, /dev/hda1 is the first primary partition on the first IDE hard disk, and /dev/sdb7 is the third extended partition on the second SCSI hard disk. The device list in XXX (device list) gives more information.
Memory Management
Table of Contents
Sharing swap spaces with other operating systems
"Minnet, jag har tappat mitt minne, är jag svensk eller finne, kommer inte ihåg..." (Bosse Österberg)
This section describes the Linux memory management features, i.e., virtual memory and the disk buffer cache. The purpose and workings and the things the system administrator needs to take into consideration are described.
What is virtual memory?
Linux supports virtual memory, that is, using a disk as an extension of RAM so that the effective size of usable memory grows correspondingly. The kernel will write the contents of a currently unused block of memory to the hard disk so that the memory can be used for another purpose. When the original contents are needed again, they are read back into memory. This is all made completely transparent to the user; programs running under Linux only see the larger amount of memory available and don't notice that parts of them reside on the disk from time to time. Of course, reading and writing the hard disk is slower (on the order of a thousand times slower) than using real memory, so the programs don't run as fast. The part of the hard disk that is used as virtual memory is called the swap space.
Linux can use either a normal file in the filesystem or a separate partition for swap space. A swap partition is faster, but it is easier to change the size of a swap file (there's no need to repartition the whole hard disk, and possibly install everything from scratch). When you know how much swap space you need, you should go for a swap partition, but if you are uncertain, you can use a swap file first, use the system for a while so that you can get a feel for how much swap you need, and then make a swap partition when you're confident about its size.
You should also know that Linux allows one to use several swap partitions and/or swap files at the same time. This means that if you only occasionally need an unusual amount of swap space, you can set up an extra swap file at such times, instead of keeping the whole amount allocated all the time.
A note on operating system terminology: computer science usually distinguishes between swapping (writing the whole process out to swap space) and paging (writing only fixed size parts, usually a few kilobytes, at a time). Paging is usually more efficient, and that's what Linux does, but traditional Linux terminology talks about swapping anyway. [1]
Notes
Thus quite needlessly annoying a number of computer scientists something horrible. |
Creating a swap space
A swap file is an ordinary file; it is in no way special to the kernel. The only thing that matters to the kernel is that it has no holes, and that it is prepared for use with mkswap. It must reside on a local disk, however; it can't reside in a filesystem that has been mounted over NFS due to implementation reasons.
The bit about holes is important. The swap file reserves the disk space so that the kernel can quickly swap out a page without having to go through all the things that are necessary when allocating a disk sector to a file. The kernel merely uses any sectors that have already been allocated to the file. Because a hole in a file means that there are no disk sectors allocated (for that place in the file), it is not good for the kernel to try to use them.
One good way to create the swap file without holes is through the following command:
$ dd if=/dev/zero of=/extra-swap bs=1024 count=1024
1024+0 records in
1024+0 records out
$
where /extra-swap is the name of the swap file and the size of is given after the count=. It is best for the size to be a multiple of 4, because the kernel writes out memory pages, which are 4 kilobytes in size. If the size is not a multiple of 4, the last couple of kilobytes may be unused.
A swap partition is also not special in any way. You create it just like any other partition; the only difference is that it is used as a raw partition, that is, it will not contain any filesystem at all. It is a good idea to mark swap partitions as type 82 (Linux swap); this will the make partition listings clearer, even though it is not strictly necessary to the kernel.
After you have created a swap file or a swap partition, you need to write a signature to its beginning; this contains some administrative information and is used by the kernel. The command to do this is mkswap, used like this:
$ mkswap /extra-swap 1024
Setting up swapspace, size = 1044480 bytes
$
Note that the swap space is still not in use yet: it exists, but the kernel does not use it to provide virtual memory.
You should be very careful when using mkswap, since it does not check that the file or partition isn't used for anything else. You can easily overwrite important files and partitions with mkswap! Fortunately, you should only need to use mkswap when you install your system.
The Linux memory manager limits the size of each swap space to about 127 MB (for various technical reasons, the actual limit is (4096-10) * 8 * 4096 = 133890048$ bytes, or 127.6875 megabytes). You can, however, use up to 8 swap spaces simultaneously, for a total of almost 1 GB. [1]
Using a swap space
An initialized swap space is taken into use with swapon. This command tells the kernel that the swap space can be used. The path to the swap space is given as the argument, so to start swapping on a temporary swap file one might use the following command.
$ swapon /extra-swap
$
Swap spaces can be used automatically by listing them in the /etc/fstab file.
/dev/hda8 none swap sw 0 0
/swapfile none swap sw 0 0
The startup scripts will run the command swapon -a, which will start swapping on all the swap spaces listed in /etc/fstab. Therefore, the swapon command is usually used only when extra swap is needed.
You can monitor the use of swap spaces with free. It will tell the total amount of swap space used.
$ free
total used free shared buffers
Mem: 15152 14896 256 12404 2528
-/+ buffers: 12368 2784
Swap: 32452 6684 25768
$
The first line of output (Mem:) shows the physical memory. The total column does not show the physical memory used by the kernel, which is usually about a megabyte. The used column shows the amount of memory used (the second line does not count buffers). The free column shows completely unused memory. The shared column shows the amount of memory shared by several processes; the more, the merrier. The buffers column shows the current size of the disk buffer cache.
That last line (Swap:) shows similar information for the swap spaces. If this line is all zeroes, your swap space is not activated.
The same information is available via top, or using the proc filesystem in file /proc/meminfo. It is currently difficult to get information on the use of a specific swap space.
A swap space can be removed from use with swapoff. It is usually not necessary to do it, except for temporary swap spaces. Any pages in use in the swap space are swapped in first; if there is not sufficient physical memory to hold them, they will then be swapped out (to some other swap space). If there is not enough virtual memory to hold all of the pages Linux will start to thrash; after a long while it should recover, but meanwhile the system is unusable. You should check (e.g., with free) that there is enough free memory before removing a swap space from use.
All the swap spaces that are used automatically with swapon -a can be removed from use with swapoff -a; it looks at the file /etc/fstab to find what to remove. Any manually used swap spaces will remain in use.
Sometimes a lot of swap space can be in use even though there is a lot of free physical memory. This can happen for instance if at one point there is need to swap, but later a big process that occupied much of the physical memory terminates and frees the memory. The swapped-out data is not automatically swapped in until it is needed, so the physical memory may remain free for a long time. There is no need to worry about this, but it can be comforting to know what is happening
Sharing swap spaces with other operating systems
Virtual memory is built into many operating systems. Since they each need it only when they are running, i.e., never at the same time, the swap spaces of all but the currently running one are being wasted. It would be more efficient for them to share a single swap space. This is possible, but can require a bit of hacking. The Tips-HOWTO contains some advice on how to implement this.
Allocating swap space
Some people will tell you that you should allocate twice as much swap space as you have physical memory, but this is a bogus rule. Here's how to do it properly:
· Estimate your total memory needs. This is the largest amount of memory you'll probably need at a time, that is the sum of the memory requirements of all the programs you want to run at the same time. This can be done by running at the same time all the programs you are likely to ever be running at the same time.
For instance, if you want to run X, you should allocate about 8 MB for it, gcc wants several megabytes (some files need an unusually large amount, up to tens of megabytes, but usually about four should do), and so on. The kernel will use about a megabyte by itself, and the usual shells and other small utilities perhaps a few hundred kilobytes (say a megabyte together). There is no need to try to be exact, rough estimates are fine, but you might want to be on the pessimistic side.
Remember that if there are going to be several people using the system at the same time, they are all going to consume memory. However, if two people run the same program at the same time, the total memory consumption is usually not double, since code pages and shared libraries exist only once.
The free and ps commands are useful for estimating the memory needs.
· Add some security to the estimate in step 1. This is because estimates of program sizes will probably be wrong, because you'll probably forget some programs you want to run, and to make certain that you have some extra space just in case. A couple of megabytes should be fine. (It is better to allocate too much than too little swap space, but there's no need to over-do it and allocate the whole disk, since unused swap space is wasted space; see later about adding more swap.) Also, since it is nicer to deal with even numbers, you can round the value up to the next full megabyte.
· Based on the computations above, you know how much memory you'll be needing in total. So, in order to allocate swap space, you just need to subtract the size of your physical memory from the total memory needed, and you know how much swap space you need. (On some versions of UNIX, you need to allocate space for an image of the physical memory as well, so the amount computed in step 2 is what you need and you shouldn't do the subtraction.)
· If your calculated swap space is very much larger than your physical memory (more than a couple times larger), you should probably invest in more physical memory, otherwise performance will be too low.
It's a good idea to have at least some swap space, even if your calculations indicate that you need none. Linux uses swap space somewhat aggressively, so that as much physical memory as possible can be kept free. Linux will swap out memory pages that have not been used, even if the memory is not yet needed for anything. This avoids waiting for swapping when it is needed: the swapping can be done earlier, when the disk is otherwise idle.
Swap space can be divided among several disks. This can sometimes improve performance, depending on the relative speeds of the disks and the access patterns of the disks. You might want to experiment with a few schemes, but be aware that doing the experiments properly is quite difficult. You should not believe claims that any one scheme is superior to any other, since it won't always be true.
The buffer cache
Reading from a disk [1] is very slow compared to accessing (real) memory. In addition, it is common to read the same part of a disk several times during relatively short periods of time. For example, one might first read an e-mail message, then read the letter into an editor when replying to it, then make the mail program read it again when copying it to a folder. Or, consider how often the command ls might be run on a system with many users. By reading the information from disk only once and then keeping it in memory until no longer needed, one can speed up all but the first read. This is called disk buffering, and the memory used for the purpose is called the buffer cache.
Since memory is, unfortunately, a finite, nay, scarce resource, the buffer cache usually cannot be big enough (it can't hold all the data one ever wants to use). When the cache fills up, the data that has been unused for the longest time is discarded and the memory thus freed is used for the new data.
Disk buffering works for writes as well. On the one hand, data that is written is often soon read again (e.g., a source code file is saved to a file, then read by the compiler), so putting data that is written in the cache is a good idea. On the other hand, by only putting the data into the cache, not writing it to disk at once, the program that writes runs quicker. The writes can then be done in the background, without slowing down the other programs.
Most operating systems have buffer caches (although they might be called something else), but not all of them work according to the above principles. Some are write-through: the data is written to disk at once (it is kept in the cache as well, of course). The cache is called write-back if the writes are done at a later time. Write-back is more efficient than write-through, but also a bit more prone to errors: if the machine crashes, or the power is cut at a bad moment, or the floppy is removed from the disk drive before the data in the cache waiting to be written gets written, the changes in the cache are usually lost. This might even mean that the filesystem (if there is one) is not in full working order, perhaps because the unwritten data held important changes to the bookkeeping information.
Because of this, you should never turn off the power without using a proper shutdown procedure (see Chapter 6), or remove a floppy from the disk drive until it has been unmounted (if it was mounted) or after whatever program is using it has signaled that it is finished and the floppy drive light doesn't shine anymore. The sync command flushes the buffer, i.e., forces all unwritten data to be written to disk, and can be used when one wants to be sure that everything is safely written. In traditional UNIX systems, there is a program called update running in the background which does a sync every 30 seconds, so it is usually not necessary to use sync. Linux has an additional daemon, bdflush, which does a more imperfect sync more frequently to avoid the sudden freeze due to heavy disk I/O that sync sometimes causes.
Under Linux, bdflush is started by update. There is usually no reason to worry about it, but if bdflush happens to die for some reason, the kernel will warn about this, and you should start it by hand (/sbin/update).
The cache does not actually buffer files, but blocks, which are the smallest units of disk I/O (under Linux, they are usually 1 kB). This way, also directories, super blocks, other filesystem bookkeeping data, and non-filesystem disks are cached.
The effectiveness of a cache is primarily decided by its size. A small cache is next to useless: it will hold so little data that all cached data is flushed from the cache before it is reused. The critical size depends on how much data is read and written, and how often the same data is accessed. The only way to know is to experiment.
If the cache is of a fixed size, it is not very good to have it too big, either, because that might make the free memory too small and cause swapping (which is also slow). To make the most efficient use of real memory, Linux automatically uses all free RAM for buffer cache, but also automatically makes the cache smaller when programs need more memory.
Under Linux, you do not need to do anything to make use of the cache, it happens completely automatically. Except for following the proper procedures for shutdown and removing floppies, you do not need to worry about it.
Filesystems
What are filesystems?
A filesystem is the methods and data structures that an operating system uses to keep track of files on a disk or partition; that is, the way the files are organized on the disk. The word is also used to refer to a partition or disk that is used to store the files or the type of the filesystem. Thus, one might say ``I have two filesystems'' meaning one has two partitions on which one stores files, or that one is using the ``extended filesystem'', meaning the type of the filesystem.
The difference between a disk or partition and the filesystem it contains is important. A few programs (including, reasonably enough, programs that create filesystems) operate directly on the raw sectors of a disk or partition; if there is an existing file system there it will be destroyed or seriously corrupted. Most programs operate on a filesystem, and therefore won't work on a partition that doesn't contain one (or that contains one of the wrong type).
Before a partition or disk can be used as a filesystem, it needs to be initialized, and the bookkeeping data structures need to be written to the disk. This process is called making a filesystem.
Most UNIX filesystem types have a similar general structure, although the exact details vary quite a bit. The central concepts are superblock, inode, data block, directory block, and indirection block. The superblock contains information about the filesystem as a whole, such as its size (the exact information here depends on the filesystem). An inode contains all information about a file, except its name. The name is stored in the directory, together with the number of the inode. A directory entry consists of a filename and the number of the inode which represents the file. The inode contains the numbers of several data blocks, which are used to store the data in the file. There is space only for a few data block numbers in the inode, however, and if more are needed, more space for pointers to the data blocks is allocated dynamically. These dynamically allocated blocks are indirect blocks; the name indicates that in order to find the data block, one has to find its number in the indirect block first.
UNIX filesystems usually allow one to create a hole in a file (this is done with lseek; check the manual page), which means that the filesystem just pretends that at a particular place in the file there is just zero bytes, but no actual disk sectors are reserved for that place in the file (this means that the file will use a bit less disk space). This happens especially often for small binaries, Linux shared libraries, some databases, and a few other special cases. (Holes are implemented by storing a special value as the address of the data block in the indirect block or inode. This special address means that no data block is allocated for that part of the file, ergo, there is a hole in the file.)
Holes are moderately useful. On the author's system, a simple measurement showed a potential for about 4 MB of savings through holes of about 200 MB total used disk space. That system, however, contains relatively few programs and no database files.
Filesystems galore
Linux supports several types of filesystems. As of this writing the most important ones are:
minix
The oldest, presumed to be the most reliable, but quite limited in features (some time stamps are missing, at most 30 character filenames) and restricted in capabilities (at most 64 MB per filesystem).
xia
A modified version of the minix filesystem that lifts the limits on the filenames and filesystem sizes, but does not otherwise introduce new features. It is not very popular, but is reported to work very well.
ext2
The most featureful of the native Linux filesystems, currently also the most popular one. It is designed to be easily upwards compatible, so that new versions of the filesystem code do not require re-making the existing filesystems.
ext
An older version of ext2 that wasn't upwards compatible. It is hardly ever used in new installations any more, and most people have converted to ext2.
In addition, support for several foreign filesystem exists, to make it easier to exchange files with other operating systems. These foreign filesystems work just like native ones, except that they may be lacking in some usual UNIX features, or have curious limitations, or other oddities.
msdos
Compatibility with MS-DOS (and OS/2 and Windows NT) FAT filesystems.
usmdos
Extends the msdos filesystem driver under Linux to get long filenames, owners, permissions, links, and device files. This allows a normal msdos filesystem to be used as if it were a Linux one, thus removing the need for a separate partition for Linux.
iso9660
The standard CD-ROM filesystem; the popular Rock Ridge extension to the CD-ROM standard that allows longer file names is supported automatically.
nfs
A networked filesystem that allows sharing a filesystem between many computers to allow easy access to the files from all of them.
hpfs
The OS/2 filesystem.
sysv
SystemV/386, Coherent, and Xenix filesystems.
The choice of filesystem to use depends on the situation. If compatibility or other reasons make one of the non-native filesystems necessary, then that one must be used. If one can choose freely, then it is probably wisest to use ext2, since it has all the features but does not suffer from lack of performance.
There is also the proc filesystem, usually accessible as the /proc directory, which is not really a filesystem at all, even though it looks like one. The proc filesystem makes it easy to access certain kernel data structures, such as the process list (hence the name). It makes these data structures look like a filesystem, and that filesystem can be manipulated with all the usual file tools. For example, to get a listing of all processes one might use the command
$ ls -l /proc
total 0
dr-xr-xr-x 4 root root 0 Jan 31 20:37 1
dr-xr-xr-x 4 liw users 0 Jan 31 20:37 63
dr-xr-xr-x 4 liw users 0 Jan 31 20:37 94
dr-xr-xr-x 4 liw users 0 Jan 31 20:37 95
dr-xr-xr-x 4 root users 0 Jan 31 20:37 98
dr-xr-xr-x 4 liw users 0 Jan 31 20:37 99
-r--r--r-- 1 root root 0 Jan 31 20:37 devices
-r--r--r-- 1 root root 0 Jan 31 20:37 dma
-r--r--r-- 1 root root 0 Jan 31 20:37 filesystems
-r--r--r-- 1 root root 0 Jan 31 20:37 interrupts
-r-------- 1 root root 8654848 Jan 31 20:37 kcore
-r--r--r-- 1 root root 0 Jan 31 11:50 kmsg
-r--r--r-- 1 root root 0 Jan 31 20:37 ksyms
-r--r--r-- 1 root root 0 Jan 31 11:51 loadavg
-r--r--r-- 1 root root 0 Jan 31 20:37 meminfo
-r--r--r-- 1 root root 0 Jan 31 20:37 modules
dr-xr-xr-x 2 root root 0 Jan 31 20:37 net
dr-xr-xr-x 4 root root 0 Jan 31 20:37 self
-r--r--r-- 1 root root 0 Jan 31 20:37 stat
-r--r--r-- 1 root root 0 Jan 31 20:37 uptime
-r--r--r-- 1 root root 0 Jan 31 20:37 version
$
(There will be a few extra files that don't correspond to processes, though. The above example has been shortened.)
Note that even though it is called a filesystem, no part of the proc filesystem touches any disk. It exists only in the kernel's imagination. Whenever anyone tries to look at any part of the proc filesystem, the kernel makes it look as if the part existed somewhere, even though it doesn't. So, even though there is a multi-megabyte /proc/kcore file, it doesn't take any disk space.
Which filesystem should be used?
There is usually little point in using many different filesystems. Currently, ext2fs is the most popular one, and it is probably the wisest choice. Depending on the overhead for bookkeeping structures, speed, (perceived) reliability, compatibility, and various other reasons, it may be advisable to use another file system. This needs to be decided on a case-by-case basis.
Creating a filesystem
Filesystems are created, i.e., initialized, with the mkfs command. There is actually a separate program for each filesystem type. mkfs is just a front end that runs the appropriate program depending on the desired filesystem type. The type is selected with the -t fstype option.
The programs called by mkfs have slightly different command line interfaces. The common and most important options are summarized below; see the manual pages for more.
-t fstype
Select the type of the filesystem.
-c
Search for bad blocks and initialize the bad block list accordingly.
-l filename
Read the initial bad block list from the name file.
To create an ext2 filesystem on a floppy, one would give the following commands:
$ fdformat -n /dev/fd0H1440
Double-sided, 80 tracks, 18 sec/track. Total capacity 1440 kB.
Formatting ... done
$ badblocks /dev/fd0H1440 1440 $>$ bad-blocks
$ mkfs -t ext2 -l bad-blocks /dev/fd0H1440
mke2fs 0.5a, 5-Apr-94 for EXT2 FS 0.5, 94/03/10
360 inodes, 1440 blocks
72 blocks (5.00%) reserved for the super user
First data block=1
Block size=1024 (log=0)
Fragment size=1024 (log=0)
1 block group
8192 blocks per group, 8192 fragments per group
360 inodes per group
Writing inode tables: done
Writing superblocks and filesystem accounting information: done
$
First, the floppy was formatted (the -n option prevents validation, i.e., bad block checking). Then bad blocks were searched with badblocks, with the output redirected to a file, bad-blocks. Finally, the filesystem was created, with the bad block list initialized by whatever badblocks found.
The -c option could have been used with mkfs instead of badblocks and a separate file. The example below does that.
$ mkfs -t ext2 -c /dev/fd0H1440
mke2fs 0.5a, 5-Apr-94 for EXT2 FS 0.5, 94/03/10
360 inodes, 1440 blocks
72 blocks (5.00%) reserved for the super user
First data block=1
Block size=1024 (log=0)
Fragment size=1024 (log=0)
1 block group
8192 blocks per group, 8192 fragments per group
360 inodes per group
Checking for bad blocks (read-only test): done
Writing inode tables: done
Writing superblocks and filesystem accounting information: done
$
The -c option is more convenient than a separate use of badblocks, but badblocks is necessary for checking after the filesystem has been created.
The process to prepare filesystems on hard disks or partitions is the same as for floppies, except that the formatting isn't needed.
Mounting and unmounting
Before one can use a filesystem, it has to be mounted. The operating system then does various bookkeeping things to make sure that everything works. Since all files in UNIX are in a single directory tree, the mount operation will make it look like the contents of the new filesystem are the contents of an existing subdirectory in some already mounted filesystem.
For example, Figure 4-3 shows three separate filesystems, each with their own root directory. When the last two filesystems are mounted below /home and /usr, respectively, on the first filesystem, we can get a single directory tree, as in Figure 4-4.
Figure 4-3. Three separate filesystems.
Figure 4-4. /home and /usr have been mounted.
The mounts could be done as in the following example:
$ mount /dev/hda2 /home
$ mount /dev/hda3 /usr
$
The mount command takes two arguments. The first one is the device file corresponding to the disk or partition containing the filesystem. The second one is the directory below which it will be mounted. After these commands the contents of the two filesystems look just like the contents of the /home and /usr directories, respectively. One would then say that ``/dev/hda2 is mounted on /home'', and similarly for /usr. To look at either filesystem, one would look at the contents of the directory on which it has been mounted, just as if it were any other directory. Note the difference between the device file, /dev/hda2, and the mounted-on directory, /home. The device file gives access to the raw contents of the disk, the mounted-on directory gives access to the files on the disk. The mounted-on directory is called the mount point.
Linux supports many filesystem types. mount tries to guess the type of the filesystem. You can also use the -t fstype option to specify the type directly; this is sometimes necessary, since the heuristics mount uses do not always work. For example, to mount an MS-DOS floppy, you could use the following command:
$ mount -t msdos /dev/fd0 /floppy
$
The mounted-on directory need not be empty, although it must exist. Any files in it, however, will be inaccessible by name while the filesystem is mounted. (Any files that have already been opened will still be accessible. Files that have hard links from other directories can be accessed using those names.) There is no harm done with this, and it can even be useful. For instance, some people like to have /tmp and /var/tmp synonymous, and make /tmp be a symbolic link to /var/tmp. When the system is booted, before the /var filesystem is mounted, a /var/tmp directory residing on the root filesystem is used instead. When /var is mounted, it will make the /var/tmp directory on the root filesystem inaccessible. If /var/tmp didn't exist on the root filesystem, it would be impossible to use temporary files before mounting /var.
If you don't intend to write anything to the filesystem, use the -r switch for mount to do a readonly mount. This will make the kernel stop any attempts at writing to the filesystem, and will also stop the kernel from updating file access times in the inodes. Read-only mounts are necessary for unwritable media, e.g., CD-ROM's.
The alert reader has already noticed a slight logistical problem. How is the first filesystem (called the root filesystem, because it contains the root directory) mounted, since it obviously can't be mounted on another filesystem? Well, the answer is that it is done by magic. [1] The root filesystem is magically mounted at boot time, and one can rely on it to always be mounted. If the root filesystem can't be mounted, the system does not boot. The name of the filesystem that is magically mounted as root is either compiled into the kernel, or set using LILO or rdev.
The root filesystem is usually first mounted readonly. The startup scripts will then run fsck to verify its validity, and if there are no problems, they will re-mount it so that writes will also be allowed. fsck must not be run on a mounted filesystem, since any changes to the filesystem while fsck is running will cause trouble. Since the root filesystem is mounted readonly while it is being checked, fsck can fix any problems without worry, since the remount operation will flush any metadata that the filesystem keeps in memory.
On many systems there are other filesystems that should also be mounted automatically at boot time. These are specified in the /etc/fstab file; see the fstab man page for details on the format. The details of exactly when the extra filesystems are mounted depend on many factors, and can be configured by each administrator if need be; see Chapter 6.
When a filesystem no longer needs to be mounted, it can be unmounted with umount. [2] umount takes one argument: either the device file or the mount point. For example, to unmount the directories of the previous example, one could use the commands
$ umount /dev/hda2
$ umount /usr
$
See the man page for further instructions on how to use the command. It is imperative that you always unmount a mounted floppy. Don't just pop the floppy out of the drive! Because of disk caching, the data is not necessarily written to the floppy until you unmount it, so removing the floppy from the drive too early might cause the contents to become garbled. If you only read from the floppy, this is not very likely, but if you write, even accidentally, the result may be catastrophic.
Mounting and unmounting requires super user privileges, i.e., only root can do it. The reason for this is that if any user can mount a floppy on any directory, then it is rather easy to create a floppy with, say, a Trojan horse disguised as /bin/sh, or any other often used program. However, it is often necessary to allow users to use floppies, and there are several ways to do this:
· Give the users the root password. This is obviously bad security, but is the easiest solution. It works well if there is no need for security anyway, which is the case on many non-networked, personal systems.
· Use a program such as sudo to allow users to use mount. This is still bad security, but doesn't directly give super user privileges to everyone. [3]
· Make the users use mtools, a package for manipulating MS-DOS filesystems, without mounting them. This works well if MS-DOS floppies are all that is needed, but is rather awkward otherwise.
· List the floppy devices and their allowable mount points together with the suitable options in /etc/fstab.
The last alternative can be implemented by adding a line like the following to the /etc/fstab file:
/dev/fd0 /floppy msdos user,noauto 0 0
The columns are: device file to mount, directory to mount on, filesystem type, options, backup frequency (used by dump), and fsck pass number (to specify the order in which filesystems should be checked upon boot; 0 means no check).
The noauto option stops this mount to be done automatically when the system is started (i.e., it stops mount -a from mounting it). The user option allows any user to mount the filesystem, and, because of security reasons, disallows execution of programs (normal or setuid) and interpretation of device files from the mounted filesystem. After this, any user can mount a floppy with an msdos filesystem with the following command:
$ mount /floppy
$
The floppy can (and needs to, of course) be unmounted with the corresponding umount command.
If you want to provide access to several types of floppies, you need to give several mount points. The settings can be different for each mount point. For example, to give access to both MS-DOS and ext2 floppies, you could have the following to lines in /etc/fstab:
/dev/fd0 /dosfloppy msdos user,noauto 0 0
/dev/fd0 /ext2floppy ext2 user,noauto 0 0
For MS-DOS filesystems (not just floppies), you probably want to restrict access to it by using the uid, gid, and umask filesystem options, described in detail on the mount manual page. If you aren't careful, mounting an MS-DOS filesystem gives everyone at least read access to the files in it, which is not a good idea.
Checking filesystem integrity with fsck
Filesystems are complex creatures, and as such, they tend to be somewhat error-prone. A filesystem's correctness and validity can be checked using the fsck command. It can be instructed to repair any minor problems it finds, and to alert the user if there any unrepairable problems. Fortunately, the code to implement filesystems is debugged quite effectively, so there are seldom any problems at all, and they are usually caused by power failures, failing hardware, or operator errors; for example, by not shutting down the system properly.
Most systems are setup to run fsck automatically at boot time, so that any errors are detected (and hopefully corrected) before the system is used. Use of a corrupted filesystem tends to make things worse: if the data structures are messed up, using the filesystem will probably mess them up even more, resulting in more data loss. However, fsck can take a while to run on big filesystems, and since errors almost never occur if the system has been shut down properly, a couple of tricks are used to avoid doing the checks in such cases. The first is that if the file /etc/fastboot exists, no checks are made. The second is that the ext2 filesystem has a special marker in its superblock that tells whether the filesystem was unmounted properly after the previous mount. This allows e2fsck (the version of fsck for the ext2 filesystem) to avoid checking the filesystem if the flag indicates that the unmount was done (the assumption being that a proper unmount indicates no problems). Whether the /etc/fastboot trick works on your system depends on your startup scripts, but the ext2 trick works every time you use e2fsck. It has to be explicitly bypassed with an option to e2fsck to be avoided. (See the e2fsck man page for details on how.)
The automatic checking only works for the filesystems that are mounted automatically at boot time. Use fsck manually to check other filesystems, e.g., floppies.
If fsck finds unrepairable problems, you need either in-depth knowlege of how filesystems work in general, and the type of the corrupt filesystem in particular, or good backups. The latter is easy (although sometimes tedious) to arrange, the former can sometimes be arranged via a friend, the Linux newsgroups and mailing lists, or some other source of support, if you don't have the know-how yourself. I'd like to tell you more about it, but my lack of education and experience in this regard hinders me. The debugfs program by Theodore T'so should be useful.
fsck must only be run on unmounted filesystems, never on mounted filesystems (with the exception of the read-only root during startup). This is because it accesses the raw disk, and can therefore modify the filesystem without the operating system realizing it. There will be trouble, if the operating system is confused.
Checking for disk errors with badblocks
It can be a good idea to periodically check for bad blocks. This is done with the badblocks command. It outputs a list of the numbers of all bad blocks it can find. This list can be fed to fsck to be recorded in the filesystem data structures so that the operating system won't try to use the bad blocks for storing data. The following example will show how this could be done.
$ badblocks /dev/fd0H1440 1440 > bad-blocks
$ fsck -t ext2 -l bad-blocks /dev/fd0H1440
Parallelizing fsck version 0.5a (5-Apr-94)
e2fsck 0.5a, 5-Apr-94 for EXT2 FS 0.5, 94/03/10
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Check reference counts.
Pass 5: Checking group summary information.
/dev/fd0H1440: ***** FILE SYSTEM WAS MODIFIED *****
/dev/fd0H1440: 11/360 files, 63/1440 blocks
$
If badblocks reports a block that was already used, e2fsck will try to move the block to another place. If the block was really bad, not just marginal, the contents of the file may be corrupted.
Fighting fragmentation
When a file is written to disk, it can't always be written in consecutive blocks. A file that is not stored in consecutive blocks is fragmented. It takes longer to read a fragmented file, since the disk's read-write head will have to move more. It is desireable to avoid fragmentation, although it is less of a problem in a system with a good buffer cache with read-ahead.
The ext2 filesystem attempts to keep fragmentation at a minimum, by keeping all blocks in a file close together, even if they can't be stored in consecutive sectors. Ext2 effectively always allocates the free block that is nearest to other blocks in a file. For ext2, it is therefore seldom necessary to worry about fragmentation. There is a program for defragmenting an ext2 filesystem, see XXX (ext2-defrag) in the bibliography.
There are many MS-DOS defragmentation programs that move blocks around in the filesystem to remove fragmentation. For other filesystems, defragmentation must be done by backing up the filesystem, re-creating it, and restoring the files from backups. Backing up a filesystem before defragmening is a good idea for all filesystems, since many things can go wrong during the defragmentation.
Other tools for all filesystems
Some other tools are also useful for managing filesystems. df shows the free disk space on one or more filesystems; du shows how much disk space a directory and all its files contain. These can be used to hunt down disk space wasters.
sync forces all unwritten blocks in the buffer cache (see the section called The buffer cache in Chapter 5) to be written to disk. It is seldom necessary to do this by hand; the daemon process update does this automatically. It can be useful in catastrophies, for example if update or its helper process bdflush dies, or if you must turn off power now and can't wait for update to run.
Other tools for the ext2 filesystem
In addition to the filesystem creator (mke2fs) and checker (e2fsck) accessible directly or via the filesystem type independent front ends, the ext2 filesystem has some additional tools that can be useful.
tune2fs adjusts filesystem parameters. Some of the more interesting parameters are:
· A maximal mount count. e2fsck enforces a check when filesystem has been mounted too many times, even if the clean flag is set. For a system that is used for developing or testing the system, it might be a good idea to reduce this limit.
· A maximal time between checks. e2fsck can also enforce a maximal time between two checks, even if the clean flag is set, and the filesystem hasn't been mounted very often. This can be disabled, however.
· Number of blocks reserved for root. Ext2 reserves some blocks for root so that if the filesystem fills up, it is still possible to do system administration without having to delete anything. The reserved amount is by default 5 percent, which on most disks isn't enough to be wasteful. However, for floppies there is no point in reserving any blocks.
See the tune2fs manual page for more information.
dumpe2fs shows information about an ext2 filesystem, mostly from the superblock. Figure 4-5 shows a sample output. Some of the information in the output is technical and requires understanding of how the filesystem works (see appendix XXX ext2fspaper), but much of it is readily understandable even for layadmins.
Figure 4-5. Sample output from dumpe2fs
dumpe2fs 0.5b, 11-Mar-95 for EXT2 FS 0.5a, 94/10/23
Filesystem magic number: 0xEF53
Filesystem state: clean
Errors behavior: Continue
Inode count: 360
Block count: 1440
Reserved block count: 72
Free blocks: 1133
Free inodes: 326
First block: 1
Block size: 1024
Fragment size: 1024
Blocks per group: 8192
Fragments per group: 8192
Inodes per group: 360
Last mount time: Tue Aug 8 01:52:52 1995
Last write time: Tue Aug 8 01:53:28 1995
Mount count: 3
Maximum mount count: 20
Last checked: Tue Aug 8 01:06:31 1995
Check interval: 0
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
Group 0:
Block bitmap at 3, Inode bitmap at 4, Inode table at 5
1133 free blocks, 326 free inodes, 2 directories
Free blocks: 307-1439
Free inodes: 35-360
debugfs is a filesystem debugger. It allows direct access to the filesystem data structures stored on disk and can thus be used to repair a disk that is so broken that fsck can't fix it automatically. It has also been known to be used to recover deleted files. However, debugfs very much requires that you understand what you're doing; a failure to understand can destroy all your data.
dump and restore can be used to back up an ext2 filesystem. They are ext2 specific versions of the traditional UNIX backup tools. See Chapter 10 for more information on backups.
Notes
For more information, see the kernel source or the Kernel Hackers' Guide. | |
It should of course be unmount, but the n mysteriously disappeared in the 70's, and hasn't been seen since. Please return it to Bell Labs, NJ, if you find it. | |
It requires several seconds of hard thinking on the users' behalf. |
Allocating disk space
Partitioning schemes
It is not easy to partition a disk in the best possible way. Worse, there is no universally correct way to do it; there are too many factors involved.
The traditional way is to have a (relatively) small root filesystem, which contains /bin, /etc, /dev, /lib, /tmp, and other stuff that is needed to get the system up and running. This way, the root filesystem (in its own partition or on its own disk) is all that is needed to bring up the system. The reasoning is that if the root filesystem is small and is not heavily used, it is less likely to become corrupt when the system crashes, and you will therefore find it easier to fix any problems caused by the crash. Then you create separate partitions or use separate disks for the directory tree below /usr, the users' home directories (often under /home), and the swap space. Separating the home directories (with the users' files) in their own partition makes backups easier, since it is usually not necessary to backup programs (which reside below /usr). In a networked environment it is also possible to share /usr among several machines (e.g., by using NFS), thereby reducing the total disk space required by several tens or hundreds of megabytes times the number of machines.
The problem with having many partitions is that it splits the total amount of free disk space into many small pieces. Nowadays, when disks and (hopefully) operating systems are more reliable, many people prefer to have just one partition that holds all their files. On the other hand, it can be less painful to back up (and restore) a small partition.
For a small hard disk (assuming you don't do kernel development), the best way to go is probably to have just one partition. For large hard disks, it is probably better to have a few large partitions, just in case something does go wrong. (Note that `small' and `large' are used in a relative sense here; your needs for disk space decide what the threshold is.)
If you have several disks, you might wish to have the root filesystem (including /usr) on one, and the users' home directories on another.
It is a good idea to be prepared to experiment a bit with different partitioning schemes (over time, not just while first installing the system). This is a bit of work, since it essentially requires you to install the system from scratch several times, but it is the only way to be sure you do it right.
Space requirements
The Linux distribution you install will give some indication of how much disk space you need for various configurations. Programs installed separately may also do the same. This will help you plan your disk space usage, but you should prepare for the future and reserve some extra space for things you will notice later that you need.
The amount you need for user files depends on what your users wish to do. Most people seem to need as much space for their files as possible, but the amount they will live happily with varies a lot. Some people do only light text processing and will survive nicely with a few megabytes, others do heavy image processing and will need gigabytes.
By the way, when comparing file sizes given in kilobytes or megabytes and disk space given in megabytes, it can be important to know that the two units can be different. Some disk manufacturers like to pretend that a kilobyte is 1000 bytes and a megabyte is 1000 kilobytes, while all the rest of the computing world uses 1024 for both factors. Therefore, my 345 MB hard disk was really a 330 MB hard disk. [1]
Swap space allocation is discussed in the section called Allocating swap space in Chapter 5.
Examples of hard disk allocation
I used to have a 109 MB hard disk. Now I am using a 330 MB hard disk. I'll explain how and why I partitioned these disks.
The 109 MB disk I partitioned in a lot of ways, when my needs and the operating systems I used changed; I'll explain two typical scenarios. First, I used to run MS-DOS together with Linux. For that, I needed about 20 MB of hard disk, or just enough to have MS-DOS, a C compiler, an editor, a few other utilities, the program I was working on, and enough free disk space to not feel claustrophobic. For Linux, I had a 10 MB swap partition, and the rest, or 79 MB, was a single partition with all the files I had under Linux. I experimented with having separate root, /usr, and /home partitions, but there was never enough free disk space in one piece to do much interesting.
When I didn't need MS-DOS anymore, I repartitioned the disk so that I had a 12 MB swap partition, and again had the rest as a single filesystem.
The 330 MB disk is partitioned into several partitions, like this:
5 MB | root filesystem |
10 MB | swap partition |
180 MB | /usr filesystem |
120 MB | /home filesystem |
15 MB | scratch partition |
The scratch partition is for playing around with things that require their own partition, e.g., trying different Linux distributions, or comparing speeds of filesystems. When not needed for anything else, it is used as swap space (I like to have a lot of open windows).
Adding more disk space for Linux
Adding more disk space for Linux is easy, at least after the hardware has been properly installed (the hardware installation is outside the scope of this book). You format it if necessary, then create the partitions and filesystem as described above, and add the proper lines to /etc/fstab so that it is mounted automatically.
Tips for saving disk space
The best tip for saving disk space is to avoid installing unnecessary programs. Most Linux distributions have an option to install only part of the packages they contain, and by analyzing your needs you might notice that you don't need most of them. This will help save a lot of disk space, since many programs are quite large. Even if you do need a particular package or program, you might not need all of it. For example, some on-line documentation might be unnecessary, as might some of the Elisp files for GNU Emacs, some of the fonts for X11, or some of the libraries for programming.
If you cannot uninstall packages, you might look into compression. Compression programs such as gzip or zip will compress (and uncompress) individual files or groups of files. The gzexe system will compress and uncompress programs invisibly to the user (unused programs are compressed, then uncompressed as they are used). The experimental DouBle system will compress all files in a filesystem, invisibly to the programs that use them. (If you are familiar with products such as Stacker for MS-DOS, the principle is the same.)
Linuxconf
Linuxconf (Figure 1) comes with Mandrake Linux and Red Hat Linux, but is also available for most modern Linux distributions. You've probably encountered this tool before if you use one of these distributions, either as the whole package or in one of its modular components. Multiple interfaces for Linuxconf have been available for years, but now we're up to four: GUI, Web, command-line and ncurses.
Figure 1. The GNOME Version of Linuxconf
Linuxconf has actually been around for years, which means its bugs have had longer to shake out than the other distribution-neutral tools. You can download and find out more about this tool at www.solucorp.qc.ca/linuxconf. Be sure to read through what each portion of the Linuxconf package is used for. There is a base package with the non-GUI components, and then there are various GUI front-end pieces from a more general X Window System version to one specifically built for GNOME.
Whether you stick with the command-line version or add a GUI front end, you run the tool by typing linuxconf. From here you navigate text or point and click menus to access a wide variety of system settings, everything from basic networking details to GRUB configuration. Linuxconf also plays well with people who refuse to use the root account for anything but the most vital tasks. If you try to run it as root, the tool simply asks for the root password--if this fact makes you nervous, then you may want to consider not using this tool, but this practice is fairly standard in modern administration tools. When you consider that anyone could just try to su to the root account at whim, you start to see why it is so important to have a secure root password in place.
Webmin
Webmin (Figure 2) comes with, and was recently acquired by, Caldera Linux. This tool is not only available for most modern Linux distributions, it also runs on most major flavors of UNIX and is available in around twenty languages (though some modules are not available in all of the languages). As you might guess, Webmin is purely a web-based application and a heavily modular one at that.
Figure 2. Webmin in Red Hat Linux 7.2, under GNOME
There is a set of core modules that handle the usual system administration functionality, and then there are the third-party modules available for administering a variety of packages and services. To download and learn more about Webmin, point your web browser to www.webmin.com/webmin. Once again, this package is available in a number of formats specific to different distributions.
Where any user can install Linuxconf, Webmin must be installed by root. After that you can access this tool from any user account as long as you know the root password.
There are three separate rows of icons on this tool's front page. On the upper right, you have a pair of administrative links, one to log you out of the Webmin tool and another that allows you to fill out a feedback form that sends your comments back to the Webmin team. In the same top row on the upper left you can click on the word Webmin and go to the product home page. On the upper bar directly beneath those links, there are a series of menu icons, which are, from left to right:
· Webmin: takes you back to the main Webmin screen.
· System: a collection of configuration issues, such as user and group manipulation, disk quotas and cron jobs.
· Servers: configuration routines for a number of servers you may have installed on your system, such as Apache, WU-FTPD and sendmail.
· Hardware: configuration utilities for hardware issues such as RAID, printers and disk partitions.
· Cluster: a collection of cluster maintenance tools.
· Others: a set of tools that system administrators typically need, such as a command prompt, an alias manager and a file manager.
Finally, there is the Webmin tab, which has a series of Webmin management tools:
· Webmin Actions Log: if you've enabled Webmin logging, this function allows you to search through the logfiles for what you've utilized this tool to do to your system.
· Webmin Configuration: takes you to the amazing number of configuration options available for Webmin, everything from strengthening your Webmin authentication requirements to upgrading either the main package or individual modules.
Yet Another System Tool (YaST)
YaST (Figure 3) and its cousin YaST2 (Figure 4) come with SuSE Linux; these items are specific only to a single distribution.
Figure 3. SuSE's YaST
Figure 4. SuSE's YaST2 in KDE
Generally speaking, you'll want to use YaST2 when at all possible; YaST2 is the graphical version, and the older YaST is a great fallback if you aren't able to get into the GUI or have not installed a GUI. YaST2 is laid out in a standard file-manager format, with the menu of categories on the left and icons for the various configuration routines on the right, which change to correspond with your category choice. The categories are:
· Software: the selection of SuSE software-management utilities, such as the ability to update your system over the Internet or add and remove packages from the SuSE CD-ROM or DVD-ROM.
· Hardware: a selection of hardware configuration routines, including printers, sound cards and scanners.
· Network/Basics: the selection of configuration tools formodem and other connectivity devices, Ethernet cards and more.
· Network/Advanced: the section where you can configure many of your network services, such as sendmail, routing and NIS+.
· Security&Users: the selection of user and group-management tools, as well as a couple of useful security tools.
· System: the selection of overall system-configuration tools, including a boot script (rc-config) file editor, boot loader configuration editor and routines for changing the language, keyboard and so on.
· Misc: a collection of tools that couldn't be otherwise grouped into the other categories. Some of the items represented here involve alternate print setup tools, some for working with log files, and some for communicating with SuSE.
The Caldera Open Administration System (COAS)
COAS comes in a rudimentary form with Caldera OpenLinux but is available for most modern Linux distributions, and is open source and covered by the GPL. This tool comes in four different formats, so you can use the interface that's most comfortable: command-line, ncurses (command-line but menu driven), GUI and Web for remote use. This tool is still under development but will be included in its entirety in later versions of Caldera OpenLinux--though there are rumors that because Caldera now has a stake in Webmin, COAS may be on its way out. This concern is born out by the fact that the COAS web site doesn't seem to have been updated since September of 1999.
This tool is modular, meaning that rather than loading its full set of features into memory when you start it, only the portions you're using come into play. Adding and removing modules is typically transparent as you work with the tool unless you require an external module, one made by a third party or one that for some reason is not included with the core release.
Right now the system is in development, specifically using KDE's Qt library set, but rumor has it that a GTK (GNOME) version is in the works. You can find out more about this tool by visiting http://www.coas.org/, including the list of the tool's anticipated capabilities.
Dee-Ann LeBlanc is a Linux writer, trainer, course developer and consultant who never seems to be able to stick with doing just one thing.
Overview of ermissions: Linux File Permissions
All operations described below require that you know the root password and can su in as root.
Setting file permissions in Linux is an integral part of administrating any Linux machine. It is especially useful in servers which allow remote access to your Linux machine, such as FTP servers, SSH remote access machines, Samba file servers, and so on. By setting the file permissions for each folder, the system administrator can control who can read, execute, or write to any file on your system. While the commands in Linux that set the file permission are straightforward, they allow for a great deal of control and flexibility. That being said, changing permissions on system files incorrectly in Linux can have a dire effect on the stability and usability of your linux install, so it is recommended that you start slowly when exploring how to change file permissions
Linux File Permissions Overview
In Linux, permissions are kept for every file and folder on your hard drive. There are three different types of permissions: read (r), write (w), and execute (x). You can check the file permissions for all the files in your current working directory by typing: # ls -la into a terminal window. A sample output for a typing the previous command in the "/" directory is as follows (Please note that the ls command will only display the file permissions when the -l option is typed) :
-rw-r--r-- 1 root daemon 1852 Sep 17 16:25 .login_global
-rw-r--r-- 1 root daemon 408 Jul 29 08:30 .logout_global
drwxrwxrwx 2 root wheel 2048 Apr 2 2003 afs/
lrwxrwxrwx 1 root wheel 9 Sep 15 11:17 bin -> ./usr/bin/
The final column on this printout is the name of the actual file or directory. The first column of this printout contains the file permissions for each object in a column of 10 characters. Each of these characters has a specified meaning:
· The first character specifies whether the object is a file (-), a directory (d), or a link to another file or directory (l).
· The next group of three characters defines the read, write, and execute permissions (in that order) for the owner of the object.
· The next group of three characters describes the permissions for the group that controls the object.
· The final group describes the permissions for everyone else, usually called the global permissions.
The owner of the object is given in the third column and the group the object belongs to is in the fourth column. In this example, everyone can access, write to, and execute from the afs directory, but everyone except root can only read the .login_global file. It is important to remember that the user root always has the ability to change the permissions on any file and directory regardless of the permissions set on them.
Changing File Permissions
Changing the permissions, owner, and groups of these different files uses three different commands which are very simple to use: chmod, chown, chgrp.
The chmod Command
The command chmod changes the actual permissions of files using a simple system of three octal (0-7) digits. Each permission is given a numerical value: read is 4, write is 2, and execute is 1. To get any combination of these permission, simply add the numbers the numbers of the permission you would like to enable and the rest will automatically be disabled. For example, if to give a file read and write but not execute permissions, the octal value would be 6. Similarly, all permissions is 7, only read is 4, and so on. Three of these octal digits are then combined, one for the owner, group, and global permissions, to create the command in the form: chmod <3 digit octal number> <File to apply to>. This is summarized below in the following table containing some common combinations:
3 Digit Number | Corresponding File Permissions |
777 | -rwxrwxrwx |
766 | -rwx-rw-rw |
755 | -rwx-r-xr-x |
774 | -rwxrwxr-- |
740 | -rwx-r----- |
700 | -rwx------ |
400 | -r-------- |
So, in order to change the permissions on a file named test.txt in /tmp/test so that everyone could read, write, and execute the file, you would first change to that directory and then type:
ComputerName:~# chmod 777 test.txt
Checking the permission on this file using the ls -la command, we get:
-rwxrwxrwx 1 root root 31 Oct 28 10:07 test.txt
Now, if we wanted only the owner of the file (root) to be able to write to it while allowing everyone else only read rights and the group execute rights we would enter:
ComputerName:~# chmod 754 ./test.txt
The output of ls -la would then be:
-rwxr-xr-- 1 root root 31 Oct 28 10:07 test.txt
An important thing to remember when changing file permissions is any given file is affected by both its permissions and the permissions of the directory it is stored in. The method for changing a directory's permissions is the same, but when doing so, it is also possible to recursively apply the permissions defined for the directory to everything contained within it using the -R option. So for example, if we wanted to change both the /tmp/test and the test.txt permissions so that only the owner has all permissions and no one else has any permissions, we would write instead of two separate statements:
ComputerName:~# chmod -R 700 /tmp/test
This would give us the following output for ls -la (the '.' represents the parent directory /tmp/test):
drwx------ 2 root root 4096 Oct 28 10:07 .
-rwx------ 1 root root 31 Oct 28 10:07 test.txt
The chown Command
Since we can set permissions for not only the global users, but also for the local owner and group owners of the object, then it would be useful to be able to switch these two properties. This can be done by the chown and chgrp commands. Both commands have a similar format to the chmod and both also support the -R recursive command mentioned above, which is useful when changing the ownership or group of an entire directory. Starting with the chown command, the basic format is chown <User> <Object to take ownership of>. As an example, changing the ownership of the /tmp/test directory and all the files within it from root to user. The command would be, using the -R option:
ComputerName:~# chown -R user /tmp/test
The output of a ls -la command within /tmp/test would then be:
drwx------ 2 user root 4096 Oct 28 10:07 .
-rwx------ 1 user root 31 Oct 28 10:07 test.txt
The chgrp command has exactly the same syntax, except the specified user is replaced with the correct group. So, to change the ownership of the /tmp/test directory to the users group, the following command would be used:
ComputerName:~# chgrp -R users /tmp/test
Again, the output of the ls -la command is:
drwx------ 2 user users 4096 Oct 28 10:07 .
-rwx------ 1 user users 31 Oct 28 10:07 test.txt
It is important to note that chgrp is a specific command of chown and that chown can be used to perform both operations in a single command with the syntax:
chown <User> : <Group> <Object to take ownership of>
It is recommended to get accustomed to these two commands on a small test file or directory before attempting to apply them in more advanced applications as they can be quite harmful if not applied correctly.
Applications of File Permissions
Local User Administration
One of the most obvious and useful applications of file management is the administration of the local files. If the machine you are administrating will be used be a variety of users, it may be advantageous to plan ahead and set the file permissions both for places you want these user to go and not to go. This planning is much easier if the users are separated into groups with different levels of access to user data. One simple way to improve file security for local users is to create a public and private folder within each user's home directory. By correctly setting the permissions on these two directories, each user can have a folder that is publicly accessible for both reading and writing and a folder which no one else can even read except for the user themselves and the root administrator. A set of commands to set up this above system for the user "user" who is in group "users" is given below. Setting up directories to be accessible by groups of users is similar, just change the chmod value from 700 to 770.
ComputerName:~# cd /home/user
ComputerName:/home/user# mkdir ./private
ComputerName:/home/user# mkdir ./public
ComputerName:/home/user# chown -R user /home/user
ComputerName:/home/user# chgrp -R users /home/user
ComputerName:/home/user# chmod -R 777 ./public
ComputerName:/home/user# chmod -R 700 ./private
ComputerName:/home/user# ls -la
drwxr-xr-x 5 user users 4096 Oct 30 09:18 .
drwx------ 2 user users 4096 Oct 30 09:18 private
drwxrwxrwx 2 user users 4096 Oct 30 09:18 public
Samba Share-level Administration
Allowing people to access network shares on your computer through Samba is another application for file management. When a user connects to a share on your machine, the authentication of that user is handled by Samba, but the permissions of whether that user can read, write, or execute in that shared folder are handled by the normal Linux file permission commands. It also important to note that Samba uses the same user names and groups as are defined by the local Linux users, so make sure that any groups or users that you want set up are created before trying to create any network shares. Thus, having them set up correctly determines whether or not the user will be able to save his or her work in your share. There are two main examples: a private share accessible by only a single user and a group share accessible only by a particular group of private users. For a private share, you will want to make a directory similar to the private directory in the example above. This example will use the directory /tmp/testshare. The share in the smb.conf file should look something like this:
[test]
path = /tmp/testshare
valid users = user
public = no
writable = yes
printable = no
This share is defined as writable but only for the users in the "valid users=" list since public is set to no. To set this up this directory on the local system, we simply need to make sure it is owned by the user "user" and under control of the correct group, "users" in this case, and set its permissions to allow only the owner to access it. The commands below implement these properties:
ComputerName:~# mkdir /tmp/testshare
ComputerName:~# chown -R user /tmp/testshare
ComputerName:~# chgrp -R users /tmp/testshare
ComputerName:~# chmod -R 700 /tmp/testshare
ComputerName:~# ls -la /tmp
drwx------ 2 user users 4096 Oct 30 10:10 testshare
It is important to note that any new files created by a user will be owned by them and by default will be readable by all people in that users group and publicly, but since this share is only accessible by the user "user", this is not a problem. This does become more of a problem in the following example of a group share. In this example, the directory /tmp/testshare is again used to now illustrate a directory to be accessible by anyone in a given group. The directory itself should be owned by a user in the group, "user" in this example, and should by put under the group in question with the chgrp command, "users" here. The share as defined in the smb.conf file should look something like this:
[test]
path = /tmp/testshare
valid users = @users
public = no
writable = yes
printable = no
force create mode = 0770
force directory mode = 0770
force group = users
Here the "@users" option in the valid users line defines the group "users". In addition, the last three lines setup the default permissions for any new files or directories that will be written to this share by the users in group "users". The "force create mode" and "force directory mode" specify the permissions to be applied to any newly created files by using the octal permissions discussed above plus another so-called "sticky-bit" in the front that we will not worry about here, so it should always be set to 0. The "force group" command forces all created objects to be set under the "users" group. There is also a corresponding "force user" command which be used if you wish to have all the files to be owned by a single user. To set up the current files and directory permissions locally, we simply have to change the chmod command in the example for the private share from 700 to 770 and the share is ready to go. There are many other examples of shares and how to set their permissions in the example smb.conf file that comes with the Samba package. For further help, the man page for smb.conf is a great place to start otherwise the Samba homepage http://www.samba.org/ also has excellent documentation on how to set up the smb.conf file and other aspects of Samba.
Processes:
Process status: 1. What does 'ps'mean?
ps is the shortage for Process Status. The command should be used to display the currently running processes on Unix/Linux systems. If you know the 'Task-Manager' which pops up under Windows NT/2000/XP when you press CTRL+ALT+DEL then you have a clue what ps does under Unix/Linux. Ps can show you the running processes on your system in different ways. I will describe the basic ones you should know.
2. Why is it good to know how the ps command works?
If you have a process which seems to hang (e.g. netscape navigator on some buggy websites) and you want to stop the process, then you can determine the process id of the process. Why do you need the process id? You can stop the process with the help of the 'kill' command. The kill command needs a process number otherwise it won't know to which process it should send the 'kill' signal. Other example. You have started a process and now your machines becomes slower and slower. You stop the process but your machine still gets slower and slower. Now it will be helpful if you can stop the process. In this case you also need its pid.
Next example: You want to know what processes your friend just uses (you know he has an remote session open), then you can also use the ps command to find out which processes belong to him.
Killing processes:
Process priority:
Processes
Did you ever had some program 'hanging' on you? Indeed 'hanging' so badly that it gobbled up all system resources, slowing down all operations to a crawl? And didn't you wish there were just a way to kill it? Were you ever interested in knowing how much system resources a program uses?
Well, you have come to right place, then :-). What are processes? Basically everything going on on your system: your web browser is a process, your web server - if you happen to run one - is also a process. A process boots your machine and another shuts it down. Every tiny script starts a process, albeit usually a very short one. "A process" says the textbook, "is an instance of a program in execution."
Processes can be started, stopped, killed, given priorities, scheduled. You do many of these actions already without knowing about it: when you open a program, you start a process, by closing the program, you kill the process. It's not some fancy Linux thing either, every modern operation system has them, because processes allow for multi-tasking: by keeping a unified list of what is going on, the operating system can run a schedule, thus providing its user with the illusion of being able to do many things at once.
Usually the operating system,i.e. the kernel, manages processes for you, you only need to interfere if you have special preferences (e.g. want to have a process to get more resources) or if a process goes awry and starts eating all the available system resources.
Process Monitors
The traditional Unix/Linux tools for viewing and handling processes are the console programs 'top' and 'ps' (part of the 'procps' RPM) as well as a bunch of command line programs like 'kill' or 'nice'.
There are graphical interfaces for these programs, however, which you might find more convenient and easier to use: 'ksysguard' (package 'kdeadmin', menu: Applications - Monitoring - KDE System Guard), 'kpm' (package 'kdeutils', menu: Applications - Monitoring - Process Management) and 'gtop' (package 'gtop', menu: Applications - Monitoring - GTop). 'ksysguard' even allows you to watch and control processes on other machines.
Since all these tools use the same information from the '/proc' directory (more on that later), they all display processes and their attributes in similar fashion.
If you open them, you'll find a dynamic (apart from 'ps') column display with a row for each process. Columns are:
· PID: This is the Process IDentification number. To the system, processes are not known by their name, but by a unique number. Each new process gets a higher PID. The more processes you run and / or the longer your machine runs, the higher those numbers get. That's normal.
· User (also Login): The user who 'owns', i.e. started this process. Users can only manipulate processes they own. Only 'root' can manipulate all processes.
· Pri (also Nice or NI): The priority of a process in relation to all other processes. Lower numbers indicate higher priority. Standard priority is '0'.
· Size (also VSZ or VmSize): The total amount of virtual system memory a process uses. As explained later on, that's by no means a fair indicator of how much memory the process actually uses. Compare next entry.
· Resident (also VmRSS or RSS): The amount of physical system memory a process uses up. This is more accurate for measuring how much system memory a process needs.
· Stat (missing in 'ksysguard'): The status of a process. Sane processes are either 'R'unning or 'S'leeping. It is normal that only one process is displayed as 'running' at any given time while the others are 'sleeping' (i.e. waiting for a signal).
· CPU (subdivided in User% and System% by 'ksysguard'): Percentage of the processor time taken up by the process since the last display refresh.
· MEM: Percentage of physical system memory consumed by the process.
· Time (missing in 'ksysguard'): Total amount of processor time the process has used since started.
· Command (also Cmd): Command name of the process.
Manipulating processes via the graphical utilities usually just involves 'highlighting' the process you want to handle and using the right mouse button click context menu. 'top' is controlled via shortcut keys, read man top for a list.
Notice that there's a standard key combination for killing graphical applications, <ALT> <CTRL> <ESC>. This will change the mouse pointer to a 'death' symbol. Click on an application window and that application will be killed. You still want to check if the processes of that application had been killed, too, though.
Processes In Detail
The father of all processes is 'init' (initializer). The pstree command shows this quite impressively.
'init' manages the runlevels, it controls which processes are started and stopped at which runlevels and it is responsible for bringing the system to these runlevels. If you shutdown your system, you essentially tell 'init' to bring it to runlevel 0. It's the same if you type shutdown -h now or init 0.
Information on current processes is stored in the '/proc' file system. The '/proc' file system consists of kernel data changed in real-time. It is therefore not a 'real' file system like the others: the data contained take up no place on the hard-disk and you can't write to it. But you can extract and change the information contained therein (e.g. using the 'cat' command).
Listing '/proc', you'll find a lot of directories which names only consist of numbers. These numbers are the 'process IDs' (PIDs). If you list such a directory, you'll find files like 'cmdline', which contains the command line the process was started with, or 'stat', which contains number codes on the current process.
For example:
# cat /proc/1/cmdline
init
Which means: 'The process with the PID 1 was started with the command 'init''. 'init' will always be the first process and thus is the only process with a permanent PID. All other processes will have increasing PIDs, the maximum PID being 32768 (although the maximum number of processes on Linux is only limited by the amount of physical memory available).
The children of 'init' can have child processes themselves. If you start a program by typing its name on a command line, the program will be a child of the program which handles the command line. In most cases, this will be 'bash' (the 'Bourne-Again-SHell').
One important thing to know is that child processes 'inherit' some assets of their fathers. If you run a process with 'root' privileges, a child of this process will also have these privileges.
Now you might ask: 'Since init runs with 'root' privileges, and it is the father of all processes, does that mean that all processes and programs on Linux run with root privileges?'
It's obvious they don't do that. Servers like the font server, web servers, mail servers run on their own user account. The font server for example runs on the 'xfs' account, the Apache web server on the 'apache' user account. Servers which don't come with their own user account use the standard 'nobody' user account. These are not 'real' accounts: nobody can login via these so-called system accounts.
The reason for this is security: servers can be accessed from outside the machine. If one of them contains a software flaw which allows attackers to execute commands on your machine as the user who is running the server, it would be a disaster if that user would be 'root'. Notice that this isn't a system mechanism, but a standard feature of Linux servers. You should refrain at all cost from using servers which do not include it.
Another mechanism is the 'login' command. Have a look at the output of 'pstree' again. You will see that every process and program you started after boot is a child of the 'login' process. 'login' switches the permissions of these child processes depending on how you log into the system. If you log in as 'root', all children will have 'root' privileges, if you log in as a user, the children will run with the permissions (the 'UserID', UID) of this user. The latter is preferable for the system's security and stability. If you need 'root' privileges, you can always log into the 'root' account from your user account by using 'su' or 'sudo'.
setUID
As you've already seen, processes can run with a UserID different from the user who started them. Servers are started by 'root' but drop this privilege as soon as possible and run on their own system account. In Unix-speak, one says they are running 'setuid account' (e.g. Apache runs 'setuid apache').
Of course, you can also do that the other way round by providing processes started by a user root privileges.
The most prominent example is the graphical subsystem, 'X'. 'X' needs 'root' privileges to run. However, the children of X do not run as 'root'. How come?
'X' is started via a so-called wrapper, '/usr/X11/bin/Xwrapper'. If you look at the permission bits of this program, you will see it's 'setuid root': -rws--x--x. This wrapper handles the permissions part when starting X, so that X itself doesn't need to be 'setuid root'. It runs as 'root' but its children don't inherit its UID.
Priority
Processes run either in user mode or in kernel mode. The operating system and its core services run in the privileged kernel mode, whereas 'normal' applications run in user mode, this way badly behaving user processes do not interfere with system operation. Some user processes however need access to privileged kernel functions. For this they switch to kernel mode, execute the needed function and switch back to user mode.
All user processes get the same share on the processor schedule. But you can change that with the nice -n value process command. Negative values increase the priority. They can only be applied by 'root', whereas positive values can be applied by any user.
nice -n value process starts a process with a given priority, the command renice priority PID changes the priority of a running process.
Processes And Threads
A program may open several processes if needed. This however isn't very efficient: each process would need its own address (memory) space and switching between these would cost a considerable amount of system resources. This is where 'threads' come in. Threads are started by a process, but they remain within the original address space used by the process. Switching between threads consumes much less resources than between processes. Applications which use this technique are called 'multi-threaded'.
Programming thread-enabled applications cleanly is very hard which justifies the use of threads only in very complex pieces of software, like the Mozilla browser.
As a system administrator, you only have to know about them that process monitors on Linux can not discriminate a thread from a process. Point in case is the Mozilla browser. If you start it, the process monitors will show multiple entries for it:
tom 11290 13.4 2.3 33080 21124 ? S 11:47 0:02 /usr/lib/mozilla/
tom 11296 0.0 2.3 33080 21124 ? S 11:47 0:00 /usr/lib/mozilla/
tom 11297 0.0 2.3 33080 21124 ? S 11:47 0:00 /usr/lib/mozilla/
tom 11298 0.0 2.3 33080 21124 ? S 11:47 0:00 /usr/lib/mozilla/
tom 11299 0.7 2.3 33080 21124 ? S 11:47 0:00 /usr/lib/mozilla/
This does not mean that the Mozilla browser has started five processes, but that's one process with four threads, all in the same address space. How can one tell? Well, you don't get any real proof, since the monitor doesn't know either, but there are hints like the same memory consumption numbers, the same command name and the same start time. Only the first entry is valid when looking at system resource usage.
For more on Linux process management, read the Linux Gazette article Processes on Linux and Windows NT by Glen Flower.
Process Accounting
Process accounting enables you to keep detailed accounting information for the system resources used, their allocation among users, and system monitoring.
(Enabling Process Accounting on Linux HOWTO)
You will need to install the 'psacct' package from the first Mandrake Linux CD which will in turn install a system service of the same name. Documentation (and an amusing bit of Unix lore) can then be found in info accounting.
Both 'Webmin' and 'Linuxconf' offer graphical modules to configure and monitor process accounting.
Estimating Process Resource Usage
Estimating the resource usage, especially the memory consumption of processes is by far more complicated than it looks like at a first glance.
How much resources a process needs, depends on many factors, most of them varying. One is the amount of physical RAM in the system. If there is much free RAM available, more caching will be performed and thus more memory consumed. However this doesn't really count as resource usage, since this cached memory is available in case some other process needs it. Up to 20% of system memory can be used for buffering and caching. This dramatically improves disk accesses, so you don't 'loose' anything when memory is assigned to this task. Only unused memory is wasted ;-).
Graphical applications depend on certain collections of code libraries for displaying their interface (buttons, menus etc). Common collections are GTK+ (GIMP, GNOME), Qt (KDE), Tcl/Tk and others. If you use applications of many different collections, all these libraries have to be loaded into system memory, thus using more resources. An application written with Qt for KDE will use more system resources in GNOME than on KDE. If you start a second Qt application, however, it will use about as much resources as on KDE, since the first application already did load the needed interface libraries. If you close both applications, not all resources will be freed up again, instead the loaded libraries will be cached (if the amount of free RAM permits it).
The numbers for each process presented by process monitors tend to mislead and are easily misinterpreted. On the previous page you've already read about threads. But even if the process doesn't spawn threads, it is in most cases almost impossible to determine how much system memory an application really consumes.
Let me give a simple example:
1. top: Mem: 900040K av, 411864K used, 488176K free, 1048K shrd, 58592K buff, 166816K cac
2. Starting the 'bluefish' HTML editor.
3. top entry for bluefish: 4716 (size) 4716 (real) 3580 (shared).
top entry for system: Mem: 900040K av, 414268K used, 485772K free, 1432K shrd, 58620K buff, 167592K cac
According to the process table entry, 'bluefish' uses 4716 KB of real memory and 3580 KB of shared memory. The system however only reports an increase of 2404 KB general memory usage, of these being 400 KB more in shared memory, 30 KB more in buffers and 600 KB more in cache.
Regardless of how you add or subtract these numbers, you will never get the numbers from the process entry and the system numbers in sync.
I rather trust the system numbers, but even they can be misinterpreted quite easily:
42 processes: 41 sleeping, 1 running, 0 zombie, 0 stopped
CPU states: 0.9% user, 1.3% system, 0.0% nice, 97.6% idle
Mem: 257676K av, 252940K used, 4736K free, 202852K shrd, 7464K buff
Swap: 130748K av, 256K used, 130492K free, 197620K cached
What usually confuses people here is the cache and buffer management of Linux. Let's have a look at it:
Mem: 257676K av, 252940K used, 4736K free, 202852K shrd, 7464K buff
Swap: 130748K av, 256K used, 130492K free 197620K cached
'Mem' shows some 252 MB RAM installed. In fact it's 256 MB. What happened to the remaining 4 MB? Simple: they are used for 'shadowing' the system's BIOS and contain the GNU/Linux kernel.
The next fields seem to indicate that the system is on the brink of collapse: almost all of the memory seems to be used up. Is that so? No, it isn't. Look at the last entries of those lines. These tell you that about 200 MB of system memory are used for caching and buffering. This memory is available for every application which needs it.
The 'free' command line tool gives a more comprehensible overview:
$ free
total used free [...]
Mem: 257676 253624 4052 [...]
-/+ buffers/cache: 50360 207316
Swap: 130748 256 130492
Have a look at the third line:
-/+ buffers/cache: 50360 207316
displaying again the actual amount of memory in use by applications (50 MB) and free (202 MB). The rest is used for caching and buffering, i.e. to make your system faster.
To estimate the resource consumption of a process, it's easiest to run 'free' before, during and after running the process. The figures will vary because of the already mentioned reasons, so you should do this several times to get an average.
Setting Resource Limits
PAM (Pluggable Authentication Modules, man 8 pam) offers a mechanism to set limits to resource usage, setting limits via shell mechanisms ('ulimit' in bash, 'limit' in csh etc) is deprecated.
You configure these limits as 'root' in '/etc/security/limits.conf'. Among other things you can set the maximum number of processes per user or group, process priorities, maximum CPU time and more.
Parameters and examples are supplied in the configuration file.
More information can be found in the appropriate chapter of the L-PAMSAG.
Dealing With Bad Processes
If a process behaves badly, the kernel sends it a certain signal (man 7 signal) to terminate it.
The most infamous signal is SIGNAL 11 (alias SIGSEGV, alias 'segmentation fault'). The process has tried to access a memory segment which hasn't been allocated to it.
Reasons for a segfault might be a programming error, a wrong library version or a hardware failure. The default action of a process when receiving a Signal 11, is to terminate itself and write the contents of the system memory to a 'core' file (in Unix slang, the process 'dumps the core'). This core file can be useful to a programmer for debugging. Notice that Mandrake Linux by default disables the creation of core files.
Things get a bit hairy however when the process is that bad that it ignores this signal and starts to hoard all the available processor time on the system, thus slowing down ('starving out') all the other processes. The kernel marks this process as 'uninterruptible' and its status in the process table will be switched from 'R' or 'S' to 'D'. If you see a process entry with a 'D' in its STATUS row, it's definitely time for you to do something.
Almost all signals can be handled by processes which means that they can also be ignored by them. There is however one signal which can't be ignored by any process and that's SIGNAL 9 (SIGKILL), the appropriately named 'kill signal'. If you send that signal to a process, you will quite literally kill it by taking away all its resources and removing it from the process table. You can then safely go on in your daily work. All unsaved data of that process will be lost, too, though.
Most system monitors offer you a possibility to kill a process via the right mouse click context menu (in console 'top', press the <k> key). There is one problem though: processes run with the permissions of the user who started them. A user can only kill processes he himself has started, only 'root' can kill all processes. Your system monitoring program will and should usually run under your user account which means you can't kill processes from other users or 'root' via it.
If a process started by root or another user misbehaves, you have to switch to the 'root' account on a console using the su command. You can then either kill the process by its process ID (PID) or by its name.
If you don't know the PID of the process yet, you can find out with
# ps aux --sort %cpu
which lists the most CPU hungry process last, the PID being the leftmost number. You can then go on to actually kill the process with the 'kill' command:
# kill -s 9 PID
You can also kill a process by name. Notice that this can be dangerous, since you either might accidentally kill processes you didn't intend to kill (or none at all). The command is
# killall -s 9 name
If you ever happen to come across a non-Linux Unix system, make sure to read the man page for 'killall' on that system. Some Unixes take this command literally which can get very ugly ...
A rather harmless type of a bad process is the so-called 'Zombie' (STATUS: Z, 'defunct'). If you've read the previous page, you know that processes can have 'children', i.e. spawn new processes. If the parent process is terminated normally, it will send a termination signal to all its children signals, too. If however the parent is terminated abnormally, it might not have gotten around to send its children the termination signal. Without their parent process these processes turn into 'living dead': they still appear in the process table, but they don't use any resources and there is no way of 'reaching' these processes. Unlike their namesakes, process zombies don't do any harm. They will vanish from the process table upon the next reboot.
No comments:
Post a Comment