Linux Systems Administrator Tasks

To be a Systems Administrator one has to be able to work on just about anything that plugs into just about anything. Over time you will learn how to do all that and, if you're really lucky, you may even get it right some of the time. Most of the skills you'll need you'll acquire in a rush of panic and fear, but at least you won't soon forget the experience.

The basic knowledge you'll need are those that look good on your resume:

• Server protocols and configuration
• Server hardware and peripherals, installation and configuration
• Operating Systems installation and configuration
• Network protocols and devices
• Network cabling
• Hardware and software troubleshooting
• One or more programming or scripting languages

You'll also acquire some other, less technical skills:

• Patience with idiots
• The ability to explain problems to non-techies
• The ability to produce coherent documentation
• The certainty that Windows isn't worth the trouble
• The certainty that Linux (or UNIX) is the only way to go
• Patience with idiots

The other consideration is that I prefer Linux or UNIX and I'm the one writing all this. I especially like the capability to create on-the-fly scripts to manage most of the routine tasks. Linux provides several scripting shells but bash is the default. Once you master shell scripts, you'll find all manner of ways to minimize the more tedious Sysadmin tasks. For more detail see this. For a fairly brief overview of Linux, click Here

The files and directories you use will change depend on which distribution of Linux you've installed. To keep it simple (for me) I'm going to assume Redhat or one of its derivatives. Redhat is a kind of System V type system rather the BSDish offshoots of other distros and it's pretty much a universal standard.

Most of the system level configuration scripts are located in the /etc/ directory. This will be the system passwd file, shadow (for encrypted passwords), the init scripts for booting (inittab is still used so check it out) the directories for starting the daemons (the rc*.d directories, init.d,, xinet.d, mail, etc.) and the rc.local file for adding your own stuff to the boot process. The daemons themselves will be scattered around in various places but you can usually find them by looking in their startup scripts.

If you compile add-on programs from source, the default install directory will usually be /usr/local with various executables in /usr/local/bin. Other distros will prefer /opt. The /sbin directory is where the system level commands live, things like chkconfig, ifconfig and other scripts and executables that the Sysadmin needs but aren't generally necessary for regular users. There is also a /bin directory where most of the general purpose, user level commands are located.

Removable device like CDs, DVDs, floppy drives, usb devices, etc will usually have a directory in the /media directory (possibly /mnt). You will usually use one of these to mount a removable device although you could mount it to any empty directory.

The /usr directory is a kind of catchall that has all manner of mostly user level stuff. It has the on-line manual, /usr/bin and /usr/sbin for scripts and executables, /usr/local for new programs and packages, libraries and kernel modules and the kernel itself. It's usually partitioned separately from the root partition.

the /lib directory has libraries and modules and other bits and pieces that other programs depend on.

The /home directory is where the system will put all new users by default. It does this by consulting the /etc/default files and copying the stuff in /etc/skel to the new user's home directory. If you want to change these initial settings, do it here. This directory should also be mounted separately.

The /var directory is where most of the log files are stored, the mail files and mail spool files, the web site, the crontabs and lots of other things that change often. This directory also needs to a separate partition.

The /boot directory is where the kernel and other components needed for booting are kept. The /boot/grub directory is where the grub.conf file is found. You can edit this file to change the order of which kernel boots, the default wait time before booting the default kernel and some other useful things.

The graphics configuration is in the /etc/Xll/xorg.conf file. If the display screws up, check this file and tinker with the settings until you get it right. Make sure the /var/gdm has the group of gdm. In normal operation a live, production server shouldn't really be running in graphics mode anyway (runlevel 5) so this shouldn't be an issue.

The best time to partition the hard disk(s) is during a fresh install. Always select the manual partition option. While it's a little more trouble, it will save you headaches later on. You should also read up on LVM (Logical Volume Management) and configure it as a kind of safety overflow in case a partition starts to fill up. The following partitioning scheme has worked well for me:

/boot	This is where the boot files are, including grub. Make this 100Mb
/root	This will be the basic Operating System, /etc, /bin, /sbin, /lib. This is what you need to get a functional boot. Make it about 2Gb minimum.
/usr	This is almost all the user level stuff including the graphical user interface and all the general purpose apps and utilities. Make this about 4Gb
/usr/local	Create this partition if you like to add lots of source packages. Make this about 4Gb.
/var	This needs to fairly large since the files here tend to grow large over time. 4Gb should be about right.
/home	Depending on your users, this should have plenty of room for growth. 4Gb should be adequate unless you have lots of busy users.
/download	If you have some space left, add partitions for downloaded files. You might also want a place to store archived files, backups for this machine and maybe others as well.

/boot/grub/grub.conf	Check the kernel being installed
/etc/inittab	See the default runlevel
/etc/init.d	Read the control scripts for the services you're running
/etc/fstab	See what file systems are mounted on which devices
/etc/xinit.d	Check which daemons are enabled
/etc/bashrc	If the shell is bash, set the environment here
/etc/profile	If the shell is sh or ksh, set the environment here
/etc/httpd/conf	This is the default location for Apache
/usr/local/apache2/conf	This is Apache if you compile from source
/etc/passwd	User account information
/etc/shadow	User encrypted password file
/etc/r.local	Where you put your add-ons to the boot process
/etc/rc*.d	Directories where the daemons get started
/etc/sysconfig	Options and setting for daemons
/etc/sysconfig/network-scripts	Where NIC parameters are set
/etc/resolv.conf	Set the nameservers here
/etc/hosts	add hosts here to avoid name lookups
/etc/named.conf	Namerserver configuration
/var/named	nameserver records
/etc/mail	The default location for sendmail files
/sbin/chkconfig	Command to see what services are enabled
/sbin/ifconfig	Show or change NIC settings
/sbin/route	Show or change routing configuration
/sbin/ip	show or change network settings
/usr/bin/man	Lookup commands in the on-line manual

If you want to make a change to anything always make a backup of the original first. Edit the original. Test it. If somethings goes wrong copy the saved original back to its old filename. If possible create a directory somewhere, copy the files you need there and work on those. Mistakes can be disastrous so always plan on a way to recover. Ideally you will use something like RCS to keep track of what you've done.

You also need to learn various backup and recovery techniques, just in case. The rsync utility is a handy way to backup important stuff but so is dump and restore. Even tar and cpio provide you ways to stash critical files and directories somewhere safe. However you do it, DO IT.