Mastering the shell

Why would you want to be an expert with the shell? Well, perhaps some developers or CS elitists would look down their nose upon shell scripting and consider that to be a “lowly” IT skill. To that I say bullhockey!

As a software developer, the shell is your interface to the OS. (At least in the *nix world). Why wouldn’t you want to be a shell expert?

When one begins to understand the scope of the shell, the breadth and depth of the power available, only then have you begun to master it.

Any idea how many different commands are available to you at the $ prompt? For starters just do this;

nikon:/u/makuch
$ echo $PATH | sed -e ‘s/:/ /g’ | xargs ls -1 | wc
5206 5196 53982

5,206 commands! (varies by system) I don’t know how many of those are accessible via the gui, but I’ll bet it’s a small fraction. And of course as soon as you introduce a single pipe into your command line you pretty much eliminate the gui as competition.

When you want to learn more about how to interact with your OS just start reading the man pages on the commands in your PATH. Or for starters just do

$ ls /usr/bin

and start reading man on those that look interesting…

A few reasons that a proficiency with shell programming comes in handy for a CS professional;

– data manipulation
– log file manipulation
– automated build scripts
– automated test scripts (sure you have a great QA department who does this stuff but you need to do some automated testing of your own. Or, you don’t have QA department capable of producing automated test scripts!)
– backup scripts – say you don’t like how your IT department is (or is not!) backing up your stuff – do it yourself!
– at a small (startup?) company there may only BE developers i.e. no IT people yet.
– need to insert a corporate copywrite comment at the top of all of your 50k source files (this actually happened to me)
– need to build some script wrappers around your archaic SCCS (source code control system) system commands (again, this happened to me).
– maybe you need to search a bunch of text and perform a find-and-replace in multiple files.
– perhaps you want to keep a tarball of your home directory as it existed yesterday.
– shell programming lets you slice and dice tasks that would be painfully slow if you were to use the GUI.
– I needed an alarm the other day. ” $ sleep 20m && madplay PinkFloyd-Time.mp3″
– when you know shell programming, you find uses for it that simplify your life. The example before was simple, more fun than useful, but it can save you tons of time in filesystem operations and a miriad other things.
– System Administration. Sure, you’re not an IT guy. But you have hosts of your own in your own server closet don’t you!
– Just google “why should I learn the shell”

I tend to keep anything ever written even if only needed once. I have a total of 579 scripts in my ~/scripts dir all written by me over 20 years or so.

Mastering the shell is about productivity. There’s just a ton of things you can easily do having the tool set at your disposal that you wouldn’t otherwise even consider being able to do.

I like this quote on the subject;

Linux supports the notion of a command line or a shell for the same reason that only children read books with only pictures in them.

Language, be it English or something else, is the only tool flexible enough to accomplish a sufficiently broad range of tasks.

Bill Garrett

I have no idea who Bill Garrett is but I agree with him on this subject!

Think you’re an expert with regexp? Try groking the longest regexp ever;

http://stackoverflow.com/questions/2245282/what-is-the-longest-regular-expression-you-have-seen

How bout the 25 best awk commands;

http://blog.urfix.com/25-awk-commands-tricks/

Or top 25 sed commands;

http://blog.urfix.com/top-25-sed-commands/

I’ve a couple more blog posts about the shell here Most used shell commands and here Piping shell commands.