Shell Script Language – Use Perl, Not Bash

To me, a shell script is a script that automates repetitive tasks. But that is not the ‘official’ definition. Wikipedia has this definition

A shell script is a script written for the shell, or command line interpreter, of an operating system.

I use a definition that defines the purpose of the script – while the others prefer a definition that defines the technology used. I am not going to claim that my definition is better than the other definition – that’s pointless. Besides, even I think that the ‘other’ definition is the right one. But I will try to show you the advantages of my approach.

The Purpose

I like to automate things(in other words, I’m lazy). So I have a nice little collection of custom shell scripts. But there is a huge barrier to writing shell scripts – the language used. Traditionally shell scripts are written in a language provided by the shell – like bash or tcsh. That is the problem – these languages are Bad – with a capital ‘B’.

The Problem

To people who are accustomed to decent languages, these shell languages will seem clunky – or even evil. But since most shell scripts are small, most people don’t mind the torture.

In bash, the control flow commands seem to be thrown in as a after thought rather than something that’s built into the language. If you don’t believe me, compare the ‘if’ loop of bash and Perl.

Bash

This code checks wether the variable ‘$value’ has the value 0 – if so it prints the message ‘No Value’

if [ $value -eq 0 ] ; then
	echo "No Value"
fi

Perl

The same code in Perl…

if($value == 0) {
	print "No Value";
}

Of course, Perl experts will go for if(!$value) or even unless($value) – but that’s not the point. See how better the code looks in Perl. Yeah, even I am surprised to hear those words – Perl is considered by many to be an ‘ugly’ language. But when compared to shell languages, Perl is a gem(sorry about the pun – couldn’t resist).

The Solution

The solution is simple – don’t use a shell language to write your shell scripts – use a high level scripting language like Perl. Or Python, Ruby, PHP, Tcl or even JavaScript.

I still use bash to write shell scripts – but if the shell script has an if condition(other than a simple argument check), I use a higher language – usually Perl.

Advantages of using a Shell Language

  • Its supported in even the tinest linux distros – even in embedded systems.
  • Its the method preferred by the majority – I hope this will change soon.
  • Command calls look more natural in a shell language.

Advantages of using a High Level Language

  • Better Code
  • Easy to maintain
  • Faster development
  • Libraries provide a lot of functionality
  • Easier to port to different platforms.

So why am I telling you all this? Two reasons…

One, the next time you are going to write a shell script, I want you to choose a high level language rather than using bash.

The second reason is that now that my series on Linux MP3 Players are over, I am going to take a small break from desktop posts and write on more ‘linuxy’ topics. And one of those topic is Shell Scripting. So in the future posts, I am going to share some of my shell scripts with you. So when I publish a Perl script and call it a shell script, I don’t want you to get confused .

Update: This article is translated to Serbo-Croatian language.
Translation into Portuguese

42 comments

  1. In your example, you seem to be using -eq instead of != just to make Bash seem more foreign.

    Unlike perl you generally don’t have to install anything extra on your system for BASH

    Is there an easy way to do the following in PERL? Because in BASH doing operations on multiple files at once is very easy.

    The below will also work with filenames with strange characters in their name; it will figure out the escape characters for you so passing in the filenames to another program is no trouble.

    for x in /home/*; do
    echo $x
    done

    Is there an easy way of telling if a file/directory exists using Perl?

    # use -d for directories
    if [ -e /tmp/file.txt ]; then
    echo “file.txt exists”
    fi

    PERL is a nice language, but there is no reason to use it over BASH for automated tasks.

  2. Of course there are easy ways:

    print “$_\n” for ;

    and

    print “file.txt exists” if -e /tmp/file.txt;

    for easy tasks i also use bash – but with strings, calculations … involved i am much more happy with perl

  3. uups, in the first example my angle brackets disappeared..
    it should read

    print “$_\n” for (/home/*);

    and now replace the round brackets with angle ones 😉

  4. > Is there an easy way to do the following in PERL? Because in BASH doing operations on multiple files at once is very easy.
    > The below will also work with filenames with strange characters in their name; it will figure out the escape characters for you so passing in the filenames to another program is no trouble.
    > for x in /home/*; do
    > echo $x
    > done
    In perl, this can be done by
    chdir(‘/home/binnyva’);
    foreach $f (<*>) {
    print $f . “\n”;
    }

    > Is there an easy way of telling if a file/directory exists using Perl?
    > # use -d for directories
    > if [ -e /tmp/file.txt ]; then
    > echo “file.txt exists”
    > fi
    Sure..
    if(-e “/tmp/hello.txt”) {
    print “Its there”;
    }

    > PERL is a nice language, but there is no reason to use it over BASH for automated tasks.
    If you are comfortable with bash, there is very little reason to switch over to perl – but most people are not that comfortable with bash. I, for one, prefer using high level languages over bash. But bash has its advantages – as noted in the article.

  5. >I still use bash to write shell scripts – but if the shell script has an if condition(other than a simple argument check), I use a higher language – usually Perl.

    I agree with you about using a higher language for more complicated scripts. Regarding the language, I prefer Python.

  6. @nongeekboy
    Python, Perl – it does’nt matter. I say tomato, you say tomoto(yeah, the joke is totally lost when written).

    As long as its not bash, I’m happy.

  7. Will: In the shell, using -eq forces a numeric comparison v/s using ==, which does a string comparison. It’s better to use the -eq/-ne versions in the shell when comparing numbers. It’s slightly faster, and will generate a warning if non-numeric input is presented.

    To counter the argument that perl looks better, consider a case statement in the shell – that has to be a big if/elsif/else block in perl, since perl has no case statement (yeah, there’s a non-standard module which sortof adds a case-like structure). There’s no perl equivalent which looks better than this shell code.

    case $var in
    1 ) print “var is one” ;;
    2 ) print “var is two” ;;
    * ) print “var is not one or two” ;;
    esac

    In any event…

    I use the shell (ksh, usually, but Bash is fine) for operations which involve a lot of system calls (automating processes) or looking at filenames. I use perl if I’m doing more than a quick grep on files or anything more involved – though one can do some pretty complicated things if one looks into the shell’s more advanced capabilities. They’re two different tools, and each has an area in which they’re the better choice. Much as I love perl (and I was a proessional perl developer for a while), the shell has its niche. 🙂

  8. I never tried perl for shell script before but now I find it to be useful for some instances and I agree with Danny that it has it’s niche.

  9. Hi Binny, thanks for the post — very imformative.

    As part of my career advancement with my current testing of software applications that sit on top of linux, I am learning scripting using Python — b/c that is the High Level Language the Lead Software developer on our team evangelizes.

    If it is worth you writing a post that compares the major High Level Languages, then I would love to see it.

    Sounds like, in the future, I might also desire to then learn some scripting in straight bash.

    My sitting in the lab for one-hour-plus, manually grepping for “errors” on log file results from an overnight test, is getting boring.

    “I’m feeling the pain, so I’m seeing the light.”

  10. this article is very very informative ..
    hats off to Binny..good work guy…
    could u please answer my simple query
    Ques -1>>what are all the scenarios under which perl is useful for software testing??
    Since i am a Sr. Software Test Engineer and planning to explore this world if it may helpful in carrying my testing Task.

    So your expert suggestion is solicted?

  11. @Binny
    Not a problem Binney…I have one more question..
    Suppose i have couple of file in my c:\ drive and i want to replace TRASECTION to CARD_TRANSECTION to give better visibility in around 50 file.So i have perl script habdy.
    But how i run this..in Window platform..

    Could we execute this script on Windows Platform after installing PERL softwate..Since i though i will install trial version of the same.
    or
    We cant execute perl script on windows platform?
    or
    Could we install PERL on Windows platform?

    I know this all are my silly question ,But if you could answer these
    it i would be very gteatful to you my brother..
    You could send your answer rahulkumarbangalore@yahoo.com
    Regards,
    Rahul

  12. ok i am downloading Windows installer —http://www.activestate.com/store/download_file.aspx?binGUID=c1b8de46-32d8-440c-a17a-80eb31e68903

    from above site.But it says SYSTEM REQUIREMENT as below
    1>x86 or x64 architecture

    2>Windows 2000, XP or Vista: No additional requirements.

    3>Perl for ISAPI: ISAPI compatible Web server such as IIS 4.0+ or PWS 4.0+

    4>PerlScript: ActiveX scripting host such as IE 4.0+, or Windows Scripting Host

    RQUIRMT1> I alredy have
    RQUIRMT2> I have winXP
    RQUIRMT3>wt is this.how to full fill this
    RQUIRMT4>is this requirement talking
    about IE version if so i have
    IF7.What is windows scripting
    HOST?

    Could you please give some more idea what i need to do to satisfy requirement 3 and 4?

    If i am not wrong RQUIRMT3 talks about some web server..Could we download this from Microsoft? if yes i could google it and install the same..
    One more question why this is really require..i still didnt get..i know for my application it is not require since i need to replace string with new string..
    But i might guess it could be useful somrwhere..So could you let me know where it could play a role?

    Thanks & Regards,
    Rahul

  13. hi,
    i need small favour from this forum.
    i need to search a string into a file and i need to replace it with new string.
    but i am unable to do so.i hve done some home work as well.i am pasting down…
    $data_file=”aaa.txt”;
    #appending
    open (handler,”>>$data_file”) || die(“coudnt open”);
    @raw_data=;
    for ($i=o ;$i<4 ;$i++)
    if (@raw_data[i] eq “old_string”)#if matched is found
    wt should i write here to replace??
    i am executing this perl script on Window through Command prompt

    end if
    close (handler)

    Regards,
    Lax!!

  14. you may be able to use something like:

    #!/bin/perl -w
    #
    use strict;

    my $search_string=”this”;
    my $replace_string=”that”;

    while() {
    $_ =~ s/$search_string/$replace_string/ig;
    print $_;
    }

    or from a command line you could just run it as :

    $:> cat aaa.txt | perl -ne ‘$_ =~ s/This/that/ig; print ‘;

  15. Hello,

    I have a question. I use shell script (bash, ksh, etc.) to automate a lot of things like grouping all OS commands together and run them in a script. For example, in an Oracle RAC environment running on Linux, I use srvctl extensively for adding and removing services, etc. If I use perl, how can I do this? Please give examples as I do not know Perl well.

    Regards,

  16. could any one do a favour for me.. i want a bash or python or perl script to run more than one Bash scripts… please help me out…

  17. Every unix sysadmin should be well versed in shell scripting. It’s mandatory. For that same group of people, perl is not mandatory, but a very nice to have. Take a look at the OS init scripts. They’re all shell scripts. These are the kinds of scripts sysadmins work with the great majority of the time. Yes, there are times when perl makes more sense, but I would say this is rare…for the sysadmin crowd. If you’re a sysadmin, look at any decent init script. If you can’t write something similar…with the same level of difficulty/elegance, you need to improve your shell scripting skills. Another aspect to keep in mind is shell scripting is basically just blocks of system commands. As a sysadmin, the better you can shell script, the better you are at the command line and vice versa. A sysadmin spends most of his time operating on the commandline. You might as well write as much of your scripts in bash as possible to complement your commandline skills.

  18. I have written a complete system for automatization in Bash:
    -> Basic Mechanism: SSH/SCP remote scripting through expect to machine’s lists and SQL scripts launch.
    -> Intepreters: build bash scripts based on tag language that will be launched by the basic mechanism.

    For me, it was a painful experience, especially character mapping and so on. I solved this issue using temporal files with bash environment on it.

    It took me a whole year to have something functional (tag definition language).

    Bash is a requirement in my enviroment but I was wondering, is there any other tool that simplifies the stuff. Bash is really painful. The core, right now, is working beutifully due to it’s simplicity. But it is a challenge to code a whole project (with modules, and so on).

    Another thing is that it is difficult to parallized task keeping a clear execution path. Using nohup with counters to know if all processes ended.

    As a conclusion: bash can be fine for little scripts only. I guess perl and so on allow you to manage threads and stuff like that and have a logger (you don’t have to build your own logger), to make it faster, and, hopefully, faster development.

    Any ideas? It is working fine right now, and it is flexible to extend since each module do very little things. But each platform brings new stuff to change, particulary the character mapping and so on.

    Thanks,

    Cesar

  19. If the script I want to write does many system calls, I usually use the bash. If the script I want to write does many calculations, I use perl. If it does both, I decide which is easier. It’s a matter of task.

  20. As a contractor that values job duration over value, I much more prefer bash:) Honestly, these kids today checkout one my bash scripts and their heads just about fall off!

  21. Found this after frustratingly trying to write such a simple bash script.
    I haven’t even been formally taught bash (or even formally introduced), but have been “using” it for years now.
    I I still absolutely hate it. I have a programming background and can not come to grips with bash syntax. Whether or not this makes me a dunce – i do no care – bash KILLS my productivity. Reading vague man pages and messing with string manipulations just takes to much time. I like this article because it makes me feel like a real human again after suffering over the most simple scripts. Perl or python and don’t look back! Shoot i can write these functions out in C in maybe 1/8 the time.
    I really do want to use bash efficiently – but its far to annoying to bother with.

    PS while using bash it’s almost like everything in /bin is a function, or a library.
    most programmers have their toolset of libraries they like to use; with scripting you have many choices of apps that do the same thing in different ways – compound that on the fact that man pages are often lacking or incomplete in information, and it is a real pain in the neck to find the right command to use in a script — whereas a header file is easy to read.

  22. Jay – the O’Reilly book on Bash is a really useful book to read. While you can find a lot of information on the web about basic shell scripting, you have to look harder to gain an understanding of the more advanced string manipulation function, the special filehandles which allow you to make network connections, etc. And on your comment about stuff in /bin being functions, you’re not too far off – several things which are binaries in bin are actually overridden as builtins within the shell, so they’re really more like functions and not like external binaries which have the fork/exec overhead you’d expect.

    Anyway, the book is a good place to find that info which basically shows you what you don’t know, and then you can find more information on those topics (once you know what you’re looking for) online.

  23. Perl is ugly … indeed, when compared to other languages like C or even python, not to shell scripts

  24. Good discussions/brainstorming about perl vs bash scripting. One comment I have is about the “strict” variable checking in Perl. By using “use strict”, we can force perl to do strict variable checking i.e. if a variable is erroneously used (and it wasn’t created in the first place), perl would loudly complain. This is very useful during development and to eliminate all those typos that could crrep into your code.

    I am wondering if there is a equivalent feature in bash? For example in bash if I do a typo in one of the variables I intend to use, bash would happily go on without complaining. But this would lead to serious and very hard to find bugs in the code, since the variable with the typo wasn’t defined in the first place. I have struggled with this issue in bash.

    So still my question is, is there a way to do strict variable checking in shell scripts (particularly bash)? If not that is a major problem.

  25. Raj – I prefer to use
    set -o nounset
    (which I think is more readable / self-documenting). 🙂

    You probably also want to look into the “typeset -i” (and synonymous “integer”) builtin, which will treat a variable as a number and throw an error if you attempt to store a non-numeric value.

    The “set -o errexit” command is also useful, as that will cause your script to exit if any command fails without being trapped (by “trapped” I mean not the conditional in an if statement or of the form “command || action”).

  26. I totally agree with VU as a sysadmin I run into shell scripts *all* the time.

    I do not have a developers background and I found myself really enjoying writing -scripts in bash. Like VU said it is chunks of command line mixed in with some cool logic.

    Then as the scripts got more and more complex I often find myself pulling at hair that is just not there. I often joke that if I had a mortal enemy I would call him syntax.

    Now I am left wondering if I should learn a higher level language, would that help or confuse things. As I do feel mostly comfortable with bash scripting but often still run into hurdles.

Leave a Reply

Your email address will not be published. Required fields are marked *