Lab 1 : Simple Shell Scripting

The purpose of this lab is to gain familiarity with the some Unix utilities and the shell by examining a text file and generating some reports.

This is the type of investigation you might do to determine how a server had been compromised or to see whether a system is a particular target.

  1. Acquire messages.1 file from here
    This is a log file from a linux server. It includes the log of many failed ssh attacks and other system anomalies.

    The ssh application is used to communicate securely between remote computer systems. It's often used as a remote access tool for routers, web servers, mail servers, etc.

    There are thousands of robots out there looking for systems running the ssh daemon (sshd). When they find an ssh daemon, they try simple password guessing attacks using common userids (like admin) and simple passwords (like admin again).

    These attacks aren't a serious threat if you've used good passwords.

    In Cuckoo's Egg Clifford Stoll described tracking down the Hanover Gang who infiltrated many computer's because the administrators never changed the default login/password from admin/admin.

    This lab will look at ways we can extract information from this mass of data. solution

  2. Open a command window and use the cd command to change to the folder with the messages.1 file.
    You can view the file contents by opening it within an editor window, or using the less command:
    less messages.1
    You can select only the ssh messages with a command like:
    grep sshd messages.1

    It's usually safer to avoid finding the match string within another word, by adding a space to the beginning and end of the word and putting quotes around the string like this:

    grep ' sshd ' messages.1

    Enter this to see what happens.

    Rework the previous grep command so that it has a leading space and a trailing open-square-bracket. solution

    Note that the grep command will accept a pattern to match, as well as exact characters. This causes some punctuation characters including "*", "[" and "]" to be special characters. When looking for these characters, they will need to be escaped to keep the grep command from trying to interpret them as something other than the character that they are.

  3. The wc command will tell you how many lines, words or characters are in a file or in stdin.

    Use the grep and wc commands to count number of ssh Failed logins from the command line.

    Notice that there are multiple lines reported for each sshd message. You can pipe the results of one grep to another to remove extra lines.


  4. Use echo with grep and wc to generate a human readable report like:
    There were 12345 Attacks solution

  5. Put this line into a text file, chmod it to be executable and prove it works.

    The examples in the lesson used the echo command to create a file with certain text in it.

    It's easier to work with a file using an editor.

    If you are familiar with a general-purpose editor like vi, vim, emacs, pico, nano, kwrite, etc. Feel free to use that. Otherwise, download the free Komodo Edit package from ActiveState

    KomodoEdit will run on Windows, Mac, or Linux. We'll use it for the Tcl part of the class.

  6. You can read a command line variable as $1. Modify the text file to report the attacks on a single day. You should be able to run it like this:
    %> 22
        613   Attacks on Dec 22

  7. Use a for loop to report the number of attacks on each day of Dec.
    Output should resemble:
       3294   Attacks on Dec 20
          2   Attacks on Dec 21
        613   Attacks on Dec 22
        176   Attacks on Dec 23
          0   Attacks on Dec 24
          0   Attacks on Dec 25
          0   Attacks on Dec 26
          0   Attacks on Dec 27

  8. Loops can be nested within each other like this:
     for x in 1 2 3 4
       for y in a b c d
         echo "$x$y"
    The message.1 file contains messages from these daemons:

    Modify the previous code so that it uses two nested loops, one for the daemons and one for the days to report how many times each type of message was generated on each day.

    The results should look like this:

    183 messages from dhcpd on Dec 20
    3 messages from last on Dec 20
    2350 messages from named on Dec 20
    0 messages from ntpd on Dec 20
    8391 messages from sshd on Dec 20
    1 messages from syslogd on Dec 20
    271 messages from dhcpd on Dec 21
    9 messages from last on Dec 21
    3198 messages from named on Dec 21

  9. You may have noticed that some instances of a daemon report multiple lines like this:
    Dec 20 05:26:00 bastion sshd[31714]: Invalid user admin from
    Dec 20 05:26:00 bastion sshd[31714]: error: Could not get shadow information for NOUSER
    Dec 20 05:26:00 bastion sshd[31714]: Failed password for invalid user admin from port 55272 ssh2

    This means that the previous report reflects the number of lines, not the number of reportable events.

    Add a cut and sort command to the previous script reduce the number of reported messages to the number of unique reports. solution

  10. OPTIONAL The selections in a for command can be the results of another command.

    For example, we can extract the IP addresses of sites that have tried to login with an invalid user with a line like this

    grep ssh mess*1 | grep invalid | cut -d ' ' -f 13 | sort -u
    This could be used with a for loop to find the number of attacks from each site like this:
    for ip in ` grep ssh mess*1 | grep invalid | cut -d ' ' -f 13 | sort -u`
      echo "`grep $ip mess*1 | grep invalid | wc -l` invalid logins From $ip"
    This will produce output like this:
    1302 invalid logins From
    8 invalid logins From
    94 invalid logins From
    1 invalid logins From
    Add another loop to iterate through the days of the month and report how many attacks from each IP address occurred on each day. The output should resemble this:
       1302   20832  143327 invalid logins From on Dec 20
          0       0       0 invalid logins From on Dec 21
          0       0       0 invalid logins From on Dec 22
          0       0       0 invalid logins From on Dec 23
Copyright Clif Flynt 2009