Reading a List of Data into a Function Using a Script

My job requires the managing of large amounts of files and data (about 6500 files/items per query/set). Managing and querying such large amounts of data by hand is impossible and a waste of time when you can have a KornShell script do this for you. This tech-recipe is just an example of what you can do with some variables, menus, and functions.


Overview: In this tutorial, I will give a few sample scripts and explain how each works. From there, you can hopefully use the fundamental concepts in each and apply it to your own area/field/use. However, there is more going on in each of my examples than just the script. You also have to be sure that the way you are setting up the data to be queried is going to work. (Take for instance an example in which we have 100 customers who are identified in our system by a 5 digit customer number. Be sure to keep the data structure for them consistent. When you go back to write a script to query the data, it makes the process much easier). Functions are not executed until called within the script.

Creating a Script that Uses a Function to Call in a List of Data

Script Example:
Copy the text in the quote below to a file named script.ksh.
Then create list.txt file in the same folder with these numbers on a line:
1
2
3
Then create three text files with the same name as number in the the list. (i.e., 1.txt, 2.txt, 3.txt). Put different data in each.

    # basic script that uses a function to call in a list of data to then query
    # function_test.ksh

    # creates function called readDATA
    function readDATA
    {
    echo $data’.txt’
    sleep 1
    }

    #main script that will call function
    echo “Test Script: Read Data from List, test.txt”

    while read -r data
    do
    readDATA
    done < list.txt

Here is what each script is doing:

The first thing we did was create the function. A function is basically just a mini-script within a script that you can call at any time. The function we set up is just going to echo the contents of the file $data.txt to the screen.

The next thing we set up was the while statement to execute the script.
This will read the list file, list.txt. Then for each line in the list.txt, it will temporarily assign it to be the $data variable and send that variable through the function.

Let’s see it in action.

while -r read (using our list from above of 1, 2 , 3 on separate lines)
the first time through it would see the read = 1
It then passes the 1 value to the readDATA function.

It then translates the $data variable to 1 and will echo 1.txt to the screen.

The script will keep running through the file until it reaches the end of the file. ( done < list.txt defines this.)

This is a basic script and usagee of reading from a list. I have a script I use at work that allows me to define the list files and also have more than one variable per line and so on. Understanding how to read the data from a list and how functions work was a great way for me to really start to understand the power of ksh.

 

About Jimmy Selix

Jimmy Selix is an early adopter that loves to be one of the first on the block to have the latest and greatest in technology and gadgets. Another love of his is being able to share his knowledge to others seeking it. Feel free to drop any comments or questions that you may have.
View more articles by Jimmy Selix

The Conversation

Follow the reactions below and share your own thoughts.

3 Responses to “Reading a List of Data into a Function Using a Script”

  1. April 12, 2010 at 5:26 pm, Anonymous said:

    i have a data file that has 2 columns, name1 and name2. How do i load this info in to 2 seperate arrays call, array_n1 and array_n2? Also my data file has # and space. i need to skip these when i load my arrays. i am using k shell.

    Reply

  2. August 09, 2010 at 3:17 pm, Lauritalamasbonita17 said:

    hola amigossoy dee bogota solo escrivo esto para saver como se crea una carperta unix si saven por favor escrivan es urgente

    Reply

  3. September 19, 2012 at 3:08 am, Joel said:

    Hey Jimmy. Just wanted to know that this came up in a google result, and really helped me get the leg up on the logic that I wanted. I still don’t 100% understand the implications of the while -r read yet, but I’ve used your tutorial to set up an automated script which can read from a list, and egrep out the desired parts, and append the result to another dump/log file.

    Really appreciate it, nice work.

    Reply

Leave a Reply