Select an arbitrary column of text in UNIX

Contributed by qmchenry

October 6, 2003

Tagged: UNIX

UNIX has wonderfully powerful text processing capabilities. There are numerous ways to solve the same problem. Frequently, for example, it is necessary to extract a single column of data from a text file or output stream. This recipe will present several solutions to this problem.

Many data files have data fields delimited by a single character like a tab or colon. To extract the full name field out of /etc/passwd, the fifth colon-delimited field, use:

cut -d : -f 5 /etc/passwd

The cut command allows a great deal of flexibility in cutting data. In this case, the -d : directs cut to use a colon character as the delimiter. The -f 5 parameter directs cut to extract only the fifth field. The field parameter makes cut extrememly flexible. Other examples are -f 2-5 to extract fields 2 through 5, -f 1,3,7 to extract the first, third, and seventh fields.

To extract a fixed set of columns, for example the column numbers 44 through 49 from a long directory listing (ls -l), use the following command:

ls -l | cut -c 44-49

On many UNIX systems, these columns represent the modification date. Like the -f parameter, the -c parameter can accept alternative values such as -c 5,7,6,8 will present those character positions in that order.

One of the trickier column extraction involves the presence of a variable amount of whitespace between fields. To extract the process id (second) field from a process listing (ps -ef), cut will not work. Another powerful text manipulator in UNIX is awk which understands that several spaces should be counted as a single whitespace. To extract the pid from a ps -ef, use:

ps -ef | head | awk '{print $2}'

Awk is an incredibly powerful tool, and this is a trivial but useful application of it.

Previous recipe | Next recipe |

Viewing 2 Comments

Anonymous 2 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

I don't know if you've experienced consistency problems, but I have been using cut to get pid information from ps for a long time. For example 'ps -ef | cut -c 10-15' works fine for me.

reply edit reblog flag

Anonymous 2 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

ps -ef|awk '{print $2}'

I use it like follows...

ps -ef|grep <filter string>|awk '{print $2}'|xargs kill -15

Cheers.

reply edit reblog flag