Remove ^M characters at end of lines in vi | UNIX | Tech-Recipes

Remove ^M characters at end of lines in vi

Contributed by qmchenry

October 4, 2003

Tagged: UNIX

UNIX treats the end of line differently than other operating systems. Sometimes when editing files in both Windows and UNIX environments, a CTRL-M character is visibly displayed at the end of each line as ^M in vi.

To remove the ^M characters at the end of all lines in vi, use:

:%s/^V^M//g

The ^v is a CONTROL-V character and ^m is a CONTROL-M. When you type this, it will look like this:

:%s/^M//g

In UNIX, you can escape a control character by preceeding it with a CONTROL-V. The :%s is a basic search and replace command in vi. It tells vi to replace the regular expression between the first and second slashes (^M) with the text between the second and third slashes (nothing in this case). The g at the end directs vi to search and replace globally (all occurrences).

Previous recipe | Next recipe |

Viewing 20 Comments

bofh468 5 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

Another way to do this is with the dos2unix utility. You can also use unix2dos to add in the extra ^M (carriage-return) characters to a text file so it's readable under DOS.

reply edit reblog flag

Michilimackinac 4 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

Where foo is the file that has the control M's

cat foo | col -b > foo2

reply edit reblog flag

nxnwaus 4 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

type the following command:

:%s/<control-v><control-m>//g

^M is the result of a control V, control M at the end of the line. if you actually hit control-v then control-m, "^M" will appear and will get rid of the character at the end of the line. hope this helps someone!!!

reply edit reblog flag

SubZero 4 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

col -bx <dosfile> newfile
does the biz. too :)

reply edit reblog flag

Anonymous 4 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

i want to include into .profile the possibility to dont give a user the prompt

reply edit reblog flag

BigJoe 3 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

The information about this issue was exactly what I was looking for.

Right On!

reply edit reblog flag

jibin.thomas@gmail.com 3 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

Hi this was so useful it help me a lot and our compnpay H.L.L India

reply edit reblog flag

Anonymous 3 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

...I also needed this desperately!! Thanks for posting it!

reply edit reblog flag

johnny 3 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

the dos2unix command does this too - though it may be deprecated.

reply edit reblog flag

johnny 3 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

ok so no flames :oops:

reply edit reblog flag

Richard 3 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

:D Perfect.....just what I needed ! Many Thanks !

reply edit reblog flag

Prasad 3 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

Thanks to all techy guys out there who posted solutions for this.

reply edit reblog flag

Anonymous 3 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

<ul id="quote"><h6>Richard wrote:</h6>:D Perfect.....just what I needed ! Many Thanks !</ul>

reply edit reblog flag

sasi 3 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

Hi,

THere is lot of non-continous blank lines in my shell. I want to find the blank line and I want to remove it. What is the command for that?

Sasi, India

reply edit reblog flag

qmchenry 2 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

If you want to remove all blank lines in your file, search for empty lines and replace them with nothing.. since ^ means beginning of line and $ means end of line, ^$ means a blank line, so

:%s/^$//g

will find blank lines and make them disapear.

reply edit reblog flag

Anonymous 2 years ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

All- I need to clean a bunch of files in UNIX that contain the ^M characters.

While I can do each one individually by :%s/^M//g I would like to know if there is an easy command line execution to serarch for any files containing the errors and correct them.

I tried this for all *.sql files in a given directory, but it didn't work:

for name in `ls *.sql` ; do sed 's/^M//' $name > ${name/.sql/N.sql} ; mv ${name/.sql/N.sql} $name ; done

If anyone has a suggestion or knows a simple script that can be run please let me know.

Thanks

reply edit reblog flag

Phobian 1 month ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

You could try using sed, the stream editor - it works on similar principles to vi's replace tool, but you can pipe into and out of it.

So, to change one file:
cat fileName | sed s/<ctrl-v><ctrl-m>//g >tmp && mv tmp fileName

Do NOT try to read from and write to the same file in one pipe, or you'll blank the file

To change all files in a directory with the .sql ending, run the following script whilst inside that directory:
#/bin/ksh
for fileName in $(ls *.sql); do
cat $fileName | sed s/^M//g >tmp && mv tmp $fileName
done

(bear in mind to enter ^M you need to enter <ctrl-v><ctrl-m> as above.

Hope that helps people
-phobiandarkmoon

reply edit reblog flag

Maxlen 3 weeks ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

type the following command in vi

:%s/.$//

reply edit reblog flag

Yo Whirrd up 3 weeks ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

Try perl -pi -e'tr/\015//d' <filename>

reply edit reblog flag

pradeep 2 days ago 1 point
Please login to rate.

Do you already have an account? Log in and claim this comment.

yes the ":%s/^ v ^M//g" is working... thank you very much...

reply edit reblog flag

blog comments powered by Disqus