Remove ^M characters at end of lines in vi

Contributor Icon Contributed by qmchenry Date Icon October 4, 2003  
Tag Icon Tagged: UNIX

UNIX treats the end of line differently than other operating systems. Sometimes when editing files in both Windows and UNIX environments, a CTRL-M character is visibly displayed at the end of each line as ^M in vi.


To remove the ^M characters at the end of all lines in vi, use:

:%s/^V^M//g

The ^v is a CONTROL-V character and ^m is a CONTROL-M. When you type this, it will look like this:

:%s/^M//g

In UNIX, you can escape a control character by preceeding it with a CONTROL-V. The :%s is a basic search and replace command in vi. It tells vi to replace the regular expression between the first and second slashes (^M) with the text between the second and third slashes (nothing in this case). The g at the end directs vi to search and replace globally (all occurrences).

Previous recipe | Next recipe |
 
  • bofh468
    Another way to do this is with the dos2unix utility. You can also use unix2dos to add in the extra ^M (carriage-return) characters to a text file so it's readable under DOS.
  • Michilimackinac
    Where foo is the file that has the control M's

    cat foo | col -b > foo2
  • Reetesh
    Thanks it is working fine
  • a
    this works ! thanks.
  • ucsbliu
    Thank you, original post did not work for me, but this did.
  • nxnwaus
    type the following command:

    :%s/<control-v><control-m>//g

    ^M is the result of a control V, control M at the end of the line. if you actually hit control-v then control-m, "^M" will appear and will get rid of the character at the end of the line. hope this helps someone!!!
  • roy
    You are a genius
    roy@roy.com
  • SubZero
    col -bx <dosfile> newfile
    does the biz. too :)
  • Anonymous
    i want to include into .profile the possibility to dont give a user the prompt
  • BigJoe
    The information about this issue was exactly what I was looking for.

    Right On!
  • jibin.thomas@gmail.com
    Hi this was so useful it help me a lot and our compnpay H.L.L India
  • Anonymous
    ...I also needed this desperately!! Thanks for posting it!
  • johnny
    the dos2unix command does this too - though it may be deprecated.
  • johnny
    ok so no flames :oops:
  • Richard
    :D Perfect.....just what I needed ! Many Thanks !
  • Prasad
    Thanks to all techy guys out there who posted solutions for this.
  • Anonymous
    <ul id="quote">
    Richard wrote:
    :D Perfect.....just what I needed ! Many Thanks !</ul>
  • sasi
    Hi,

    THere is lot of non-continous blank lines in my shell. I want to find the blank line and I want to remove it. What is the command for that?

    Sasi, India
  • qmchenry
    If you want to remove all blank lines in your file, search for empty lines and replace them with nothing.. since ^ means beginning of line and $ means end of line, ^$ means a blank line, so

    :%s/^$//g


    will find blank lines and make them disapear.
  • Anonymous
    All- I need to clean a bunch of files in UNIX that contain the ^M characters.

    While I can do each one individually by :%s/^M//g I would like to know if there is an easy command line execution to serarch for any files containing the errors and correct them.

    I tried this for all *.sql files in a given directory, but it didn't work:

    for name in `ls *.sql` ; do sed 's/^M//' $name > ${name/.sql/N.sql} ; mv ${name/.sql/N.sql} $name ; done

    If anyone has a suggestion or knows a simple script that can be run please let me know.

    Thanks
  • Phobian
    You could try using sed, the stream editor - it works on similar principles to vi's replace tool, but you can pipe into and out of it.

    So, to change one file:
    cat fileName | sed s/<ctrl-v><ctrl-m>//g >tmp && mv tmp fileName

    Do NOT try to read from and write to the same file in one pipe, or you'll blank the file

    To change all files in a directory with the .sql ending, run the following script whilst inside that directory:
    #/bin/ksh
    for fileName in $(ls *.sql); do
    cat $fileName | sed s/^M//g >tmp && mv tmp $fileName
    done

    (bear in mind to enter ^M you need to enter <ctrl-v><ctrl-m> as above.

    Hope that helps people
    -phobiandarkmoon
  • Maxlen
    type the following command in vi

    :%s/.$//
  • Yo Whirrd up
    Try perl -pi -e'tr/\015//d' <filename>
  • SOM
    Great dude .. this works
  • pradeep
    yes the ":%s/^ v ^M//g" is working... thank you very much...
  • soma
    this solution is not working for ubuntu..:(
  • thank you for the nice tips. I have plenty of ctrl+M symbols in the dump txt files from dol.
  • Neo
    great info.. thanks
  • 3pe
    another way to get rid of those ^M's
    :%s/\r//g
    btw other systems then *nix are treating newlines differently :P
  • Ashish
    :%s/\r/\r/g
  • xx
    use $ to replace only at the end
    :%s@^M$@@g
  • sam
    Excellent
  • S
    thanks it works well
  • Lovely! I completely forgot about dos2unix and sincerely I like to use vim better if I can :)
  • Vishal
    I am trying ti get the diff of a file into a temp.txt file.
    When I do :%s/^M//g in my file, it says "Pattern not Found: ^M".
    But when I see the temp.txt file for the diff it shows ^M on all the lines.
    Help me out for this, I dont want ^M in temp.txt file
  • raod
    Thank you very much...for me saved lot of time.
  • Thanks! :D
  • awais
    hi,
    grt man. it worked for me. keep up the good work.

    br
  • Nice, thank you!
  • mhannesy
    Just curious, :%s/\r//g works but :%s/\r\n/\n/g does not, why is that?
  • mhannesy
    Ok, I found out why I ended up with the NUL control characters instead. From :help insert:

    "If you enter a value of 10, it will end up in the file as a 0. The 10 is a
    <NL>, which is used internally to represent the <Nul> character. When writing
    the buffer to a file, the <NL> character is translated into <Nul>. The <NL>
    character is written at the end of each line. Thus if you want to insert a
    <NL> character in a file you will have to make a line break."
  • Brett
    You Sir are a genius. I award you five internets!
  • Hedi
    You could try :
    echo file | sed 's/\r//g'
  • Name
    I am brand new to Vim. I am trying to remove ^M and replace it with a tab (actually replace it with spaces). I can't seem to get this to work. I've done the following things and it isn't working. Any advice?

    :set expandtab
    :%s/^M/<Tab>/g

    All it is doing is putting "<Tab>" with the "^M" used to be. Any way to actually make the spaces appear? Thanks for your help.
  • Name
    Nevermind! I figured it out. I simply had to push "control + the tab key" and that resulted in what I needed. Thanks.
  • vijaycgowda
    hi,
    Thanks for the info.. I need to move the lines to the top after the ctrlM charecter. Is this possible ??
  • mohmedarif
    Hi ,

    I did worked for me as well . Thanks guys for your help .

    Arif
  • shraddhab
    too good. this worked for me!
blog comments powered by Disqus