Remove ^M characters at end of lines in vi

Contributor Icon Contributed by qmchenry  
Tag Icon Tagged: UNIX  

UNIX treats the end of line differently than other operating systems. Sometimes when editing files in both Windows and UNIX environments, a CTRL-M character is visibly displayed at the end of each line as ^M in vi.


To remove the ^M characters at the end of all lines in vi, use:

:%s/^V^M//g

The ^v is a CONTROL-V character and ^m is a CONTROL-M. When you type this, it will look like this:

:%s/^M//g

In UNIX, you can escape a control character by preceeding it with a CONTROL-V. The :%s is a basic search and replace command in vi. It tells vi to replace the regular expression between the first and second slashes (^M) with the text between the second and third slashes (nothing in this case). The g at the end directs vi to search and replace globally (all occurrences).

 

71 Comments -


  1. Phobian said on October 24, 2008

    You could try using sed, the stream editor – it works on similar principles to vi’s replace tool, but you can pipe into and out of it.

    So, to change one file:
    cat fileName | sed s///g >tmp && mv tmp fileName

    Do NOT try to read from and write to the same file in one pipe, or you’ll blank the file

    To change all files in a directory with the .sql ending, run the following script whilst inside that directory:
    #/bin/ksh
    for fileName in $(ls *.sql); do
    cat $fileName | sed s/^M//g >tmp && mv tmp $fileName
    done

    (bear in mind to enter ^M you need to enter as above.

    Hope that helps people
    -phobiandarkmoon

  2. Maxlen said on November 6, 2008

    type the following command in vi

    :%s/.$//

  3. Yo Whirrd up said on November 7, 2008

    Try perl -pi -e’tr/15//d’

  4. pradeep said on December 1, 2008

    yes the “:%s/^ v ^M//g” is working… thank you very much…

  5. soma said on December 11, 2008

    this solution is not working for ubuntu..:(

  6. Reetesh said on December 25, 2008

    Thanks it is working fine

  7. thanks said on January 14, 2009

    thank you for the nice tips. I have plenty of ctrl+M symbols in the dump txt files from dol.

  8. Neo said on February 12, 2009

    great info.. thanks

  9. 3pe said on February 24, 2009

    another way to get rid of those ^M’s
    :%s/r//g
    btw other systems then *nix are treating newlines differently :P

  10. Ashish said on April 28, 2009

    :%s/r/r/g

  11. a said on April 29, 2009

    this works ! thanks.

  12. xx said on May 5, 2009

    use $ to replace only at the end
    :%s@^M$@@g

  13. roy said on May 5, 2009

    You are a genius
    roy@roy.com

  14. sam said on May 31, 2009

    Excellent

  15. S said on June 1, 2009

    thanks it works well

  16. SOM said on July 6, 2009

    Great dude .. this works

  17. Ciprian said on July 6, 2009

    Lovely! I completely forgot about dos2unix and sincerely I like to use vim better if I can :)

  18. Vishal said on July 7, 2009

    I am trying ti get the diff of a file into a temp.txt file.
    When I do :%s/^M//g in my file, it says “Pattern not Found: ^M”.
    But when I see the temp.txt file for the diff it shows ^M on all the lines.
    Help me out for this, I dont want ^M in temp.txt file

  19. Anonymous said on July 23, 2009

    Thank you very much…for me saved lot of time.

  20. SnowLeopard said on August 30, 2009

    Thanks! :D

  21. awais said on September 1, 2009

    hi,
    grt man. it worked for me. keep up the good work.

    br

  22. Dmitriy Golub said on October 1, 2009

    Nice, thank you!

  23. Anonymous said on October 1, 2009

    Thank you, original post did not work for me, but this did.

  24. mhannesy said on October 21, 2009

    Just curious, :%s/r//g works but :%s/rn/n/g does not, why is that?

  25. Brett said on October 29, 2009

    You Sir are a genius. I award you five internets!

  26. Hedi said on November 11, 2009

    You could try :
    echo file | sed ’s/r//g’

  27. mhannesy said on November 21, 2009

    Ok, I found out why I ended up with the NUL control characters instead. From :help insert:

    “If you enter a value of 10, it will end up in the file as a 0. The 10 is a
    , which is used internally to represent the character. When writing
    the buffer to a file, the character is translated into . The
    character is written at the end of each line. Thus if you want to insert a
    character in a file you will have to make a line break.”

  28. Name said on December 4, 2009

    I am brand new to Vim. I am trying to remove ^M and replace it with a tab (actually replace it with spaces). I can’t seem to get this to work. I’ve done the following things and it isn’t working. Any advice?

    :set expandtab
    :%s/^M//g

    All it is doing is putting “” with the “^M” used to be. Any way to actually make the spaces appear? Thanks for your help.

  29. Name said on December 5, 2009

    Nevermind! I figured it out. I simply had to push “control + the tab key” and that resulted in what I needed. Thanks.

  30. Anonymous said on December 16, 2009

    hi,
    Thanks for the info.. I need to move the lines to the top after the ctrlM charecter. Is this possible ??

  31. Anonymous said on February 4, 2010

    Hi ,

    I did worked for me as well . Thanks guys for your help .

    Arif

  32. Anonymous said on February 8, 2010

    too good. this worked for me!

  33. Anonymous said on March 28, 2010

    Hi,

    Thanks for the solution. It worked.

    I also noticed other characters like and when using vi. I assume these are also converted because of Windows.

    How can I revert these back?

    Thanks,
    Allan

  34. p1 said on March 30, 2010

    Can anyone please help in removing character displayed in vi ?

  35. Anonymous said on April 8, 2010

    Hi,

    In my case, when I transfer my file from windows to unix, this control+m (^m) character gets added. I there any utility where I can do some changes in file at windows end so that ^M doesn’t get added to it when I transfer it to Unix?

    Thanks,
    Jatin Kapoor

  36. MiDSuMMeR said on May 13, 2010

    Type the following command:

    :%s/rn/r/g

    It works fine!

  37. Bugged123 said on June 21, 2010

    This worked..Thank you ..Original did not ..

  38. Lopezscu said on June 22, 2010

    same for me, the post author is a retard…

  39. Lopezscu said on June 22, 2010

    correct answer is
    :%s/{Ctrl+V}{Ctrl+M}//{Enter}

    where {Ctrl+V }{Ctrl+M} stands for, you must press Ctrl+V and then Ctrl+M

  40. lota said on July 12, 2010

    More likely you don’t know how to use vi.

  41. Leena said on August 10, 2010

    Hi,

    Remove the last line from a txt file.

    I have txt file with data like
    1235 5678
    3455 5678

    and after 2nd line I pressed the enter. I tried to remove that enter in using many unix commands. but in unix I think it doesnt consider it as new line as when I check wc -l it gives me 2 but when I send this file to my C function it is considered as 3rd line.
    Please telme the work around to remove the last line.

    Thanks,
    Leena

  42. Nitin said on September 24, 2010

    I know that ^M comes when we transfer a file from windows machine to a Unix machine from Telnet.

    At first place what is the harm in having ^M characters? Is it going to cause any kind of issues by letting them just around? Just curious to know.

  43. Yashaswikumar said on October 14, 2010

    Thank u so very much.. This really worked :) Saved my day :)

  44. Suchi said on October 22, 2010

    Thanks! The string re[place hadnt been working, but col -b worked for me :)

  45. Tallis said on October 28, 2010

    only tools cant use tools properly

  46. Ramakrishna Aazad said on November 2, 2010

    vi file name

    :%s[ctrl v][ctrl[m]//g worked for me to remove the cntrl m characters from the file. Thanks a lot :)

  47. Malice said on November 8, 2010

    The original post is fine, you just have to be able to read.

  48. Grit said on November 25, 2010

    u meatheads who posted dos2unix etc….the heading is “Remove ^M characters at end of lines in vi” not oustide of…….dummies

  49. Team_Awesome said on December 8, 2010

    nothing else people posted on here worked. Thank you tons!!!

  50. Vaibhav said on December 24, 2010

    perfect solution

  51. Bob said on January 3, 2011

    only a dummy does not know you can invoke any command, dos2unix etc…from within vi …idiot

  52. Vinoo Usa said on January 19, 2011

    Thank you so mcuh lopezzzzzzzzzz……….. you are my man… :)

  53. S_zarembo said on January 28, 2011

    the original post works just fine; you just need to read it carefully.
    Thanks

  54. Guest said on February 4, 2011

    col -bx worked for me…

  55. PR said on March 9, 2011

    This helps

  56. BMQ said on March 29, 2011

    I found dos2unix run against mysql slow log caused log to stop accruing entries. This forced me to restart mysql to restore slow log entries.

  57. Piyush said on April 29, 2011

    It worked .. thx
    I also replaced ‘^@’ with the same technique.

  58. Louispendis5 said on May 26, 2011

    you’re the man!

  59. Petalmelissa said on July 8, 2011

    Your attitude is unnecessary and not conducive to the sharing environment being attempted to be created here.. 

  60. eskay said on July 21, 2011

    It didn’t worked. I got an error as Pattern Not Found: ^M

    All I did is,
    vi filename –> esc –> :%s/^v^m//g –> enter

  61. ataraxic said on July 25, 2011

    the same for me –> Pattern Not Found: ^M
    What is the problem here???

  62. nit said on August 3, 2011

    Thanks Lopezscu ,sed version of ur examplecat foo | sed ’s/^M//g’the “cat foo | col -b > foo2″ version in below posts is not working for me

  63. Cesar said on October 14, 2011

    To everyone having the “Pattern not found” message:

    You MUST NOT WRITE ^V ^M

    ^V is: “press control, press V, release V, release control.”
    The same with ^M: “press control, press M, release M, release control.”

    Sure it will work as sure as you are using vi/vim.

  64. george said on November 1, 2011

    Thanks very much.

  65. jason said on November 23, 2011

    Ok,

    For those where :%s/^M//g was not working for them, I have a workaround for you.

    I was finding it wasn’t doing the replace, however when I compiled the file I was working on, it was complaining about the ^M which was really annoying.

    I can confirm that the following though will work.

    cat fileName | sed s/^M//g >tmp

    Just move the tmp file back to fileName and you have a file minus the ^M.

  66. Narsimulu said on November 28, 2011

    To everyone having the “Pattern not found” message:

    Please type control character twice and then type V and M.

    “press control, press control, press V, press control,press control, press M”

    -Narsimulu

  67. Tamil said on December 16, 2011

    Yes, It works fine, I also had the “Pattern Not Found” issue. Please press the following keys to have this function.

    1. Type %s
    2. press (ctrl + v) = it will give the (^) symbol
    3. Press (ctrl + m) = it will give the ^M in blue color.

    then add rest of the symbols. It will work.

  68. mooselix said on December 20, 2011

    Uh, all you need is this. No control-characters, etc.

    :%s/\r//g

  69. raj malhotra said on January 4, 2012

    Thanks Lopezscu
    yours worked and it saved time greatly :)

  70. #dhk said on January 11, 2012

    Its works
    :%s/[^0-9]$//g

  71. Manu said on January 19, 2012

    Hi Everyone,

    I am new to vi. I tried removing the ^M character from the file using

    sed s/\r//g

    but it didn’t work for me. I tried using (ctrl+M) instead of \r but that is also not working. whenever I press Ctrl+M it is taking me to the next line.

    I have written down my code like below:
    #!/bin/sh

    while read line
    do
    echo “$line” | sed s/[^0-9]//g >> temp
    done

 

RSS feed for comments on this post. TrackBack URL

Leave a comment -