Remove ^M characters at end of lines in vi

UNIX treats the end of line differently than other operating systems. Sometimes when editing files in both Windows and UNIX environments, a CTRL-M character is visibly displayed at the end of each line as ^M in vi.


To remove the ^M characters at the end of all lines in vi, use:

:%s/^V^M//g

The ^v is a CONTROL-V character and ^m is a CONTROL-M. When you type this, it will look like this:

:%s/^M//g

In UNIX, you can escape a control character by preceeding it with a CONTROL-V. The :%s is a basic search and replace command in vi. It tells vi to replace the regular expression between the first and second slashes (^M) with the text between the second and third slashes (nothing in this case). The g at the end directs vi to search and replace globally (all occurrences).

 

About Quinn McHenry

Quinn was one of the original co-founders of Tech-Recipes. He is currently crafting iOS applications as a senior developer at Small Planet Digital in Brooklyn, New York.
View more articles by Quinn McHenry

The Conversation

Follow the reactions below and share your own thoughts.

93 Responses to “Remove ^M characters at end of lines in vi”

  1. October 24, 2008 at 8:57 am, Phobian said:

    You could try using sed, the stream editor – it works on similar principles to vi’s replace tool, but you can pipe into and out of it.

    So, to change one file:
    cat fileName | sed s///g >tmp && mv tmp fileName

    Do NOT try to read from and write to the same file in one pipe, or you’ll blank the file

    To change all files in a directory with the .sql ending, run the following script whilst inside that directory:
    #/bin/ksh
    for fileName in $(ls *.sql); do
    cat $fileName | sed s/^M//g >tmp && mv tmp $fileName
    done

    (bear in mind to enter ^M you need to enter as above.

    Hope that helps people
    -phobiandarkmoon

    Reply

  2. November 06, 2008 at 9:11 pm, Maxlen said:

    type the following command in vi

    :%s/.$//

    Reply

    • July 02, 2012 at 1:29 pm, rohit said:

      > Awesome it works :)

      Reply

    • August 23, 2012 at 3:50 am, Arbab said:

      This worked for me

      Reply

    • October 24, 2012 at 8:07 pm, kan said:

      Thanks Man… It worked…. Your knowledge helped me…..>

      Reply

    • October 27, 2012 at 10:56 pm, Mohamed said:

      > this one works thanks

      Reply

    • February 21, 2013 at 4:50 pm, Mike said:

      > Keep in mind, if you happen to have edited the file and put some normal newlines in and then use this, you will loose a character off the end of every line. Which is why the :%s/^V^M// is a better way to go.

      Reply

      • September 13, 2013 at 7:40 am, yspatro said:

        It worked !! thx much

        Reply

      • March 10, 2014 at 5:04 am, Sachin said:

        > +1. It removes the last character of every line, irrespective of if its the ^M or anything else.

        Reply

  3. November 07, 2008 at 4:48 am, Yo Whirrd up said:

    Try perl -pi -e’tr/15//d’

    Reply

    • July 06, 2009 at 4:19 am, SOM said:

      Great dude .. this works

      Reply

  4. December 01, 2008 at 11:37 am, pradeep said:

    yes the “:%s/^ v ^M//g” is working… thank you very much…

    Reply

  5. December 11, 2008 at 7:25 am, soma said:

    this solution is not working for ubuntu..:(

    Reply

    • January 11, 2013 at 10:32 pm, rseal said:

      :%s/^q^M//

      Reply

    • January 11, 2013 at 10:32 pm, rseal said:

      Use CTRL-q CTRL-m

      Reply

      • September 26, 2013 at 10:50 pm, varsha said:

        > the solution is not working on AIX box.

        Reply

  6. December 25, 2008 at 2:52 am, Reetesh said:

    Thanks it is working fine

    Reply

  7. January 14, 2009 at 6:21 pm, thanks said:

    thank you for the nice tips. I have plenty of ctrl+M symbols in the dump txt files from dol.

    Reply

  8. February 12, 2009 at 7:42 am, Neo said:

    great info.. thanks

    Reply

  9. February 24, 2009 at 10:59 am, 3pe said:

    another way to get rid of those ^M’s
    :%s/r//g
    btw other systems then *nix are treating newlines differently :P

    Reply

  10. April 28, 2009 at 11:14 am, Ashish said:

    :%s/r/r/g

    Reply

  11. April 29, 2009 at 8:52 am, a said:

    this works ! thanks.

    Reply

  12. May 05, 2009 at 10:14 am, xx said:

    use $ to replace only at the end
    :%s@^M$@@g

    Reply

  13. May 05, 2009 at 5:29 pm, roy said:

    You are a genius
    roy@roy.com

    Reply

  14. May 31, 2009 at 8:20 pm, sam said:

    Excellent

    Reply

  15. June 01, 2009 at 3:56 am, S said:

    thanks it works well

    Reply

  16. July 06, 2009 at 6:32 am, Ciprian said:

    Lovely! I completely forgot about dos2unix and sincerely I like to use vim better if I can :)

    Reply

  17. July 07, 2009 at 7:12 am, Vishal said:

    I am trying ti get the diff of a file into a temp.txt file.
    When I do :%s/^M//g in my file, it says “Pattern not Found: ^M”.
    But when I see the temp.txt file for the diff it shows ^M on all the lines.
    Help me out for this, I dont want ^M in temp.txt file

    Reply

  18. July 23, 2009 at 2:51 am, Anonymous said:

    Thank you very much…for me saved lot of time.

    Reply

  19. August 30, 2009 at 1:19 am, SnowLeopard said:

    Thanks! :D

    Reply

  20. September 01, 2009 at 11:25 am, awais said:

    hi,
    grt man. it worked for me. keep up the good work.

    br

    Reply

  21. October 01, 2009 at 1:36 pm, Dmitriy Golub said:

    Nice, thank you!

    Reply

  22. October 01, 2009 at 8:24 pm, Anonymous said:

    Thank you, original post did not work for me, but this did.

    Reply

  23. October 21, 2009 at 8:06 pm, mhannesy said:

    Just curious, :%s/r//g works but :%s/rn/n/g does not, why is that?

    Reply

    • November 21, 2009 at 6:04 pm, mhannesy said:

      Ok, I found out why I ended up with the NUL control characters instead. From :help insert:

      “If you enter a value of 10, it will end up in the file as a 0. The 10 is a
      , which is used internally to represent the character. When writing
      the buffer to a file, the character is translated into . The
      character is written at the end of each line. Thus if you want to insert a
      character in a file you will have to make a line break.”

      Reply

  24. October 29, 2009 at 5:24 am, Brett said:

    You Sir are a genius. I award you five internets!

    Reply

  25. November 11, 2009 at 6:24 pm, Hedi said:

    You could try :
    echo file | sed ‘s/r//g’

    Reply

  26. December 04, 2009 at 5:31 pm, Name said:

    I am brand new to Vim. I am trying to remove ^M and replace it with a tab (actually replace it with spaces). I can’t seem to get this to work. I’ve done the following things and it isn’t working. Any advice?

    :set expandtab
    :%s/^M//g

    All it is doing is putting “” with the “^M” used to be. Any way to actually make the spaces appear? Thanks for your help.

    Reply

    • December 05, 2009 at 3:33 am, Name said:

      Nevermind! I figured it out. I simply had to push “control + the tab key” and that resulted in what I needed. Thanks.

      Reply

  27. December 16, 2009 at 4:46 pm, Anonymous said:

    hi,
    Thanks for the info.. I need to move the lines to the top after the ctrlM charecter. Is this possible ??

    Reply

  28. February 04, 2010 at 11:03 am, Anonymous said:

    Hi ,

    I did worked for me as well . Thanks guys for your help .

    Arif

    Reply

  29. February 08, 2010 at 7:17 am, Anonymous said:

    too good. this worked for me!

    Reply

  30. March 28, 2010 at 5:34 am, Anonymous said:

    Hi,

    Thanks for the solution. It worked.

    I also noticed other characters like and when using vi. I assume these are also converted because of Windows.

    How can I revert these back?

    Thanks,
    Allan

    Reply

  31. March 30, 2010 at 6:45 am, p1 said:

    Can anyone please help in removing character displayed in vi ?

    Reply

  32. April 08, 2010 at 11:31 pm, Anonymous said:

    Hi,

    In my case, when I transfer my file from windows to unix, this control+m (^m) character gets added. I there any utility where I can do some changes in file at windows end so that ^M doesn’t get added to it when I transfer it to Unix?

    Thanks,
    Jatin Kapoor

    Reply

  33. May 13, 2010 at 11:35 am, MiDSuMMeR said:

    Type the following command:

    :%s/rn/r/g

    It works fine!

    Reply

  34. June 21, 2010 at 8:44 pm, Bugged123 said:

    This worked..Thank you ..Original did not ..

    Reply

    • June 22, 2010 at 10:16 pm, Lopezscu said:

      same for me, the post author is a retard…

      Reply

      • July 12, 2010 at 12:45 am, lota said:

        More likely you don’t know how to use vi.

        Reply

      • October 28, 2010 at 3:05 pm, Tallis said:

        only tools cant use tools properly

        Reply

    • November 08, 2010 at 9:45 pm, Malice said:

      The original post is fine, you just have to be able to read.

      Reply

  35. June 22, 2010 at 10:21 pm, Lopezscu said:

    correct answer is
    :%s/{Ctrl+V}{Ctrl+M}//{Enter}

    where {Ctrl+V }{Ctrl+M} stands for, you must press Ctrl+V and then Ctrl+M

    Reply

    • April 29, 2011 at 1:00 pm, Piyush said:

      It worked .. thx
      I also replaced ‘^@’ with the same technique.

      Reply

    • May 26, 2011 at 2:47 pm, Louispendis5 said:

      you’re the man!

      Reply

  36. August 10, 2010 at 4:35 am, Leena said:

    Hi,

    Remove the last line from a txt file.

    I have txt file with data like
    1235 5678
    3455 5678

    and after 2nd line I pressed the enter. I tried to remove that enter in using many unix commands. but in unix I think it doesnt consider it as new line as when I check wc -l it gives me 2 but when I send this file to my C function it is considered as 3rd line.
    Please telme the work around to remove the last line.

    Thanks,
    Leena

    Reply

    • September 24, 2010 at 7:12 pm, Nitin said:

      I know that ^M comes when we transfer a file from windows machine to a Unix machine from Telnet.

      At first place what is the harm in having ^M characters? Is it going to cause any kind of issues by letting them just around? Just curious to know.

      Reply

  37. October 14, 2010 at 5:06 am, Yashaswikumar said:

    Thank u so very much.. This really worked :) Saved my day :)

    Reply

  38. October 22, 2010 at 5:11 am, Suchi said:

    Thanks! The string re[place hadnt been working, but col -b worked for me :)

    Reply

  39. November 02, 2010 at 9:50 am, Ramakrishna Aazad said:

    vi file name

    :%s[ctrl v][ctrl[m]//g worked for me to remove the cntrl m characters from the file. Thanks a lot :)

    Reply

  40. November 25, 2010 at 12:44 pm, Grit said:

    u meatheads who posted dos2unix etc….the heading is “Remove ^M characters at end of lines in vi” not oustide of…….dummies

    Reply

  41. December 08, 2010 at 1:43 am, Team_Awesome said:

    nothing else people posted on here worked. Thank you tons!!!

    Reply

  42. December 24, 2010 at 7:31 am, Vaibhav said:

    perfect solution

    Reply

  43. January 03, 2011 at 11:44 pm, Bob said:

    only a dummy does not know you can invoke any command, dos2unix etc…from within vi …idiot

    Reply

  44. January 19, 2011 at 9:56 am, Vinoo Usa said:

    Thank you so mcuh lopezzzzzzzzzz……….. you are my man… :)

    Reply

  45. January 28, 2011 at 10:29 pm, S_zarembo said:

    the original post works just fine; you just need to read it carefully.
    Thanks

    Reply

  46. February 04, 2011 at 9:29 pm, Guest said:

    col -bx worked for me…

    Reply

  47. March 09, 2011 at 6:43 pm, PR said:

    This helps

    Reply

  48. March 29, 2011 at 1:45 pm, BMQ said:

    I found dos2unix run against mysql slow log caused log to stop accruing entries. This forced me to restart mysql to restore slow log entries.

    Reply

  49. July 08, 2011 at 6:18 pm, Petalmelissa said:

    Your attitude is unnecessary and not conducive to the sharing environment being attempted to be created here.. 

    Reply

  50. July 21, 2011 at 4:06 pm, eskay said:

    It didn’t worked. I got an error as Pattern Not Found: ^M

    All I did is,
    vi filename –> esc –> :%s/^v^m//g –> enter

    Reply

    • July 25, 2011 at 1:49 pm, ataraxic said:

      the same for me –> Pattern Not Found: ^M
      What is the problem here???

      Reply

  51. August 03, 2011 at 11:30 am, nit said:

    Thanks Lopezscu ,sed version of ur examplecat foo | sed ‘s/^M//g’the “cat foo | col -b > foo2″ version in below posts is not working for me

    Reply

  52. October 14, 2011 at 10:51 am, Cesar said:

    To everyone having the “Pattern not found” message:

    You MUST NOT WRITE ^V ^M

    ^V is: “press control, press V, release V, release control.”
    The same with ^M: “press control, press M, release M, release control.”

    Sure it will work as sure as you are using vi/vim.

    Reply

  53. November 01, 2011 at 9:59 am, george said:

    Thanks very much.

    Reply

  54. November 23, 2011 at 5:26 pm, jason said:

    Ok,

    For those where :%s/^M//g was not working for them, I have a workaround for you.

    I was finding it wasn’t doing the replace, however when I compiled the file I was working on, it was complaining about the ^M which was really annoying.

    I can confirm that the following though will work.

    cat fileName | sed s/^M//g >tmp

    Just move the tmp file back to fileName and you have a file minus the ^M.

    Reply

  55. November 28, 2011 at 8:39 pm, Narsimulu said:

    To everyone having the “Pattern not found” message:

    Please type control character twice and then type V and M.

    “press control, press control, press V, press control,press control, press M”

    -Narsimulu

    Reply

  56. December 16, 2011 at 7:03 am, Tamil said:

    Yes, It works fine, I also had the “Pattern Not Found” issue. Please press the following keys to have this function.

    1. Type %s
    2. press (ctrl + v) = it will give the (^) symbol
    3. Press (ctrl + m) = it will give the ^M in blue color.

    then add rest of the symbols. It will work.

    Reply

  57. December 20, 2011 at 2:05 pm, mooselix said:

    Uh, all you need is this. No control-characters, etc.

    :%s/\r//g

    Reply

  58. January 04, 2012 at 2:58 am, raj malhotra said:

    Thanks Lopezscu
    yours worked and it saved time greatly :)

    Reply

  59. January 11, 2012 at 9:03 am, #dhk said:

    Its works
    :%s/[^0-9]$//g

    Reply

  60. January 19, 2012 at 10:42 pm, Manu said:

    Hi Everyone,

    I am new to vi. I tried removing the ^M character from the file using

    sed s/\r//g

    but it didn’t work for me. I tried using (ctrl+M) instead of \r but that is also not working. whenever I press Ctrl+M it is taking me to the next line.

    I have written down my code like below:
    #!/bin/sh

    while read line
    do
    echo “$line” | sed s/[^0-9]//g >> temp
    done

    Reply

  61. February 09, 2012 at 7:20 am, rufwork said:

    For Vishal and others who experience “I am trying t0 get the diff of a file into a temp.txt file. When I do :%s/^M//g in my file, it says “Pattern not Found: ^M”.”…

    You’re obviously, well, not on UNIX. The short answer is that you want to use Ctrl-Q in place of Ctrl-V in this recipe if you’re getting the error.

    The long answer is here:

    http://stackoverflow.com/questions/426896/vim-ctrl-v-conflict-with-windows-paste

    Profit.

    Reply

  62. June 08, 2012 at 2:29 am, Niv said:

    Thank you. It was very helpful

    Reply

  63. July 09, 2012 at 3:05 pm, divya said:

    :%s/^M//g –Dint work :(

    Reply

  64. August 17, 2012 at 3:08 am, Chai said:

    Thanks Quinn. You just save my life.

    Reply

  65. August 23, 2012 at 12:45 pm, Dinesh Kumar Selvam said:

    Just Try this

    #strings file.name > newfile.name

    ^M will vanish….

    Reply

  66. September 04, 2012 at 12:24 am, tiebob said:

    Mm…. I opened file with “nano”, and save it again. The ^M character disappear….

    Reply

  67. October 16, 2012 at 1:47 pm, Piyush said:

    Thanks Dinesh Kumar Selvam strings file.name > newfile.name works nice ;)

    Reply

  68. January 14, 2013 at 9:39 am, Nicolas Joyard said:

    Thanks for this article, but the last paragraph is a bit wrong.

    The “%” at the beginning of the command tells vi to do the substitution on every line; the “g” at the end tells it to substitute every occurence, on each affected line ; without it vi replaces only the first occurence on each line. “g” is actually not necessary here as you should only have one ^M per line.

    Reply

  69. March 13, 2013 at 3:48 am, anjana said:

    tr -d ‘\r’ $targetDir

    Reply

  70. April 01, 2013 at 1:09 am, Narendra Kumar Achari said:

    Use :e ++ff=dos for vim to remove ^M charecters

    First, use :set ff? to figure out the file format your file is.

    Reply

  71. February 24, 2014 at 3:43 am, Shailesh Bohra said:

    I have a data in file temp_source1.dat containing MT-0040GB14 in the starting. But when is use sed s/^M//g temp_source1.dat > temp_source_data.dat, MT-0040GB14 is gets populated in the file temp_source_data.dat.
    Is there any solution?

    Reply

    • March 01, 2014 at 10:23 am, Shailesh Bohra said:

      > Finally after investigating, i found that to remove control characters, we should use sed s/ctrl+v+m//g which will appear as sed s/^M//g, where ^M is a single character rather than ‘^’ and ‘M’ as inividual characters.

      :)

      Reply

Leave a Reply