Remove ^M characters at end of lines in vi

UNIX treats the end of line differently than other operating systems. Sometimes when editing files in both Windows and UNIX environments, a CTRL-M character is visibly displayed at the end of each line as ^M in vi.


To remove the ^M characters at the end of all lines in vi, use:

:%s/^V^M//g

The ^v is a CONTROL-V character and ^m is a CONTROL-M. When you type this, it will look like this:

:%s/^M//g

In UNIX, you can escape a control character by preceeding it with a CONTROL-V. The :%s is a basic search and replace command in vi. It tells vi to replace the regular expression between the first and second slashes (^M) with the text between the second and third slashes (nothing in this case). The g at the end directs vi to search and replace globally (all occurrences).

 

About Quinn McHenry

Quinn was one of the original co-founders of Tech-Recipes. He is currently crafting iOS applications as a senior developer at Small Planet Digital in Brooklyn, New York.
View more articles by Quinn McHenry

The Conversation

Follow the reactions below and share your own thoughts.

  • Phobian

    You could try using sed, the stream editor – it works on similar principles to vi’s replace tool, but you can pipe into and out of it.

    So, to change one file:
    cat fileName | sed s///g >tmp && mv tmp fileName

    Do NOT try to read from and write to the same file in one pipe, or you’ll blank the file

    To change all files in a directory with the .sql ending, run the following script whilst inside that directory:
    #/bin/ksh
    for fileName in $(ls *.sql); do
    cat $fileName | sed s/^M//g >tmp && mv tmp $fileName
    done

    (bear in mind to enter ^M you need to enter as above.

    Hope that helps people
    -phobiandarkmoon

  • Maxlen

    type the following command in vi

    :%s/.$//

    • rohit

      > Awesome it works :)

    • Arbab

      This worked for me

    • kan

      Thanks Man… It worked…. Your knowledge helped me…..>

    • Mohamed

      > this one works thanks

    • Mike

      > Keep in mind, if you happen to have edited the file and put some normal newlines in and then use this, you will loose a character off the end of every line. Which is why the :%s/^V^M// is a better way to go.

      • yspatro

        It worked !! thx much

      • Sachin

        > +1. It removes the last character of every line, irrespective of if its the ^M or anything else.

  • Yo Whirrd up

    Try perl -pi -e’tr/15//d’

    • SOM

      Great dude .. this works

  • pradeep

    yes the “:%s/^ v ^M//g” is working… thank you very much…

  • soma

    this solution is not working for ubuntu..:(

    • rseal

      :%s/^q^M//

    • rseal

      Use CTRL-q CTRL-m

      • varsha

        > the solution is not working on AIX box.

  • Reetesh

    Thanks it is working fine

  • http://www.h1bwage.com thanks

    thank you for the nice tips. I have plenty of ctrl+M symbols in the dump txt files from dol.

  • Neo

    great info.. thanks

  • 3pe

    another way to get rid of those ^M’s
    :%s/r//g
    btw other systems then *nix are treating newlines differently :P

  • Ashish

    :%s/r/r/g

  • a

    this works ! thanks.

  • xx

    use $ to replace only at the end
    :%s@^M$@@g

  • roy
  • sam

    Excellent

  • S

    thanks it works well

  • http://freebsdhowto.com/ Ciprian

    Lovely! I completely forgot about dos2unix and sincerely I like to use vim better if I can :)

  • Vishal

    I am trying ti get the diff of a file into a temp.txt file.
    When I do :%s/^M//g in my file, it says “Pattern not Found: ^M”.
    But when I see the temp.txt file for the diff it shows ^M on all the lines.
    Help me out for this, I dont want ^M in temp.txt file

  • Anonymous

    Thank you very much…for me saved lot of time.

  • http://slhosting.net.au/ SnowLeopard

    Thanks! :D

  • awais

    hi,
    grt man. it worked for me. keep up the good work.

    br

  • http://twitter.com/scodi Dmitriy Golub

    Nice, thank you!

  • Anonymous

    Thank you, original post did not work for me, but this did.

  • mhannesy

    Just curious, :%s/r//g works but :%s/rn/n/g does not, why is that?

    • mhannesy

      Ok, I found out why I ended up with the NUL control characters instead. From :help insert:

      “If you enter a value of 10, it will end up in the file as a 0. The 10 is a
      , which is used internally to represent the character. When writing
      the buffer to a file, the character is translated into . The
      character is written at the end of each line. Thus if you want to insert a
      character in a file you will have to make a line break.”

  • Brett

    You Sir are a genius. I award you five internets!

  • Hedi

    You could try :
    echo file | sed ‘s/r//g’

  • Name

    I am brand new to Vim. I am trying to remove ^M and replace it with a tab (actually replace it with spaces). I can’t seem to get this to work. I’ve done the following things and it isn’t working. Any advice?

    :set expandtab
    :%s/^M//g

    All it is doing is putting “” with the “^M” used to be. Any way to actually make the spaces appear? Thanks for your help.

    • Name

      Nevermind! I figured it out. I simply had to push “control + the tab key” and that resulted in what I needed. Thanks.

  • Anonymous

    hi,
    Thanks for the info.. I need to move the lines to the top after the ctrlM charecter. Is this possible ??

  • Anonymous

    Hi ,

    I did worked for me as well . Thanks guys for your help .

    Arif

  • Anonymous

    too good. this worked for me!

  • Anonymous

    Hi,

    Thanks for the solution. It worked.

    I also noticed other characters like and when using vi. I assume these are also converted because of Windows.

    How can I revert these back?

    Thanks,
    Allan

  • p1

    Can anyone please help in removing character displayed in vi ?

  • Anonymous

    Hi,

    In my case, when I transfer my file from windows to unix, this control+m (^m) character gets added. I there any utility where I can do some changes in file at windows end so that ^M doesn’t get added to it when I transfer it to Unix?

    Thanks,
    Jatin Kapoor

  • MiDSuMMeR

    Type the following command:

    :%s/rn/r/g

    It works fine!

  • Bugged123

    This worked..Thank you ..Original did not ..

    • Lopezscu

      same for me, the post author is a retard…

      • lota

        More likely you don’t know how to use vi.

      • Tallis

        only tools cant use tools properly

    • Malice

      The original post is fine, you just have to be able to read.

  • Lopezscu

    correct answer is
    :%s/{Ctrl+V}{Ctrl+M}//{Enter}

    where {Ctrl+V }{Ctrl+M} stands for, you must press Ctrl+V and then Ctrl+M

    • Piyush

      It worked .. thx
      I also replaced ‘^@’ with the same technique.

    • Louispendis5

      you’re the man!

  • Leena

    Hi,

    Remove the last line from a txt file.

    I have txt file with data like
    1235 5678
    3455 5678

    and after 2nd line I pressed the enter. I tried to remove that enter in using many unix commands. but in unix I think it doesnt consider it as new line as when I check wc -l it gives me 2 but when I send this file to my C function it is considered as 3rd line.
    Please telme the work around to remove the last line.

    Thanks,
    Leena

    • Nitin

      I know that ^M comes when we transfer a file from windows machine to a Unix machine from Telnet.

      At first place what is the harm in having ^M characters? Is it going to cause any kind of issues by letting them just around? Just curious to know.

  • Yashaswikumar

    Thank u so very much.. This really worked :) Saved my day :)

  • Suchi

    Thanks! The string re[place hadnt been working, but col -b worked for me :)

  • Ramakrishna Aazad

    vi file name

    :%s[ctrl v][ctrl[m]//g worked for me to remove the cntrl m characters from the file. Thanks a lot :)

  • Grit

    u meatheads who posted dos2unix etc….the heading is “Remove ^M characters at end of lines in vi” not oustide of…….dummies

  • Team_Awesome

    nothing else people posted on here worked. Thank you tons!!!

  • Vaibhav

    perfect solution

  • Bob

    only a dummy does not know you can invoke any command, dos2unix etc…from within vi …idiot

  • Vinoo Usa

    Thank you so mcuh lopezzzzzzzzzz……….. you are my man… :)

  • S_zarembo

    the original post works just fine; you just need to read it carefully.
    Thanks

  • Guest

    col -bx worked for me…

  • PR

    This helps

  • BMQ

    I found dos2unix run against mysql slow log caused log to stop accruing entries. This forced me to restart mysql to restore slow log entries.

  • Petalmelissa

    Your attitude is unnecessary and not conducive to the sharing environment being attempted to be created here.. 

  • eskay

    It didn’t worked. I got an error as Pattern Not Found: ^M

    All I did is,
    vi filename –> esc –> :%s/^v^m//g –> enter

    • ataraxic

      the same for me –> Pattern Not Found: ^M
      What is the problem here???

  • nit

    Thanks Lopezscu ,sed version of ur examplecat foo | sed ‘s/^M//g’the “cat foo | col -b > foo2″ version in below posts is not working for me

  • Cesar

    To everyone having the “Pattern not found” message:

    You MUST NOT WRITE ^V ^M

    ^V is: “press control, press V, release V, release control.”
    The same with ^M: “press control, press M, release M, release control.”

    Sure it will work as sure as you are using vi/vim.

  • george

    Thanks very much.

  • jason

    Ok,

    For those where :%s/^M//g was not working for them, I have a workaround for you.

    I was finding it wasn’t doing the replace, however when I compiled the file I was working on, it was complaining about the ^M which was really annoying.

    I can confirm that the following though will work.

    cat fileName | sed s/^M//g >tmp

    Just move the tmp file back to fileName and you have a file minus the ^M.

  • Narsimulu

    To everyone having the “Pattern not found” message:

    Please type control character twice and then type V and M.

    “press control, press control, press V, press control,press control, press M”

    -Narsimulu

  • Tamil

    Yes, It works fine, I also had the “Pattern Not Found” issue. Please press the following keys to have this function.

    1. Type %s
    2. press (ctrl + v) = it will give the (^) symbol
    3. Press (ctrl + m) = it will give the ^M in blue color.

    then add rest of the symbols. It will work.

  • mooselix

    Uh, all you need is this. No control-characters, etc.

    :%s/\r//g

  • raj malhotra

    Thanks Lopezscu
    yours worked and it saved time greatly :)

  • #dhk

    Its works
    :%s/[^0-9]$//g

  • Manu

    Hi Everyone,

    I am new to vi. I tried removing the ^M character from the file using

    sed s/\r//g

    but it didn’t work for me. I tried using (ctrl+M) instead of \r but that is also not working. whenever I press Ctrl+M it is taking me to the next line.

    I have written down my code like below:
    #!/bin/sh

    while read line
    do
    echo “$line” | sed s/[^0-9]//g >> temp
    done

  • http://myfreakinname.blogspot.com rufwork

    For Vishal and others who experience “I am trying t0 get the diff of a file into a temp.txt file. When I do :%s/^M//g in my file, it says “Pattern not Found: ^M”.”…

    You’re obviously, well, not on UNIX. The short answer is that you want to use Ctrl-Q in place of Ctrl-V in this recipe if you’re getting the error.

    The long answer is here:

    http://stackoverflow.com/questions/426896/vim-ctrl-v-conflict-with-windows-paste

    Profit.

  • Niv

    Thank you. It was very helpful

  • divya

    :%s/^M//g –Dint work :(

  • Chai

    Thanks Quinn. You just save my life.

  • Dinesh Kumar Selvam

    Just Try this

    #strings file.name > newfile.name

    ^M will vanish….

  • tiebob

    Mm…. I opened file with “nano”, and save it again. The ^M character disappear….

  • Piyush

    Thanks Dinesh Kumar Selvam strings file.name > newfile.name works nice ;)

  • Nicolas Joyard

    Thanks for this article, but the last paragraph is a bit wrong.

    The “%” at the beginning of the command tells vi to do the substitution on every line; the “g” at the end tells it to substitute every occurence, on each affected line ; without it vi replaces only the first occurence on each line. “g” is actually not necessary here as you should only have one ^M per line.

  • anjana

    tr -d ‘\r’ $targetDir

  • Narendra Kumar Achari

    Use :e ++ff=dos for vim to remove ^M charecters

    First, use :set ff? to figure out the file format your file is.

  • Shailesh Bohra

    I have a data in file temp_source1.dat containing MT-0040GB14 in the starting. But when is use sed s/^M//g temp_source1.dat > temp_source_data.dat, MT-0040GB14 is gets populated in the file temp_source_data.dat.
    Is there any solution?

    • Shailesh Bohra

      > Finally after investigating, i found that to remove control characters, we should use sed s/ctrl+v+m//g which will appear as sed s/^M//g, where ^M is a single character rather than ‘^’ and ‘M’ as inividual characters.

      :)