Remove ^M characters at end of lines in vi
Posted by Quinn McHenry in UNIX
UNIX treats the end of line differently than other operating systems. Sometimes when editing files in both Windows and UNIX environments, a CTRL-M character is visibly displayed at the end of each line as ^M in vi.
To remove the ^M characters at the end of all lines in vi, use:
:%s/^V^M//g
The ^v is a CONTROL-V character and ^m is a CONTROL-M. When you type this, it will look like this:
:%s/^M//g
In UNIX, you can escape a control character by preceeding it with a CONTROL-V. The :%s is a basic search and replace command in vi. It tells vi to replace the regular expression between the first and second slashes (^M) with the text between the second and third slashes (nothing in this case). The g at the end directs vi to search and replace globally (all occurrences).
About Quinn McHenry
Quinn was one of the original co-founders of Tech-Recipes. He is currently crafting iOS applications as a senior developer at Small Planet Digital in Brooklyn, New York.
View more articles by Quinn McHenry
View more articles by Quinn McHenry
The Conversation
Follow the reactions below and share your own thoughts.
October 24, 2008 at 8:57 am, Phobian said:
You could try using sed, the stream editor – it works on similar principles to vi’s replace tool, but you can pipe into and out of it.
So, to change one file:
cat fileName | sed s///g >tmp && mv tmp fileName
Do NOT try to read from and write to the same file in one pipe, or you’ll blank the file
To change all files in a directory with the .sql ending, run the following script whilst inside that directory:
#/bin/ksh
for fileName in $(ls *.sql); do
cat $fileName | sed s/^M//g >tmp && mv tmp $fileName
done
(bear in mind to enter ^M you need to enter as above.
Hope that helps people
-phobiandarkmoon
November 06, 2008 at 9:11 pm, Maxlen said:
type the following command in vi
:%s/.$//
November 07, 2008 at 4:48 am, Yo Whirrd up said:
Try perl -pi -e’tr/15//d’
July 06, 2009 at 4:19 am, SOM said:
Great dude .. this works
December 01, 2008 at 11:37 am, pradeep said:
yes the “:%s/^ v ^M//g” is working… thank you very much…
December 11, 2008 at 7:25 am, soma said:
this solution is not working for ubuntu..:(
December 25, 2008 at 2:52 am, Reetesh said:
Thanks it is working fine
January 14, 2009 at 6:21 pm, thanks said:
thank you for the nice tips. I have plenty of ctrl+M symbols in the dump txt files from dol.
February 12, 2009 at 7:42 am, Neo said:
great info.. thanks
February 24, 2009 at 10:59 am, 3pe said:
another way to get rid of those ^M’s
:%s/r//g
btw other systems then *nix are treating newlines differently
April 28, 2009 at 11:14 am, Ashish said:
:%s/r/r/g
April 29, 2009 at 8:52 am, a said:
this works ! thanks.
May 05, 2009 at 10:14 am, xx said:
use $ to replace only at the end
:%s@^M$@@g
May 05, 2009 at 5:29 pm, roy said:
You are a genius
roy@roy.com
May 31, 2009 at 8:20 pm, sam said:
Excellent
June 01, 2009 at 3:56 am, S said:
thanks it works well
July 06, 2009 at 6:32 am, Ciprian said:
Lovely! I completely forgot about dos2unix and sincerely I like to use vim better if I can
July 07, 2009 at 7:12 am, Vishal said:
I am trying ti get the diff of a file into a temp.txt file.
When I do :%s/^M//g in my file, it says “Pattern not Found: ^M”.
But when I see the temp.txt file for the diff it shows ^M on all the lines.
Help me out for this, I dont want ^M in temp.txt file
July 23, 2009 at 2:51 am, Anonymous said:
Thank you very much…for me saved lot of time.
August 30, 2009 at 1:19 am, SnowLeopard said:
Thanks!
September 01, 2009 at 11:25 am, awais said:
hi,
grt man. it worked for me. keep up the good work.
br
October 01, 2009 at 1:36 pm, Dmitriy Golub said:
Nice, thank you!
October 01, 2009 at 8:24 pm, Anonymous said:
Thank you, original post did not work for me, but this did.
October 21, 2009 at 8:06 pm, mhannesy said:
Just curious, :%s/r//g works but :%s/rn/n/g does not, why is that?
November 21, 2009 at 6:04 pm, mhannesy said:
Ok, I found out why I ended up with the NUL control characters instead. From :help insert:
“If you enter a value of 10, it will end up in the file as a 0. The 10 is a
, which is used internally to represent the character. When writing
the buffer to a file, the character is translated into . The
character is written at the end of each line. Thus if you want to insert a
character in a file you will have to make a line break.”
October 29, 2009 at 5:24 am, Brett said:
You Sir are a genius. I award you five internets!
November 11, 2009 at 6:24 pm, Hedi said:
You could try :
echo file | sed ‘s/r//g’
December 04, 2009 at 5:31 pm, Name said:
I am brand new to Vim. I am trying to remove ^M and replace it with a tab (actually replace it with spaces). I can’t seem to get this to work. I’ve done the following things and it isn’t working. Any advice?
:set expandtab
:%s/^M//g
All it is doing is putting “” with the “^M” used to be. Any way to actually make the spaces appear? Thanks for your help.
December 05, 2009 at 3:33 am, Name said:
Nevermind! I figured it out. I simply had to push “control + the tab key” and that resulted in what I needed. Thanks.
December 16, 2009 at 4:46 pm, Anonymous said:
hi,
Thanks for the info.. I need to move the lines to the top after the ctrlM charecter. Is this possible ??
February 04, 2010 at 11:03 am, Anonymous said:
Hi ,
I did worked for me as well . Thanks guys for your help .
Arif
February 08, 2010 at 7:17 am, Anonymous said:
too good. this worked for me!
March 28, 2010 at 5:34 am, Anonymous said:
Hi,
Thanks for the solution. It worked.
I also noticed other characters like and when using vi. I assume these are also converted because of Windows.
How can I revert these back?
Thanks,
Allan
March 30, 2010 at 6:45 am, p1 said:
Can anyone please help in removing character displayed in vi ?
April 08, 2010 at 11:31 pm, Anonymous said:
Hi,
In my case, when I transfer my file from windows to unix, this control+m (^m) character gets added. I there any utility where I can do some changes in file at windows end so that ^M doesn’t get added to it when I transfer it to Unix?
Thanks,
Jatin Kapoor
May 13, 2010 at 11:35 am, MiDSuMMeR said:
Type the following command:
:%s/rn/r/g
It works fine!
June 21, 2010 at 8:44 pm, Bugged123 said:
This worked..Thank you ..Original did not ..
June 22, 2010 at 10:16 pm, Lopezscu said:
same for me, the post author is a retard…
July 12, 2010 at 12:45 am, lota said:
More likely you don’t know how to use vi.
October 28, 2010 at 3:05 pm, Tallis said:
only tools cant use tools properly
November 08, 2010 at 9:45 pm, Malice said:
The original post is fine, you just have to be able to read.
June 22, 2010 at 10:21 pm, Lopezscu said:
correct answer is
:%s/{Ctrl+V}{Ctrl+M}//{Enter}
where {Ctrl+V }{Ctrl+M} stands for, you must press Ctrl+V and then Ctrl+M
April 29, 2011 at 1:00 pm, Piyush said:
It worked .. thx
I also replaced ‘^@’ with the same technique.
May 26, 2011 at 2:47 pm, Louispendis5 said:
you’re the man!
August 10, 2010 at 4:35 am, Leena said:
Hi,
Remove the last line from a txt file.
I have txt file with data like
1235 5678
3455 5678
and after 2nd line I pressed the enter. I tried to remove that enter in using many unix commands. but in unix I think it doesnt consider it as new line as when I check wc -l it gives me 2 but when I send this file to my C function it is considered as 3rd line.
Please telme the work around to remove the last line.
Thanks,
Leena
September 24, 2010 at 7:12 pm, Nitin said:
I know that ^M comes when we transfer a file from windows machine to a Unix machine from Telnet.
At first place what is the harm in having ^M characters? Is it going to cause any kind of issues by letting them just around? Just curious to know.
October 14, 2010 at 5:06 am, Yashaswikumar said:
Thank u so very much.. This really worked
Saved my day 
October 22, 2010 at 5:11 am, Suchi said:
Thanks! The string re[place hadnt been working, but col -b worked for me
November 02, 2010 at 9:50 am, Ramakrishna Aazad said:
vi file name
:%s[ctrl v][ctrl[m]//g worked for me to remove the cntrl m characters from the file. Thanks a lot
November 25, 2010 at 12:44 pm, Grit said:
u meatheads who posted dos2unix etc….the heading is “Remove ^M characters at end of lines in vi” not oustide of…….dummies
December 08, 2010 at 1:43 am, Team_Awesome said:
nothing else people posted on here worked. Thank you tons!!!
December 24, 2010 at 7:31 am, Vaibhav said:
perfect solution
January 03, 2011 at 11:44 pm, Bob said:
only a dummy does not know you can invoke any command, dos2unix etc…from within vi …idiot
January 19, 2011 at 9:56 am, Vinoo Usa said:
Thank you so mcuh lopezzzzzzzzzz……….. you are my man…
January 28, 2011 at 10:29 pm, S_zarembo said:
the original post works just fine; you just need to read it carefully.
Thanks
February 04, 2011 at 9:29 pm, Guest said:
col -bx worked for me…
March 09, 2011 at 6:43 pm, PR said:
This helps
March 29, 2011 at 1:45 pm, BMQ said:
I found dos2unix run against mysql slow log caused log to stop accruing entries. This forced me to restart mysql to restore slow log entries.
July 08, 2011 at 6:18 pm, Petalmelissa said:
Your attitude is unnecessary and not conducive to the sharing environment being attempted to be created here..
July 21, 2011 at 4:06 pm, eskay said:
It didn’t worked. I got an error as Pattern Not Found: ^M
All I did is,
vi filename –> esc –> :%s/^v^m//g –> enter
July 25, 2011 at 1:49 pm, ataraxic said:
the same for me –> Pattern Not Found: ^M
What is the problem here???
August 03, 2011 at 11:30 am, nit said:
Thanks Lopezscu ,sed version of ur examplecat foo | sed ‘s/^M//g’the “cat foo | col -b > foo2″ version in below posts is not working for me
October 14, 2011 at 10:51 am, Cesar said:
To everyone having the “Pattern not found” message:
You MUST NOT WRITE ^V ^M
^V is: “press control, press V, release V, release control.”
The same with ^M: “press control, press M, release M, release control.”
Sure it will work as sure as you are using vi/vim.
November 01, 2011 at 9:59 am, george said:
Thanks very much.
November 23, 2011 at 5:26 pm, jason said:
Ok,
For those where :%s/^M//g was not working for them, I have a workaround for you.
I was finding it wasn’t doing the replace, however when I compiled the file I was working on, it was complaining about the ^M which was really annoying.
I can confirm that the following though will work.
cat fileName | sed s/^M//g >tmp
Just move the tmp file back to fileName and you have a file minus the ^M.
November 28, 2011 at 8:39 pm, Narsimulu said:
To everyone having the “Pattern not found” message:
Please type control character twice and then type V and M.
“press control, press control, press V, press control,press control, press M”
-Narsimulu
December 16, 2011 at 7:03 am, Tamil said:
Yes, It works fine, I also had the “Pattern Not Found” issue. Please press the following keys to have this function.
1. Type %s
2. press (ctrl + v) = it will give the (^) symbol
3. Press (ctrl + m) = it will give the ^M in blue color.
then add rest of the symbols. It will work.
December 20, 2011 at 2:05 pm, mooselix said:
Uh, all you need is this. No control-characters, etc.
:%s/\r//g
January 04, 2012 at 2:58 am, raj malhotra said:
Thanks Lopezscu
yours worked and it saved time greatly
January 11, 2012 at 9:03 am, #dhk said:
Its works
:%s/[^0-9]$//g
January 19, 2012 at 10:42 pm, Manu said:
Hi Everyone,
I am new to vi. I tried removing the ^M character from the file using
sed s/\r//g
but it didn’t work for me. I tried using (ctrl+M) instead of \r but that is also not working. whenever I press Ctrl+M it is taking me to the next line.
I have written down my code like below:
#!/bin/sh
while read line
do
echo “$line” | sed s/[^0-9]//g >> temp
done
February 09, 2012 at 7:20 am, rufwork said:
For Vishal and others who experience “I am trying t0 get the diff of a file into a temp.txt file. When I do :%s/^M//g in my file, it says “Pattern not Found: ^M”.”…
You’re obviously, well, not on UNIX. The short answer is that you want to use Ctrl-Q in place of Ctrl-V in this recipe if you’re getting the error.
The long answer is here:
http://stackoverflow.com/questions/426896/vim-ctrl-v-conflict-with-windows-paste
Profit.