Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > windows one liner to output unix line feed

Reply
Thread Tools

windows one liner to output unix line feed

 
 
boman
Guest
Posts: n/a
 
      08-19-2009
I have a simple one liner running on Windows that does a substitution.
However, with the -p option, the line endings are coming out \r\n, and
I need them to be just \n.

I searched for the -l option, but couldn't find how to specify unix
line breaks on output

Here's the code:

perl -pi.bak -e "s|foo|bar|g" myfile.txt

thanks
 
Reply With Quote
 
 
 
 
Brian Wakem
Guest
Posts: n/a
 
      08-22-2009
David Harmon wrote:

> On 19 Aug 2009 20:25:50 GMT in comp.lang.perl.misc, Glenn Jackman
> <(E-Mail Removed)> wrote,
>>At 2009-08-19 01:28PM, "boman" wrote:
>>> I have a simple one liner running on Windows that does a substitution.
>>> However, with the -p option, the line endings are coming out \r\n, and
>>> I need them to be just \n.
>>>
>>> perl -pi.bak -e "s|foo|bar|g" myfile.txt

>>
>>perhaps you need to chomp the windows line ending, and add the unix line
>>ending manually:
>>
>> perl -pi.bak -e "s|foo|bar|g; chomp; print qq{$_\n}" myfile.txt

>
> How could that work when the output stream converts \n to the
> system-defined line ending?



It wouldn't.

Using \x0A instead of \n should work.


--
Brian Wakem
 
Reply With Quote
 
 
 
 
Keith Thompson
Guest
Posts: n/a
 
      08-22-2009
Brian Wakem <(E-Mail Removed)> writes:
> David Harmon wrote:
>
>> On 19 Aug 2009 20:25:50 GMT in comp.lang.perl.misc, Glenn Jackman
>> <(E-Mail Removed)> wrote,
>>>At 2009-08-19 01:28PM, "boman" wrote:
>>>> I have a simple one liner running on Windows that does a substitution.
>>>> However, with the -p option, the line endings are coming out \r\n, and
>>>> I need them to be just \n.
>>>>
>>>> perl -pi.bak -e "s|foo|bar|g" myfile.txt
>>>
>>>perhaps you need to chomp the windows line ending, and add the unix line
>>>ending manually:
>>>
>>> perl -pi.bak -e "s|foo|bar|g; chomp; print qq{$_\n}" myfile.txt

>>
>> How could that work when the output stream converts \n to the
>> system-defined line ending?

>
>
> It wouldn't.
>
> Using \x0A instead of \n should work.


That seems unlikely, since "\x0a" and "\n" have exactly the same
value.

--
Keith Thompson (The_Other_Keith) http://www.velocityreviews.com/forums/(E-Mail Removed) <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
 
Reply With Quote
 
Scott Bryce
Guest
Posts: n/a
 
      08-23-2009
Keith Thompson wrote:
> Brian Wakem <(E-Mail Removed)> writes:
>> Using \x0A instead of \n should work.

>
> That seems unlikely, since "\x0a" and "\n" have exactly the same
> value.



Not on a Windows system.
 
Reply With Quote
 
sln@netherlands.com
Guest
Posts: n/a
 
      08-23-2009
On Sat, 22 Aug 2009 14:54:06 -0700, Keith Thompson <(E-Mail Removed)> wrote:

>Brian Wakem <(E-Mail Removed)> writes:
>> David Harmon wrote:
>>
>>> On 19 Aug 2009 20:25:50 GMT in comp.lang.perl.misc, Glenn Jackman
>>> <(E-Mail Removed)> wrote,
>>>>At 2009-08-19 01:28PM, "boman" wrote:
>>>>> I have a simple one liner running on Windows that does a substitution.
>>>>> However, with the -p option, the line endings are coming out \r\n, and
>>>>> I need them to be just \n.
>>>>>
>>>>> perl -pi.bak -e "s|foo|bar|g" myfile.txt
>>>>
>>>>perhaps you need to chomp the windows line ending, and add the unix line
>>>>ending manually:
>>>>
>>>> perl -pi.bak -e "s|foo|bar|g; chomp; print qq{$_\n}" myfile.txt
>>>
>>> How could that work when the output stream converts \n to the
>>> system-defined line ending?

>>
>>
>> It wouldn't.
>>
>> Using \x0A instead of \n should work.

>
>That seems unlikely, since "\x0a" and "\n" have exactly the same
>value.


I don't know how unix deals with CR. Apparently it just does LF's
on output instead of CRLF.
It really doesen't matter what the :crlf layer does in unix.
It only matters how the console device interprets CR's as a tty.
You can have a thousand CR's and one LF before you print and it will
still print on the next line (Mac may be different).

If his file eol is all CR's ala Mac, then he opened it without an eol
and processed the whole file as one line. Otherwise, a series of
text embedded with just CR's on a console where a CR is a control character,
will result in overwrites of the lines without LF's.

If I had a unix machine I could try it out. It seems it would be a major
fopah if Perl didn't get the basic unix console correct, lol.

binmode (STDOUT, ":raw");
print " \nthis \x0a is \x0d\x0d\x0d\x0d\x0a a \x0a test\x0d\x0d\x0a";
print " \nthis \x0a\x0d is \x0d\x0d\x0d\x0d\x0a\x0d a \x0a test\x0d\x0d\x0a";
print " \nthis \x0d is \x0d\x0d\x0d\x0d\x0d a \x0d test\x0d\x0d\x0d";

-sln
 
Reply With Quote
 
sln@netherlands.com
Guest
Posts: n/a
 
      08-23-2009
On Sat, 22 Aug 2009 18:08:20 -0600, Scott Bryce <(E-Mail Removed)> wrote:

>Keith Thompson wrote:
>> Brian Wakem <(E-Mail Removed)> writes:
>>> Using \x0A instead of \n should work.

>>
>> That seems unlikely, since "\x0a" and "\n" have exactly the same
>> value.

>
>
>Not on a Windows system.


Sure they do. Exactly, identically the same.
-sln

 
Reply With Quote
 
Dr.Ruud
Guest
Posts: n/a
 
      08-23-2009
Scott Bryce wrote:
> Keith Thompson wrote:
>> Brian Wakem <(E-Mail Removed)> writes:


>>> Using \x0A instead of \n should work.

>>
>> That seems unlikely, since "\x0a" and "\n" have exactly the same
>> value.

>
> Not on a Windows system.


You are still mixing up layers.

Find out about PerlIO,
see for starters `perldoc -f binmode`:

The operating system, device drivers, C libraries, and Perl run-time
system all work together to let the programmer treat a single character
("\n") as the line terminator, irrespective of the external
representation. On many operating systems, the native text file
representation matches the internal representation, but on some
platforms the external representation of "\n" is made up of more than
one character.

Lasagna! How Latin to name the contents after the container.

--
Ruud
 
Reply With Quote
 
Peter J. Holzer
Guest
Posts: n/a
 
      08-23-2009
On 2009-08-23 00:08, Scott Bryce <(E-Mail Removed)> wrote:
> Keith Thompson wrote:
>> Brian Wakem <(E-Mail Removed)> writes:
>>> Using \x0A instead of \n should work.

>>
>> That seems unlikely, since "\x0a" and "\n" have exactly the same
>> value.

>
>
> Not on a Windows system.


Please.

Is it really so hard to test your assumptions before posting them?
Especially when several regulars have already explained that your
assumptions are wrong?

| Microsoft Windows [Version 6.0.6001]
| Copyright (c) 2006 Microsoft Corporation. Alle Rechte vorbehalten.
|
| C:\>perl -le "print qq{\n} eq qq{\x0A}"
| 1
|
| C:\>perl -v
|
| This is perl, v5.10.0 built for MSWin32-x64-multi-thread
| (with 5 registered patches, see perl -V for more detail)
|
| Copyright 1987-2007, Larry Wall
|
| Binary build 1004 [287188] provided by ActiveState http://www.ActiveState.com
| Built Sep 3 2008 12:22:07

hp
 
Reply With Quote
 
Scott Bryce
Guest
Posts: n/a
 
      08-23-2009
Peter J. Holzer wrote:
> On 2009-08-23 00:08, Scott Bryce <(E-Mail Removed)> wrote:
>> Keith Thompson wrote:
>>> Brian Wakem <(E-Mail Removed)> writes:
>>>> Using \x0A instead of \n should work.
>>> That seems unlikely, since "\x0a" and "\n" have exactly the same
>>> value.

>>
>> Not on a Windows system.

>
> Please.
>
> Is it really so hard to test your assumptions before posting them?


Actually, I DID test this.

-----------

use strict;
use warnings;

open my $OUTFILE, '>', 'test.txt' or die 'Cannot open test.txt';
print $OUTFILE "A line of text\n";
print $OUTFILE "A line of text\n";

close $OUTFILE or die 'cannot close test.txt';

-----------

Both lines in test.txt end in \x0D\x0A

But now I see my mistake. If I binmode $OUTFILE, then both lines end in
\x0A.

The assumption I was making is consistent with the documentation:

The operating system, device drivers, C libraries, and Perl run-time
system all work together to let the programmer treat a single character
(\n ) as the line terminator, irrespective of the external representation.

which suggests that "\n" does not have a specific ASCII value. So the
bottom line is that on a Windows system, the value of "\n" depends on
whether you are working in text mode or binary mode.

A search of the docs for "\n" came up blank, but as someone else pointed
out, everything is spelled out in the docs under binmode.

I stand somewhat corrected.
 
Reply With Quote
 
Peter J. Holzer
Guest
Posts: n/a
 
      08-23-2009
On 2009-08-23 13:58, Scott Bryce <(E-Mail Removed)> wrote:
> Peter J. Holzer wrote:
>> On 2009-08-23 00:08, Scott Bryce <(E-Mail Removed)> wrote:
>>> Keith Thompson wrote:
>>>> Brian Wakem <(E-Mail Removed)> writes:
>>>>> Using \x0A instead of \n should work.
>>>> That seems unlikely, since "\x0a" and "\n" have exactly the same
>>>> value.
>>>
>>> Not on a Windows system.

>>
>> Please.
>>
>> Is it really so hard to test your assumptions before posting them?

>
> Actually, I DID test this.
>
> -----------
>
> use strict;
> use warnings;
>
> open my $OUTFILE, '>', 'test.txt' or die 'Cannot open test.txt';
> print $OUTFILE "A line of text\n";
> print $OUTFILE "A line of text\n";
>
> close $OUTFILE or die 'cannot close test.txt';
>
> -----------


No, you didn't. Your script doesn't use \x0A, so it says nothing about
whether \n and \x0A are the same or not.

Change the second print into:

print $OUTFILE "A line of text\x0A";

> Both lines in test.txt end in \x0D\x0A


With the second line changed, still both lines end with 0D 0A in the
file. This is an indication (but no proof) that "\n" and "\x0A" are
indeed the same.


> But now I see my mistake. If I binmode $OUTFILE, then both lines end in
> \x0A.
>
> The assumption I was making is consistent with the documentation:
>
> The operating system, device drivers, C libraries, and Perl run-time
> system all work together to let the programmer treat a single character
> (\n ) as the line terminator, irrespective of the external representation.
>
> which suggests that "\n" does not have a specific ASCII value.


This is true (on MacOS classic "\n" was "\x0D" and on EBCDIC based
systems it's "\x15", IIRC), but on Windows "\n" is always "\x0A".

And in any case it is always a single character, regardless of the
convention for text files on the OS.

> So the bottom line is that on a Windows system, the value of "\n"
> depends on whether you are working in text mode or binary mode.


No, it doesn't. On Windows "\n" is always the single character with the
code 10 decimal (or 0x0A hexadecimal).

The difference between text mode and binary mode is that text mode
converts to the text file convention of the local OS: For Windows that
means that when you print the single character "\n", the :crlf layer
sends to bytes (0x0D 0x0A) to the file. That's a relatively simple
conversion, there are more complicated conversions. For example, some
OSs had text files with fixed length, space padded lines. On such a
system,

print $OUTFILE "A line of text\n";

causes (for example) 80 bytes to be written to the file: "A line of
text" followed by 66 spaces. But that doesn't mean that the value of
"\n" is 66 spaces.

hp

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Ruby Command Line One Liner Alain Helfenstein Ruby 5 01-11-2009 07:25 PM
Single-liner for one-line substitute? Mike Pearson Perl Misc 13 06-30-2006 04:31 PM
Unix shell script with Perl one-liner causes error Simon O Perl Misc 8 06-15-2006 04:06 PM
perl one liner to display the third line from the end of a file Oxnard Perl Misc 13 06-15-2005 12:35 PM
one-liner to make all programs one-liners Larry Perl Misc 1 02-03-2005 11:35 PM



Advertisments