«

This question is really about sed not python, hence it's totally off.
But since lots of unix heads are frequenting this list I thought I'd
try my luck nevertheless.

If I have a file with content

1
2
3
4
5
6
7
8
........

i.e. each line contains simply its line number, then it's quite easy
to convert it into

2
3
7
8
12
13
............

using python. The pattern is that the first line is deleted, then 2
lines are kept, 3 lines are deleted, 2 lines are kept, 3 lines are
deleted, etc, etc.

But I couldn't find a way to do this with sed and since the whole
operation is currently done with a bash script I'd hate to move to
python just to do this simple task.

What would be the sed equivalent?

Cheers,
Daniel



--
Psss, psss, put it down! - http://www.cafepress.com/putitdown

Daniel Fetchinson writes:

> This question is really about sed not python, hence it's totally
> off. But since lots of unix heads are frequenting this list I
> thought I'd try my luck nevertheless.
....
> using python. The pattern is that the first line is deleted, then 2
> lines are kept, 3 lines are deleted, 2 lines are kept, 3 lines are
> deleted, etc, etc.
>
> But I couldn't find a way to do this with sed and since the whole
> operation is currently done with a bash script I'd hate to move to
> python just to do this simple task.
>
> What would be the sed equivalent?

The following appears to work here. Both parts of the address are
documented as GNU extensions in the man page: 2~5 matches line 2 and
then every 5th line, and ,+1 tells sed to match also the 1 line after
each match. With -n, do not print by default, and p is the command to
print when an address matches.

sed -n '2~5,+1 p'

Tried with GNU sed version 4.1.2, never used sed this way before.

So, is there some simple expression in Python for this? Just asking
out of curiosity when nothing comes to mind, not implying that there
should be or that Python should be changed in any way.

Jussi Piitulainen writes:
> Daniel Fetchinson writes:
>
> > This question is really about sed not python, hence it's totally
> > off. But since lots of unix heads are frequenting this list I
> > thought I'd try my luck nevertheless.
> ...
> > using python. The pattern is that the first line is deleted, then 2
> > lines are kept, 3 lines are deleted, 2 lines are kept, 3 lines are
> > deleted, etc, etc.
> >
> > But I couldn't find a way to do this with sed and since the whole
> > operation is currently done with a bash script I'd hate to move to
> > python just to do this simple task.
> >
> > What would be the sed equivalent?
>
> The following appears to work here. Both parts of the address are
> documented as GNU extensions in the man page: 2~5 matches line 2 and
> then every 5th line, and ,+1 tells sed to match also the 1 line after
> each match. With -n, do not print by default, and p is the command to
> print when an address matches.
>
> sed -n '2~5,+1 p'
>
> Tried with GNU sed version 4.1.2, never used sed this way before.
>
> So, is there some simple expression in Python for this? Just asking
> out of curiosity when nothing comes to mind, not implying that there
> should be or that Python should be changed in any way.

To expand, below is the best I can think of in Python 3 and I'm
curious if there is something much more concise built in that I am
missing.

def sed(source, skip, keep, drop):

'''First skip some elements from source,
then keep yielding some and dropping
some: sed(source, 1, 2, 3) to skip 1,
yield 2, drop 3, yield 2, drop 3, ...'''

for _ in range(0, skip):
next(source)
while True:
for _ in range(0, keep):
yield next(source)
for _ in range(0, drop):
next(source)

On 10/27/10 09:39, Jussi Piitulainen wrote:
>> So, is there some simple expression in Python for this? Just asking
>> out of curiosity when nothing comes to mind, not implying that there
>> should be or that Python should be changed in any way.
>
> To expand, below is the best I can think of in Python 3 and I'm
> curious if there is something much more concise built in that I am
> missing.
>
> def sed(source, skip, keep, drop):
>
> '''First skip some elements from source,
> then keep yielding some and dropping
> some: sed(source, 1, 2, 3) to skip 1,
> yield 2, drop 3, yield 2, drop 3, ...'''
>
> for _ in range(0, skip):
> next(source)
> while True:
> for _ in range(0, keep):
> yield next(source)
> for _ in range(0, drop):
> next(source)

Could be done as: (py2.x in this case, adjust accordingly for 3.x)

def sed(source, skip, keep, drop):
for _ in range(skip): source.next()
tot = keep + drop
for i, item in enumerate(source):
if i % tot < keep:
yield item

-tkc

Tim Chase <> writes:

> On 10/27/10 09:39, Jussi Piitulainen wrote:
>>> So, is there some simple expression in Python for this? Just asking
>>> out of curiosity when nothing comes to mind, not implying that there
>>> should be or that Python should be changed in any way.
>>
>> To expand, below is the best I can think of in Python 3 and I'm
>> curious if there is something much more concise built in that I am
>> missing.
>>
>> def sed(source, skip, keep, drop):
>>
>> '''First skip some elements from source,
>> then keep yielding some and dropping
>> some: sed(source, 1, 2, 3) to skip 1,
>> yield 2, drop 3, yield 2, drop 3, ...'''
>>
>> for _ in range(0, skip):
>> next(source)
>> while True:
>> for _ in range(0, keep):
>> yield next(source)
>> for _ in range(0, drop):
>> next(source)
>
> Could be done as: (py2.x in this case, adjust accordingly for 3.x)
>
> def sed(source, skip, keep, drop):
> for _ in range(skip): source.next()
> tot = keep + drop
> for i, item in enumerate(source):
> if i % tot < keep:
> yield item
>
> -tkc

With Python 2.7+ you can use itertools.compress:

>>> from itertools import *
>>> def sed(source, skip, keep, drop):
.... return compress(source, chain([0]*skip, cycle([1]*keep + [0]*drop)))
....
>>> list(sed(range(20), 1, 2, 3))
[1, 2, 6, 7, 11, 12, 16, 17]

--
Arnaud

Jussi Piitulainen <> writes:

> Daniel Fetchinson writes:
>
> > The pattern is that the first line is deleted, then 2 lines are
> > kept, 3 lines are deleted, 2 lines are kept, 3 lines are deleted,
> > etc, etc.
>
> So, is there some simple expression in Python for this?

(item for i, item in enumerate(input) if (i + 4)%5 < 2)

-- [mdw]