«

I have some marked up text and would like to convert it to plain text,
by simply removing all the tags. Of course I can do it from first
principles but I felt that among all Python's markup tools there must
be something that would do this simply, without having to create an
XML parser etc.

I've looked around a bit but failed to find anything, any tips?

(e.g. convert "<B>Today</B> is <U>Friday</U>" to "Today is Friday")

Regards,
Geoff

> I have some marked up text and would like to convert it to plain text,
> by simply removing all the tags. Of course I can do it from first
> principles but I felt that among all Python's markup tools there must
> be something that would do this simply, without having to create an
> XML parser etc.
>
> I've looked around a bit but failed to find anything, any tips?
>
> (e.g. convert "<B>Today</B> is <U>Friday</U>" to "Today is Friday")


Well, if all you want to do is remove everything from a "<" to a
">", you can use

>>> s = "<B>Today</B> is <U>Friday</U>"
>>> import re
>>> r = re.compile('<[^>]*>')
>>> print r.sub('', s)
Today is Friday

it should even work for semi-pathological cases such as

s = """You can find my <a
href='http://example.com'>thesis</a
> online"""

where the tag contents are split across lines. There are more
pathological cases where tags aren't well-formed, e.g.

s ="This <tag>has a > sign in it and <odd<ly>-nested> tags"

in which case you get what you deserve for making such
pathological conditions

-tkc

On 01-Feb-2008, geoffbache wrote:
> I have some marked up text and would like to convert it to plain text,
> by simply removing all the tags. Of course I can do it from first
> principles but I felt that among all Python's markup tools there must
> be something that would do this simply, without having to create an
> XML parser etc.
>
> I've looked around a bit but failed to find anything, any tips?
>
> (e.g. convert "<B>Today</B> is <U>Friday</U>" to "Today is Friday")

Quick but very dirty way:

data=urllib.urlopen('http://google.com').read()
data=''.join([x.split('>',1)[-1] for x in data.split('<')])

Tim Chase wrote:
>> I have some marked up text and would like to convert it to plain text,
>> by simply removing all the tags. Of course I can do it from first
>> principles but I felt that among all Python's markup tools there must
>> be something that would do this simply, without having to create an
>> XML parser etc.
>>
>> I've looked around a bit but failed to find anything, any tips?
>>
>> (e.g. convert "<B>Today</B> is <U>Friday</U>" to "Today is Friday")
>
>
> Well, if all you want to do is remove everything from a "<" to a
> ">", you can use
>
> >>> s = "<B>Today</B> is <U>Friday</U>"
> >>> import re
> >>> r = re.compile('<[^>]*>')
> >>> print r.sub('', s)
> Today is Friday
>
> it should even work for semi-pathological cases such as
>
> s = """You can find my <a
> href='http://example.com'>thesis</a
> > online"""
>
> where the tag contents are split across lines. There are more
> pathological cases where tags aren't well-formed, e.g.
>
> s ="This <tag>has a > sign in it and <odd<ly>-nested> tags"
>
> in which case you get what you deserve for making such
> pathological conditions
>
The real answer to this question is "learn how to use Beautiful Soup" --
see http://www.crummy.com/software/BeautifulSoup/

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

>> Well, if all you want to do is remove everything from a "<" to a
>> ">", you can use
>>
>> >>> s = "<B>Today</B> is <U>Friday</U>"
>> >>> import re
>> >>> r = re.compile('<[^>]*>')
>> >>> print r.sub('', s)
>> Today is Friday
>>
[Tim's ramblings about pathological cases snipped]
>
> The real answer to this question is "learn how to use Beautiful Soup" --
> see http://www.crummy.com/software/BeautifulSoup/

Yes, for more pathological cases, BS does a great job of parsing
junk

However, as BS isn't batteries-included [Aside: BS and pyparsing
are two common solutions to problems that would make great
additions to the standard library], using a RE to make a
best-effort guess is a good first approximation of a solution
without needing to download extra packages--no matter how useful
those extra packages may be.

-tkc

On Feb 1, 10:54*am, Tim Chase <python.l...@tim.thechases.com> wrote:
> >> Well, if all you want to do is remove everything from a "<" to a
> >> ">", you can use
>
> >> * >>> s = "<B>Today</B> is <U>Friday</U>"
> >> * >>> import re
> >> * >>> r = re.compile('<[^>]*>')
> >> * >>> print r.sub('', s)
> >> * Today is Friday
>
> [Tim's ramblings about pathological cases snipped]

pyparsing includes an example script for stripping tags from HTML
source. See it on the wiki at http://pyparsing.wikispaces.com/spac...tmlStripper.py.

-- Paul

On Feb 1, 8:07 am, geoffbache <geoff.ba...@pobox.com> wrote:
> I have some marked up text and would like to convert it to plain text,

If this is just a quick and dirty problem, you can also use one of the
lynx/elinks/links2 browsers and dump the contents to a file. On Linux
it would be
lynx -dump http://www.etc > text.txt
Lynx is also available for MS Windows, but am not sure about the other
two.

geoffbache wrote:
> I have some marked up text and would like to convert it to plain text,
> by simply removing all the tags. Of course I can do it from first
> principles but I felt that among all Python's markup tools there must
> be something that would do this simply, without having to create an
> XML parser etc.
>
> I've looked around a bit but failed to find anything, any tips?
>
> (e.g. convert "<B>Today</B> is <U>Friday</U>" to "Today is Friday")

>>> import lxml.etree as et
>>> doc = et.HTML("<b>Today</b> is <u>Friday</u>")
>>> et.tostring(doc, method='text', encoding=unicode)
u'Today is Friday'


http://codespeak.net/lxml

Stefan

geoffbache wrote:
> I have some marked up text and would like to convert it to plain text,
> by simply removing all the tags. Of course I can do it from first
> principles but I felt that among all Python's markup tools there must
> be something that would do this simply, without having to create an
> XML parser etc.
>
> I've looked around a bit but failed to find anything, any tips?
>
> (e.g. convert "<B>Today</B> is <U>Friday</U>" to "Today is Friday")

This might be of interest:

http://pypi.python.org/pypi/haufe.stripml

Stefan