Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > ANN: a mini-language for encapsulating deep-copy operations on Pythondata structures

Reply
Thread Tools

ANN: a mini-language for encapsulating deep-copy operations on Pythondata structures

 
 
Steve Howell
Guest
Posts: n/a
 
      11-18-2009
During the last few days I have written code in support of a small DDL
language that encapsulates a concise representation of the
manipulations needed to make a deep subcopy of a Python-like data
structure. It is inspired by syntax from mainstream modern languages,
including, of course, Python. The DDL can be converted to an AST. That
AST can be walked to either generate Python code that expresses the
mapping or to generate Python objects that can execute the mapping.
Either of the prior outputs can then subsequently be applied to real
world inputs to create new Python data structures from old ones, using
the mechanisms specified in the original DDL for attribute access,
dictionary lookup, iteration, method invocation, etc.

Here is an example of the DDL (and I hate the terminology "DDL," just
cannot think of anything better):


{
'show_table_of_contents',
'author' {
.person 'name',
.person 'location' as city,
.favorite_books()[
.title,
.cost() as expense
] as books}
}


There are more details here:

http://showellonprogramming.blogspot...py-schema.html

Apart from shamelessly plugging my blog, I am hoping to generate
ideas, constructive criticism, etc.

Feel free to comment here or on the blog. So far there are three
entries on the blog, all pertaining to Python.

The implementation of the idea is quite new, but it is a topic that I
have been pondering for a long time--no matter how expressive your
programming language of choice might be, there are certain programming
tasks that just seem needlessly tedious. The mini-language above
proposes to solve one problem but solve it well.

I am particularly interested to find out whether somebody has tried
something like this before.





 
Reply With Quote
 
 
 
 
Steve Howell
Guest
Posts: n/a
 
      11-18-2009
On Nov 18, 4:34*am, "M.-A. Lemburg" <(E-Mail Removed)> wrote:
> Steve Howell wrote:
> > [...]
> > Here is an example of the DDL (and I hate the terminology "DDL," just
> > cannot think of anything better):

>
> > * * * * * * {
> > * * * * * * * * 'show_table_of_contents',
> > * * * * * * * * 'author' {
> > * * * * * * * * * * .person 'name',
> > * * * * * * * * * * .person 'location' as city,
> > * * * * * * * * * * .favorite_books()[
> > * * * * * * * * * * * * .title,
> > * * * * * * * * * * * * .cost() as expense
> > * * * * * * * * * * * * ] as books}
> > * * * * * * }

>
> > There are more details here:

>
> >http://showellonprogramming.blogspot...n-python-deep-...


Thanks for your feedback, Marc-Andre! Comments inline...

>
> Personally, I find the explicit approach more intuitive, since
> you immediately see what you are going to get:
>
> show_table_of_contents = context['show_table_of_contents']
> author = context['author']
> d = {
> * * 'show_table_of_contents': show_table_of_contents,
> * * 'author': {
> * * * * 'name': author.person['name'],
> * * * * 'city': author.person['location'],
> * * * * 'books': [{
> * * * * * * 'title': item.title,
> * * * * * * 'expense': item.cost(),
> * * * * * * }
> * * * * * * for item in author.favorite_books()],
> * * * * },
> * * }
>


I understand what you mean about the explicit approach. When you go
for brevity, you sacrifice some explicitness. One of the things I'm
actually looking to do next is to be able to generate code like what
you've written above. You could make the generated code even more
procedural in terms of how it walks the structure--for example,
replace the list comprehension with a for loop. This could be useful
for inserting tracing code, for example.

> I particularly find this part non-intuitive:
>
> > * * * * * * * * * * .favorite_books()[
> > * * * * * * * * * * * * .title,
> > * * * * * * * * * * * * .cost() as expense
> > * * * * * * * * * * * * ] as books}

>


Yep, I think what I really want is this:

.favorite_books()[{
.title,
.cost() as expense
}] as books}

I am not sure the curlies make the transformation completely
intuitive, but they are more suggestive that you are generating a list
of dictionaries.

> and would leave in the square brackets for __getitem__
> lookups on these:
>
> > * * * * * * * * * * .person 'name',
> > * * * * * * * * * * .person 'location' as city,

>


Yep, I agree. The square brackets were not a deliberate omission.
They were just slightly more tricky to implement because of '[' having
double syntactical meaning.

> For additional inspiration, you might want to look at XSLT
> which provides similar transformations on XML data structures.
>
> There are also a number of other transformation languages:
>
> http://en.wikipedia.org/wiki/Model_T...transformation
>


Thanks for the links!

> Regarding the term "DDL": that's normally used for "data definition
> language" and doesn't really have all that much to do with
> transforming data. You normally define data structures using
> DDL - without actually putting data into those structures.
>
> Why not "PyDTL".... Python data transformation language ?!
>


Yep, I like that suggestion. The language does indeed describe a
transformation. In some ways I was wanting to think of it more
broadly, in terms that the transformation also acts as a schema for
any input objects. For example, you could use '{.title, .publisher,
{.name} }' as a schema to validate that an object behaves like a
book. I suppose that's still a transformation in an abstract sense,
where the output is just True, False, or maybe even the slice of the
transformation that can/cannot be executed.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
stand-alone JMS, other JDBC operations, and transactions ( ActiveMQ + JOTM + JDBC operations ) Jesus M. Salvo Jr. Java 2 02-11-2006 06:33 PM
STL :: Set operations on sorted structures Generic Usenet Account C++ 8 12-07-2005 03:26 AM
structures, structures and more structures (questions about nestedstructures) Alfonso Morra C Programming 11 09-24-2005 07:42 PM
Crystal Reports in asp.net - Encapsulating page failed - ERROR Phin ASP .Net 1 03-30-2005 12:27 AM
Encapsulating HTML in XML using XSL Greg Galloway XML 1 08-05-2004 11:09 AM



Advertisments