Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Creating an object that can track when its attributes are modified

Reply
Thread Tools

Creating an object that can track when its attributes are modified

 
 
Ben Sizer
Guest
Posts: n/a
 
      03-06-2013
I am trying to make an object that can track when its attributes have been assigned new values, and which can rollback to previous values where necessary. I have the following code which I believe works, but would like to know if there are simpler ways to achieve this goal, or if there are any bugs I haven't seen yet.


class ChangeTrackingObject(object):
def __init__(self):
self.clean()

def clean(self):
"""Mark all attributes as unmodified."""
object.__setattr__(self, '_dirty_attributes', dict())

def dirty_vals(self):
"""Returns all dirty values."""
return dict( [ (k,v) for k,v in self.__dict__.iteritems() if k in self._dirty_attributes] )

def get_changes_and_clean(self):
"""Helper that collects all the changes and returns them, cleaning the dirty flags at the same time."""
changes = self.dirty_vals()
self.clean()
return changes

def rollback(self):
"""Reset attributes to their previous values."""
for k,v in self._dirty_attributes.iteritems():
object.__setattr__(self, k, v)
self.clean()

def __setattr__(self, key, value):
# If the first modification to this attribute, store the old value
if key not in self._dirty_attributes:
if key in self.__dict__:
self._dirty_attributes[key] = object.__getattribute__(self, key)
else:
self._dirty_attributes[key] = None
# Set the new value
object.__setattr__(self, key, value)


I am aware that adding a new attribute and then calling rollback() leaves the new attribute in place with a None value - maybe I can use a special DeleteMe marker object in the _dirty_attributes dict along with a loop that calls delattr on any attribute that has that value after a rollback.

I also believe that this won't catch modification to existing attributes asopposed to assignments: eg. if one of the attributes is a list and I append to it, this system won't notice. Is that something I can rectify easily?

Any other comments or suggestions?

Thanks,
--
Ben Sizer
 
Reply With Quote
 
 
 
 
Chris Angelico
Guest
Posts: n/a
 
      03-06-2013
On Thu, Mar 7, 2013 at 3:07 AM, Ben Sizer <(E-Mail Removed)> wrote:
> I also believe that this won't catch modification to existing attributes as opposed to assignments: eg. if one of the attributes is a list and I append to it, this system won't notice. Is that something I can rectify easily?


The only way you could detect mutation of one of its attributes is
with that object's assistance. Effectively, you would need to have a
subclass of list/dict/tuple/whatever that can respond to the change.
Alternatively, you could retain a deep copy and do a comparison at
time of rollback; this, however, would have annoying consequences wrt
performance and other references and such.

What's the goal of this class? Can you achieve the same thing by
using, perhaps, a before-and-after snapshot of a JSON-encoded form of
the object?

ChrisA
 
Reply With Quote
 
 
 
 
Lele Gaifax
Guest
Posts: n/a
 
      03-06-2013
Ben Sizer <(E-Mail Removed)> writes:

> I also believe that this won't catch modification to existing
> attributes as opposed to assignments: eg. if one of the attributes is
> a list and I append to it, this system won't notice. Is that something
> I can rectify easily?


It's really up to how far you wanna go: a similar use case is
implemented by SQLAlchemy__, which "instrument" builtin collection
classes to achieve the goal. But I'd not call that "easy" though

ciao, lele.

__ http://www.sqlalchemy.org/trac/brows...collections.py
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
http://www.velocityreviews.com/forums/(E-Mail Removed) | -- Fortunato Depero, 1929.

 
Reply With Quote
 
Ben Sizer
Guest
Posts: n/a
 
      03-06-2013
On Wednesday, 6 March 2013 16:22:56 UTC, Chris Angelico wrote:
>
> Effectively, you would need to have a
> subclass of list/dict/tuple/whatever that can respond to the change.


This is certainly something I'd be interested in having, but I guess that would be fragile since the user would have the burden of having to remember to use those types.

> What's the goal of this class? Can you achieve the same thing by
> using, perhaps, a before-and-after snapshot of a JSON-encoded form of
> the object?
>


I need to be able to perform complex operations on the object that may modify several properties, and then gather the properties at the end as an efficient way to see what has changed and to store those changes. Any comparison of before-and-after snapshots could work in theory, but in practice it could be expensive to produce the snapshots on larger objects and probably expensive to calculate the differences that way too. Performance is importantso I would probably just go for an explicit function call to mark an attribute as having been modified rather than trying to do a diff like that. (Itwouldn't work for rollbacks, but I can accept that.)

--
Ben Sizer
 
Reply With Quote
 
Ben Sizer
Guest
Posts: n/a
 
      03-06-2013
On Wednesday, 6 March 2013 16:22:56 UTC, Chris Angelico wrote:
>
> Effectively, you would need to have a
> subclass of list/dict/tuple/whatever that can respond to the change.


This is certainly something I'd be interested in having, but I guess that would be fragile since the user would have the burden of having to remember to use those types.

> What's the goal of this class? Can you achieve the same thing by
> using, perhaps, a before-and-after snapshot of a JSON-encoded form of
> the object?
>


I need to be able to perform complex operations on the object that may modify several properties, and then gather the properties at the end as an efficient way to see what has changed and to store those changes. Any comparison of before-and-after snapshots could work in theory, but in practice it could be expensive to produce the snapshots on larger objects and probably expensive to calculate the differences that way too. Performance is importantso I would probably just go for an explicit function call to mark an attribute as having been modified rather than trying to do a diff like that. (Itwouldn't work for rollbacks, but I can accept that.)

--
Ben Sizer
 
Reply With Quote
 
Chris Angelico
Guest
Posts: n/a
 
      03-06-2013
On Thu, Mar 7, 2013 at 3:56 AM, Ben Sizer <(E-Mail Removed)> wrote:
> On Wednesday, 6 March 2013 16:22:56 UTC, Chris Angelico wrote:
>>
>> Effectively, you would need to have a
>> subclass of list/dict/tuple/whatever that can respond to the change.

>
> This is certainly something I'd be interested in having, but I guess thatwould be fragile since the user would have the burden of having to remember to use those types.


Since you're already overriding setattr, you could simply force all
non-string sequences to your special subclass of list. That reduces
that burden, though it'd break if there are any other references to
the object.

>> What's the goal of this class? Can you achieve the same thing by
>> using, perhaps, a before-and-after snapshot of a JSON-encoded form of
>> the object?
>>

>
> I need to be able to perform complex operations on the object that may modify several properties, and then gather the properties at the end as an efficient way to see what has changed and to store those changes. Any comparison of before-and-after snapshots could work in theory, but in practice it could be expensive to produce the snapshots on larger objects and probably expensive to calculate the differences that way too. Performance is important so I would probably just go for an explicit function call to mark an attribute as having been modified rather than trying to do a diff like that. (It wouldn't work for rollbacks, but I can accept that.)


Hmm. Interesting. The perfect solution probably is too messy, yeah.
But if you have your subclassing done, you could possibly
snapshot-on-write, which would allow the rollback. Not sure if it'd
help though.

ChrisA
 
Reply With Quote
 
88888 Dihedral
Guest
Posts: n/a
 
      03-06-2013
Ben Sizer於 2013年3月7日星期四UTC+8上午12時56分09秒 寫道:
> On Wednesday, 6 March 2013 16:22:56 UTC, Chris Angelico wrote:
>
> >

>
> > Effectively, you would need to have a

>
> > subclass of list/dict/tuple/whatever that can respond to the change.

>
>
>
> This is certainly something I'd be interested in having, but I guess thatwould be fragile since the user would have the burden of having to remember to use those types.
>
>
>
> > What's the goal of this class? Can you achieve the same thing by

>
> > using, perhaps, a before-and-after snapshot of a JSON-encoded form of

>
> > the object?

>
> >

>
>
>
> I need to be able to perform complex operations on the object that may modify several properties, and then gather the properties at the end as an efficient way to see what has changed and to store those changes. Any comparison of before-and-after snapshots could work in theory, but in practice it could be expensive to produce the snapshots on larger objects and probably expensive to calculate the differences that way too. Performance is important so I would probably just go for an explicit function call to mark an attribute as having been modified rather than trying to do a diff like that. (It wouldn't work for rollbacks, but I can accept that.)
>
>
>
> --
>
> Ben Sizer

Please hook a stack implemented as a list in python to every property
of the object that you want to track down.


 
Reply With Quote
 
88888 Dihedral
Guest
Posts: n/a
 
      03-06-2013
Ben Sizer於 2013年3月7日星期四UTC+8上午12時56分09秒 寫道:
> On Wednesday, 6 March 2013 16:22:56 UTC, Chris Angelico wrote:
>
> >

>
> > Effectively, you would need to have a

>
> > subclass of list/dict/tuple/whatever that can respond to the change.

>
>
>
> This is certainly something I'd be interested in having, but I guess thatwould be fragile since the user would have the burden of having to remember to use those types.
>
>
>
> > What's the goal of this class? Can you achieve the same thing by

>
> > using, perhaps, a before-and-after snapshot of a JSON-encoded form of

>
> > the object?

>
> >

>
>
>
> I need to be able to perform complex operations on the object that may modify several properties, and then gather the properties at the end as an efficient way to see what has changed and to store those changes. Any comparison of before-and-after snapshots could work in theory, but in practice it could be expensive to produce the snapshots on larger objects and probably expensive to calculate the differences that way too. Performance is important so I would probably just go for an explicit function call to mark an attribute as having been modified rather than trying to do a diff like that. (It wouldn't work for rollbacks, but I can accept that.)
>
>
>
> --
>
> Ben Sizer

Please hook a stack implemented as a list in python to every property
of the object that you want to track down.


 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      03-07-2013
On Wed, 06 Mar 2013 08:56:09 -0800, Ben Sizer wrote:

> I need to be able to perform complex operations on the object that may
> modify several properties, and then gather the properties at the end as
> an efficient way to see what has changed and to store those changes. Any
> comparison of before-and-after snapshots could work in theory, but in
> practice it could be expensive to produce the snapshots on larger
> objects and probably expensive to calculate the differences that way
> too. Performance is important so I would probably just go for an
> explicit function call to mark an attribute as having been modified
> rather than trying to do a diff like that. (It wouldn't work for
> rollbacks, but I can accept that.)



Premature optimization.

Unless you have been eating and breathing Python code for 15+ years, your
intuition of what is expensive and what isn't will probably be *way* off.
I've been using Python for ~15 years, and I wouldn't want to try to guess
what the most efficient way to do this will be.

Actually I lie. I would guess that the simple, most obvious way is
faster: don't worry about storing what changed, just store *everything*.
But I could be wrong.

Fortunately, Python development is rapid enough that you can afford to
develop this object the straightforward way, profile your application to
see where the bottlenecks are, and if it turns out that the simple
approach is too expensive, then try something more complicated.



--
Steven
 
Reply With Quote
 
Ben Sizer
Guest
Posts: n/a
 
      03-07-2013
On Thursday, 7 March 2013 00:07:02 UTC, Steven D'Aprano wrote:
> On Wed, 06 Mar 2013 08:56:09 -0800, Ben Sizer wrote:
>
> > I need to be able to perform complex operations on the object that may
> > modify several properties, and then gather the properties at the end as
> > an efficient way to see what has changed and to store those changes. Any
> > comparison of before-and-after snapshots could work in theory, but in
> > practice it could be expensive to produce the snapshots on larger
> > objects and probably expensive to calculate the differences that way
> > too. Performance is important so I would probably just go for an
> > explicit function call to mark an attribute as having been modified
> > rather than trying to do a diff like that. (It wouldn't work for
> > rollbacks, but I can accept that.)

>
> Premature optimization.
>
> Unless you have been eating and breathing Python code for 15+ years, your
> intuition of what is expensive and what isn't will probably be *way* off.
> I've been using Python for ~15 years, and I wouldn't want to try to guess
> what the most efficient way to do this will be.


I admit, I've only been using Python for 10 years, but I've learned a lot about optimisation and what needs optimising from my time as a game developer. This code needs to be fairly high-performing due to the role it plays inmy server and the frequency with which the behaviour gets called.

> Actually I lie. I would guess that the simple, most obvious way is
> faster: don't worry about storing what changed, just store *everything*.
> But I could be wrong.


The use case I have is not one where that is suitable. It's not the snapshots that are important, but the changes between them.

> Fortunately, Python development is rapid enough that you can afford to
> develop this object the straightforward way, profile your application to
> see where the bottlenecks are, and if it turns out that the simple
> approach is too expensive, then try something more complicated.


I don't see a more straightforward solution to the problem I have than the one I have posted. I said that a system that took snapshots of the whole object and attempted to diff them would probably perform worse, but it would probably be more complex too, given the traversal and copying requirements.

--
Ben Sizer
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
problem in running a basic code in python 3.3.0 that includes HTML file Satabdi Mukherjee Python 1 04-04-2013 07:48 PM
Its a bird, its a plane, its.. um, an Attribute based System? thunk Ruby 14 04-03-2010 10:08 AM
Its a bird, its a plane, its.. um, an Attribute based System? thunk Ruby 0 04-01-2010 10:25 PM
Its a bird, its a plane, no ummm, its a Ruide thunk Ruby 1 03-30-2010 11:10 AM
insert track reference in mp3 track D@Z Computer Support 1 05-02-2008 03:32 PM



Advertisments