remote desktop connection logoff hangs

When logging off remote desktop connection the window says that I'm logging
off, but it takes several minutes to actually logoff. The client computer is
running XP pro sp1, and it is connecting to a SBS 2003 server. Is there a
way that I can fix this so that the logoff is as close to instant as possible.

=?Utf-8?B?VmlwZXJE?=