Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Batched Searching With a JOIN

Reply
Thread Tools

Batched Searching With a JOIN

 
 
Matt Mencel
Guest
Posts: n/a
 
      04-08-2010
I have two tables (IdmLdapAccounts and IdmAdAccounts) each with 20000-30000 records in them. What I'm trying to do is join these tables so that I can find out what records from the LDAP table do NOT exist in the AD table.

uid is indexed in the LDAP table and samaccountname is indexed in the AD table.

I tried using some LEFT OUTER JOINs....but couldn't figure out how to get that to work with the Model.find_each that does batched queries.

So this is what I do now....it takes about 25 seconds to run...and that's using SQLPlus in my dev environment. I would think it would be faster when I move to a real MySQL db.

accounts = []
IdmLdapAccount.find_each do |ldap_account|
if !IdmAdAccount.exists?(:samaccountname => ldap_account.uid)
accounts << ldap_account
end
end
ap accounts.size
exit


This produces the result I want, but I don't know if it's the best way to do this. This queries the AD table once for every row in the LDAP table (about 23000 single queries)....it's pretty quick....but seems inefficient. Wouldn't a true JOIN work better?

Any thoughts?

Thanks,
Matt


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Google search result to be URL-limited when searching site, but notwhen searching Web stumblng.tumblr Javascript 1 02-04-2008 09:01 AM
You must join New York Paid To Read !!!!!!! NewYork PTR is online!!! JOIN NOW!!!!!! Alan Silver ASP .Net 0 06-05-2006 03:27 PM
list.join()... re.join()...? Do they exist? (newbie questions...) googleboy Python 1 10-01-2005 12:56 PM
Please Join My Site! Flukemanguy The Lounge 23 09-13-2005 02:04 PM
IPRIP could not join the multicast group 224.0.0.9 =?Utf-8?B?U2NvdA==?= Wireless Networking 0 12-12-2004 07:29 PM



Advertisments