Data Mining

Discussion in 'NZ Computing' started by Shane, Jun 27, 2007.

  1. Shane

    Shane Guest

    Has anyone here dabbled in said black art?
    What tools did you find helpful/worthwhile?
    Any hidden catches you care to mention?

    TIA
    --
    It is only two weeks into the term that, in a calculus class, a student
    raises his hand and asks: "Will we ever need this stuff in real life?"
    The professor gently smiles at him and says: "Of course not - if your real
    life will consist of flipping hamburgers at MacDonald's!"
    Shane, Jun 27, 2007
    #1
    1. Advertising

  2. Shane

    thingy Guest

    Shane wrote:
    > Has anyone here dabbled in said black art?
    > What tools did you find helpful/worthwhile?
    > Any hidden catches you care to mention?
    >
    > TIA


    Catches,

    Normalise your data....if you only have one data source then it is
    probably no huge issue...ie that source and that system's owner owns the
    data and its syntax...

    As an example of problems when you want to data mine and your data is
    in-consistant...

    eg Location WGTN, WN, wn, Wn, Wellington, wellington....

    Who owns the data and is therefore in control ie facilities would
    probably own the extension number a user is on, so they determine its
    layout eg 5436 or ext5436 etc etc.

    Now you might have marketing, sales, finance all having their own
    conflicting data and [web]forms for inputting it with in-adequate checks
    on the syntax/layout....

    So when you go to data mine info,and say dump it into a mailing list or
    phone list and its automated, mayhem results....

    So if you have multiple owners of each field you have to determine who
    is the owner and where you get that info from, eg a HR database might
    hold the employee number, facilities databse the ext number....IT
    database/AD the email address.....all on separate systems....

    So datamining can be very useful...however the costs can be horrendious....

    These days a green field site should consider this before they decide
    much else, it is that fundimental...

    This is why you get an enterprise architect...and no I dont mean a
    consultant....most are useless...

    regards

    Thing
    thingy, Jun 27, 2007
    #2
    1. Advertising

  3. Shane

    Shane Guest

    thingy wrote:

    > Shane wrote:
    >> Has anyone here dabbled in said black art?
    >> What tools did you find helpful/worthwhile?
    >> Any hidden catches you care to mention?
    >>
    >> TIA

    >
    > Catches,
    >
    > Normalise your data....if you only have one data source then it is
    > probably no huge issue...ie that source and that system's owner owns the
    > data and its syntax...
    >
    > As an example of problems when you want to data mine and your data is
    > in-consistant...
    >
    > eg Location WGTN, WN, wn, Wn, Wellington, wellington....
    >
    > Who owns the data and is therefore in control ie facilities would
    > probably own the extension number a user is on, so they determine its
    > layout eg 5436 or ext5436 etc etc.
    >
    > Now you might have marketing, sales, finance all having their own
    > conflicting data and [web]forms for inputting it with in-adequate checks
    > on the syntax/layout....
    >
    > So when you go to data mine info,and say dump it into a mailing list or
    > phone list and its automated, mayhem results....
    >
    > So if you have multiple owners of each field you have to determine who
    > is the owner and where you get that info from, eg a HR database might
    > hold the employee number, facilities databse the ext number....IT
    > database/AD the email address.....all on separate systems....
    >
    > So datamining can be very useful...however the costs can be
    > horrendious....
    >
    > These days a green field site should consider this before they decide
    > much else, it is that fundimental...
    >
    > This is why you get an enterprise architect...and no I dont mean a
    > consultant....most are useless...
    >
    > regards
    >
    > Thing


    Awesome, thanks thing. Im expecting my data to be even more varied than
    that, so Ive definitely found a challenge ;-)
    --
    Q: What is often used by Canadians to help solve certain differential
    equations?
    A: the Lacrosse transform.
    Shane, Jun 28, 2007
    #3
  4. Shane

    thingy Guest

    Shane wrote:
    > thingy wrote:
    >
    >> Shane wrote:
    >>> Has anyone here dabbled in said black art?
    >>> What tools did you find helpful/worthwhile?
    >>> Any hidden catches you care to mention?
    >>>
    >>> TIA

    >> Catches,
    >>
    >> Normalise your data....if you only have one data source then it is
    >> probably no huge issue...ie that source and that system's owner owns the
    >> data and its syntax...
    >>
    >> As an example of problems when you want to data mine and your data is
    >> in-consistant...
    >>
    >> eg Location WGTN, WN, wn, Wn, Wellington, wellington....
    >>
    >> Who owns the data and is therefore in control ie facilities would
    >> probably own the extension number a user is on, so they determine its
    >> layout eg 5436 or ext5436 etc etc.
    >>
    >> Now you might have marketing, sales, finance all having their own
    >> conflicting data and [web]forms for inputting it with in-adequate checks
    >> on the syntax/layout....
    >>
    >> So when you go to data mine info,and say dump it into a mailing list or
    >> phone list and its automated, mayhem results....
    >>
    >> So if you have multiple owners of each field you have to determine who
    >> is the owner and where you get that info from, eg a HR database might
    >> hold the employee number, facilities databse the ext number....IT
    >> database/AD the email address.....all on separate systems....
    >>
    >> So datamining can be very useful...however the costs can be
    >> horrendious....
    >>
    >> These days a green field site should consider this before they decide
    >> much else, it is that fundimental...
    >>
    >> This is why you get an enterprise architect...and no I dont mean a
    >> consultant....most are useless...
    >>
    >> regards
    >>
    >> Thing

    >
    > Awesome, thanks thing. Im expecting my data to be even more varied than
    > that, so Ive definitely found a challenge ;-)



    I'd expect it to be....we have had various consultants come in claiming
    to give us all this "business intelligence" and then run away....the
    hardest part is normalising I suspect, after that we dont need
    consultants....something like Politicians....

    regards

    Thing
    thingy, Jun 28, 2007
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. [G]rumpy [O]ld [D]uffer

    WebSite Data Mining

    [G]rumpy [O]ld [D]uffer, Jul 15, 2004, in forum: Computer Support
    Replies:
    1
    Views:
    434
  2. Jeff
    Replies:
    11
    Views:
    3,006
  3. Mark Maloof
    Replies:
    0
    Views:
    603
    Mark Maloof
    Feb 11, 2006
  4. imhotep

    Government Increasingly Turning to Data Mining

    imhotep, Jun 16, 2006, in forum: Computer Security
    Replies:
    0
    Views:
    421
    imhotep
    Jun 16, 2006
  5. steve

    Data mining robots

    steve, Oct 31, 2003, in forum: NZ Computing
    Replies:
    0
    Views:
    495
    steve
    Oct 31, 2003
Loading...

Share This Page