Q: Which one is better for parsing 30MB files ? ATL/COM or VB/COM

Discussion in 'MCSD' started by Ramie, Oct 30, 2003.

  1. Ramie

    Ramie Guest

    Hello,

    I am working on a B2B web serivce application. I am
    Systems Architect for this application. I am in the
    process of desigining my components. I would like to know
    which one is better over the other for parsing 30MB
    files. We would like to use one of the following two
    choices: ATL/COM or VB/COM. Can you please share your
    thoughts on this ?

    Regards,
    Ramie
     
    Ramie, Oct 30, 2003
    #1
    1. Advertising

  2. Ramie

    Jay Walters Guest

    Until another response is posted, I would go with VB only
    because of future maintainability.

    Obviously I don't know your requirements.

    Is this file created from the webservice?, and after it's
    creation you need to call a component to do something
    with it?

    Or is the file a feed from a partner that's updated at
    different times (outside of the web service)? If so you
    might consider using an NT service and filewatcher.


    >-----Original Message-----
    >Hello,
    >
    >I am working on a B2B web serivce application. I am
    >Systems Architect for this application. I am in the
    >process of desigining my components. I would like to

    know
    >which one is better over the other for parsing 30MB
    >files. We would like to use one of the following two
    >choices: ATL/COM or VB/COM. Can you please share your
    >thoughts on this ?
    >
    >Regards,
    >Ramie
    >.
    >
     
    Jay Walters, Oct 30, 2003
    #2
    1. Advertising

  3. Ramie

    Ramie Guest

    Sorry. I did not specify my requirements cleary. Web
    Service gets the input file (as big as 30MB) via HTTP
    post. The user posts file to an ASP page. Then, the ASP
    page will call MTS component, which will parse the file to
    extract three piece of info: User Id, Password, and some
    text data (called as BLOB). Now, the question is on
    choosing the language to develop the MTS component (called
    as App Controller), in VB/COM or ATL/COM ?

    >-----Original Message-----
    >Until another response is posted, I would go with VB only
    >because of future maintainability.
    >
    >Obviously I don't know your requirements.
    >
    >Is this file created from the webservice?, and after it's
    >creation you need to call a component to do something
    >with it?
    >
    >Or is the file a feed from a partner that's updated at
    >different times (outside of the web service)? If so you
    >might consider using an NT service and filewatcher.
    >
    >
    >>-----Original Message-----
    >>Hello,
    >>
    >>I am working on a B2B web serivce application. I am
    >>Systems Architect for this application. I am in the
    >>process of desigining my components. I would like to

    >know
    >>which one is better over the other for parsing 30MB
    >>files. We would like to use one of the following two
    >>choices: ATL/COM or VB/COM. Can you please share your
    >>thoughts on this ?
    >>
    >>Regards,
    >>Ramie
    >>.
    >>

    >.
    >
     
    Ramie, Oct 30, 2003
    #3
  4. Ramie

    Guest Guest

    Re: Which one is better for parsing 30MB files ? ATL/COM or VB/COM

    XML may not be the best method of moving 30mb of data across the internet,
    especially is most of it is a BLOB.

    Let me bounce this idea off you:

    1. Store your BLOB in a text or binary file, store any header or
    structured info in a smaller .xml or comma delimited file.

    2. ZIP the files on the client and then FTP to the server.

    3. Bulk load the data into SQL Server using DTS or BCP.

    WKidd


    "Ramie" <> wrote in message
    news:013601c39f01$74ca8940$...
    > Hello,
    >
    > I am working on a B2B web serivce application. I am
    > Systems Architect for this application. I am in the
    > process of desigining my components. I would like to know
    > which one is better over the other for parsing 30MB
    > files. We would like to use one of the following two
    > choices: ATL/COM or VB/COM. Can you please share your
    > thoughts on this ?
    >
    > Regards,
    > Ramie
     
    Guest, Oct 30, 2003
    #4
  5. Ramie

    Kline Sphere Guest

    Depends on the format of the files and what information you are
    parsing for.
    On Thu, 30 Oct 2003 08:18:29 -0800, "Ramie"
    <> wrote:

    >Hello,
    >
    >I am working on a B2B web serivce application. I am
    >Systems Architect for this application. I am in the
    >process of desigining my components. I would like to know
    >which one is better over the other for parsing 30MB
    >files. We would like to use one of the following two
    >choices: ATL/COM or VB/COM. Can you please share your
    >thoughts on this ?
    >
    >Regards,
    >Ramie
     
    Kline Sphere, Oct 30, 2003
    #5
  6. Ramie

    Kline Sphere Guest

    >Web
    >Service gets the input file (as big as 30MB) via HTTP
    >post. The user posts file to an ASP page. Then, the ASP
    >page will call MTS component, which will parse the file to
    >extract three piece of info: User Id, Password, and some
    >text data (called as BLOB). Now, the question is on
    >choosing the language to develop the MTS component (called
    >as App Controller), in VB/COM or ATL/COM ?


    Review your architectural design. I mean, 30 megs via a web service
    using asp, then processing through an mts component.....
     
    Kline Sphere, Oct 30, 2003
    #6
  7. Ramie

    Kline Sphere Guest

    Re: Which one is better for parsing 30MB files ? ATL/COM or VB/COM

    >2. ZIP the files on the client and then FTP to the server.

    Better still use a compressing stream, i.e.
    GZIPInputStream/GZIPOutputStream or the like of
     
    Kline Sphere, Oct 30, 2003
    #7
  8. Ramie

    Jay Walters Guest

    Re: Which one is better for parsing 30MB files ? ATL/COM or VB/COM

    Yeah - I've used Gzip and it's good. I serialized a large
    dataset using the binaryFormatter to a memorystream, get
    and Gzipped it, and converted it to a byte() to send it
    accross the wire and it worked great!


    >-----Original Message-----
    >>2. ZIP the files on the client and then FTP to the

    server.
    >
    >Better still use a compressing stream, i.e.
    >GZIPInputStream/GZIPOutputStream or the like of
    >.
    >
     
    Jay Walters, Oct 30, 2003
    #8
  9. Ramie

    Kline Sphere Guest

    Re: Which one is better for parsing 30MB files ? ATL/COM or VB/COM

    Cool!
    On Thu, 30 Oct 2003 11:19:47 -0800, "Jay Walters"
    <> wrote:

    >Yeah - I've used Gzip and it's good. I serialized a large
    >dataset using the binaryFormatter to a memorystream, get
    >and Gzipped it, and converted it to a byte() to send it
    >accross the wire and it worked great!
    >
    >
    >>-----Original Message-----
    >>>2. ZIP the files on the client and then FTP to the

    >server.
    >>
    >>Better still use a compressing stream, i.e.
    >>GZIPInputStream/GZIPOutputStream or the like of
    >>.
    >>
     
    Kline Sphere, Oct 30, 2003
    #9
  10. Ramie

    Ramie Guest

    Architecture for the Web Service is fixed, or shoud say,
    dictated by the user. We must use HTTP as the
    transportation protocol, and data is in ASCII (X12)
    format, which is encrypted before posted to the web
    service. Data is posted to an ASP page in multi-part
    binary post. Therefore, I do not have architecture design
    choice between HTTP and FTP.

    >-----Original Message-----
    >>Web
    >>Service gets the input file (as big as 30MB) via HTTP
    >>post. The user posts file to an ASP page. Then, the ASP
    >>page will call MTS component, which will parse the file

    to
    >>extract three piece of info: User Id, Password, and some
    >>text data (called as BLOB). Now, the question is on
    >>choosing the language to develop the MTS component

    (called
    >>as App Controller), in VB/COM or ATL/COM ?

    >
    >Review your architectural design. I mean, 30 megs via a

    web service
    >using asp, then processing through an mts component.....
    >.
    >
     
    Ramie, Oct 30, 2003
    #10
  11. Ramie

    Kline Sphere Guest

    >Architecture for the Web Service is fixed, or shoud say,
    >dictated by the user.


    Mad...

    >We must use HTTP as the
    >transportation protocol, and data is in ASCII (X12)
    >format, which is encrypted before posted to the web
    >service.


    Deflate it before it is sent. Having said that, If the number of times
    this 'process' gets used is small and operates through vpn or like of
    (thus not open to the whole wide world to bring your network to it's
    knees), it won't make too much difference.

    Is there no way this information can be 'parsed' at the client end?
    Thus only returning the relevant information.

    >Data is posted to an ASP page in multi-part
    >binary post.


    Seems a bit over kill when a simple socket will do the job.

    Processing the file in an MTS component is a very bad move. Operations
    conducted in as an environment should be completed in milliseconds, I
    think what you will be doing will take somewhat longer than that. In
    any case given the time the file may take to 'download', the choice of
    programming language used to parse the file will not make too much
    difference, although my choice for COM programming would always be ATL

    >Therefore, I do not have architecture design
    >choice between HTTP and FTP.


    An encrypted socket would be faster, but if you're stuck to using what
    some dumb client reckons, oh well could be worse - I think!
     
    Kline Sphere, Oct 30, 2003
    #11
  12. Sorry, but coding it in VB won't give you future maintainability. VB
    won't be around in 20 years but C++ will always be around.

    Future Jay Walters wrote:

    > Until another response is posted, I would go with VB only
    > because of future maintainability.
    >
    > Obviously I don't know your requirements.
    >
    > Is this file created from the webservice?, and after it's
    > creation you need to call a component to do something
    > with it?
    >
    > Or is the file a feed from a partner that's updated at
    > different times (outside of the web service)? If so you
    > might consider using an NT service and filewatcher.
    >
    >
    >
    >>-----Original Message-----
    >>Hello,
    >>
    >>I am working on a B2B web serivce application. I am
    >>Systems Architect for this application. I am in the
    >>process of desigining my components. I would like to

    >
    > know
    >
    >>which one is better over the other for parsing 30MB
    >>files. We would like to use one of the following two
    >>choices: ATL/COM or VB/COM. Can you please share your
    >>thoughts on this ?
    >>
    >>Regards,
    >>Ramie
    >>.
    >>
     
    General Protection Fault, Oct 30, 2003
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Coug

    Help Needed with a file called ATL.DLL

    Coug, Oct 20, 2003, in forum: Computer Support
    Replies:
    11
    Views:
    2,905
    gangle
    Oct 21, 2003
  2. Simon Cooper

    Where do I get to my 30Mb webspace?

    Simon Cooper, Nov 20, 2003, in forum: Computer Support
    Replies:
    7
    Views:
    490
    Boomer
    Nov 20, 2003
  3. Mike  Holtom

    atl.dll error

    Mike Holtom, Dec 16, 2004, in forum: Computer Support
    Replies:
    2
    Views:
    4,261
    Lady Chatterly
    Dec 16, 2004
  4. Giuen
    Replies:
    0
    Views:
    1,163
    Giuen
    Sep 12, 2008
  5. William Jones

    Sandisk ESP or 30mb/sec - Which one is faster?

    William Jones, Mar 20, 2009, in forum: Digital Photography
    Replies:
    2
    Views:
    323
    William Jones
    Mar 21, 2009
Loading...

Share This Page