Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   ASP .Net (http://www.velocityreviews.com/forums/f29-asp-net.html)
-   -   Flat files and ASP.NET (http://www.velocityreviews.com/forums/t533666-flat-files-and-asp-net.html)

=?Utf-8?B?TmF2YW5lZXRoLksuTg==?= 08-30-2007 07:06 AM

Flat files and ASP.NET
 
Hello

I am developing a system for user tracking. In this I am tracking all the
visitors that came to my website. i developed this using a HTTPModule and
storing the data in a SQL Server database. Everything worked fine till my
website gets good traffic.

When website is getting good traffic, this method is slowing down the entire
application. So I am planning to use flat files instead of SQL Server
database. My doubts are

1 - What is a flat file and how can I connect to it from my ASP.NET
application
2 - If multiple users are browsing website at same time, how can I save
data's to file, because that may be in use.
3 - Is XML files are normal flat files ?

Please help me on this issue.

Sincerly
Navaneeth

marss 08-30-2007 07:19 AM

Re: Flat files and ASP.NET
 
On 30 , 10:06, Navaneeth.K.N
<Navaneeth....@discussions.microsoft.com> wrote:
> Hello
>
> I am developing a system for user tracking. In this I am tracking all the
> visitors that came to my website. i developed this using a HTTPModule and
> storing the data in a SQL Server database. Everything worked fine till my
> website gets good traffic.
>
> When website is getting good traffic, this method is slowing down the entire
> application. So I am planning to use flat files instead of SQL Server
> database. My doubts are
>
> 1 - What is a flat file and how can I connect to it from my ASP.NET
> application
> 2 - If multiple users are browsing website at same time, how can I save
> data's to file, because that may be in use.
> 3 - Is XML files are normal flat files ?
>
> Please help me on this issue.
>
> Sincerly
> Navaneeth


I have a doubt whether saving data to a file on the hard drive is
faster then saving data in the database. Not to mention the speed of
processing these data.

Regards,
Mykola
http://marss.co.ua


Mark Rae [MVP] 08-30-2007 07:28 AM

Re: Flat files and ASP.NET
 
"Navaneeth.K.N" <Navaneeth.K.N@discussions.microsoft.com> wrote in message
news:B93CFF77-9E87-46F3-AD4A-8F776B434424@microsoft.com...

> So I am planning to use flat files instead of SQL Server database.


??? Surely not...

> 2 - If multiple users are browsing website at same time, how can I save
> data's to file, because that may be in use.


You can't...

> 3 - Is XML files are normal flat files ?


XML files are text files, specifically designed as a lingua franca of data
transmission between different systems - they should not be used as a
substitute for a database...

You need to look at optimising your database and how you are connecting to
it - that's where the bottleneck is...


--
Mark Rae
ASP.NET MVP
http://www.markrae.net


Steve C. Orr [MCSD, MVP, CSM, ASP Insider] 08-30-2007 07:48 AM

Re: Flat files and ASP.NET
 
I suggest you stick with a database. Flat files will almost certainly not
give you additional speed or scalability.
Perhaps you could use some sort of a queue to save up log records and only
write to the database every so often.

--
I hope this helps,
Steve C. Orr,
MCSD, MVP, CSM, ASPInsider
http://SteveOrr.net



"Navaneeth.K.N" <Navaneeth.K.N@discussions.microsoft.com> wrote in message
news:B93CFF77-9E87-46F3-AD4A-8F776B434424@microsoft.com...
> Hello
>
> I am developing a system for user tracking. In this I am tracking all the
> visitors that came to my website. i developed this using a HTTPModule and
> storing the data in a SQL Server database. Everything worked fine till my
> website gets good traffic.
>
> When website is getting good traffic, this method is slowing down the
> entire
> application. So I am planning to use flat files instead of SQL Server
> database. My doubts are
>
> 1 - What is a flat file and how can I connect to it from my ASP.NET
> application
> 2 - If multiple users are browsing website at same time, how can I save
> data's to file, because that may be in use.
> 3 - Is XML files are normal flat files ?
>
> Please help me on this issue.
>
> Sincerly
> Navaneeth



Aidy 08-30-2007 08:45 AM

Re: Flat files and ASP.NET
 
Yes, the OP could look into MSMQ, however what levels of traffic are we
talking here? If it isn't extremely heavy I can't understand why SQL would
be causing a bottleneck. I'd look into what is causing the delay before
looking for an alternative solution.

"Steve C. Orr [MCSD, MVP, CSM, ASP Insider]" <Steve@Orr.net> wrote in
message news:B7C80C83-25C3-4BF6-BE90-F75E547DC977@microsoft.com...
>I suggest you stick with a database. Flat files will almost certainly not
>give you additional speed or scalability.
> Perhaps you could use some sort of a queue to save up log records and only
> write to the database every so often.
>
> --
> I hope this helps,
> Steve C. Orr,
> MCSD, MVP, CSM, ASPInsider
> http://SteveOrr.net
>
>
>
> "Navaneeth.K.N" <Navaneeth.K.N@discussions.microsoft.com> wrote in message
> news:B93CFF77-9E87-46F3-AD4A-8F776B434424@microsoft.com...
>> Hello
>>
>> I am developing a system for user tracking. In this I am tracking all the
>> visitors that came to my website. i developed this using a HTTPModule and
>> storing the data in a SQL Server database. Everything worked fine till my
>> website gets good traffic.
>>
>> When website is getting good traffic, this method is slowing down the
>> entire
>> application. So I am planning to use flat files instead of SQL Server
>> database. My doubts are
>>
>> 1 - What is a flat file and how can I connect to it from my ASP.NET
>> application
>> 2 - If multiple users are browsing website at same time, how can I save
>> data's to file, because that may be in use.
>> 3 - Is XML files are normal flat files ?
>>
>> Please help me on this issue.
>>
>> Sincerly
>> Navaneeth

>




Mark Rae [MVP] 08-30-2007 08:53 AM

Re: Flat files and ASP.NET
 
"Aidy" <aidy@xxnoemailxx.com> wrote in message
news:8tWdnVfdNph3HkvbnZ2dnUVZ8seinZ2d@bt.com...

> If it isn't extremely heavy I can't understand why SQL would be causing a
> bottleneck.


Could be any number of reasons: poor design, no primary key / index,
inefficient use of ADO.NET, not closing connections etc...


--
Mark Rae
ASP.NET MVP
http://www.markrae.net


Aidy 08-30-2007 08:58 AM

Re: Flat files and ASP.NET
 
> Could be any number of reasons: poor design, no primary key / index,
> inefficient use of ADO.NET, not closing connections etc...


That's what I meant, I don't see that it can be SQL Server's fault but
something to do with the design/coding.

BTW, why would not having an index *harm* insert performance? :p



John Timney \(MVP\) 08-30-2007 08:59 AM

Re: Flat files and ASP.NET
 
A flat file likely won't help you - ultimately your database does nothing
more than write data to a file. XML is not he right approach either.

You need to either change your approach to tracking data, using perhaps a
hidden iframe in you pages that impacts a second web server instead of your
primary - or look to scale out for load by adding web servers to create a
farm.

Ultimately, a fix is a design decision as your problem could be in the
module rather than he fact that your talking to a database, you could try to
spawn your write activities off to asynchronous threads and see if it
reduces your problems or simply load your data into the application object
and every 1000 requests or so write it to the DB.

Regards

John Timney (MVP)
http://www.johntimney.com
http://www.johntimney.com/blog


"Navaneeth.K.N" <Navaneeth.K.N@discussions.microsoft.com> wrote in message
news:B93CFF77-9E87-46F3-AD4A-8F776B434424@microsoft.com...
> Hello
>
> I am developing a system for user tracking. In this I am tracking all the
> visitors that came to my website. i developed this using a HTTPModule and
> storing the data in a SQL Server database. Everything worked fine till my
> website gets good traffic.
>
> When website is getting good traffic, this method is slowing down the
> entire
> application. So I am planning to use flat files instead of SQL Server
> database. My doubts are
>
> 1 - What is a flat file and how can I connect to it from my ASP.NET
> application
> 2 - If multiple users are browsing website at same time, how can I save
> data's to file, because that may be in use.
> 3 - Is XML files are normal flat files ?
>
> Please help me on this issue.
>
> Sincerly
> Navaneeth




Mark Rae [MVP] 08-30-2007 09:00 AM

Re: Flat files and ASP.NET
 
"Aidy" <aidy@xxnoemailxx.com> wrote in message
news:lNKdncQgZNVaG0vbRVnyiwA@bt.com...

> BTW, why would not having an index *harm* insert performance? :p


:-)


--
Mark Rae
ASP.NET MVP
http://www.markrae.net


Niels Ull 08-30-2007 02:24 PM

Re: Flat files and ASP.NET
 
> Yes, the OP could look into MSMQ, however what levels of traffic are
> we talking here? If it isn't extremely heavy I can't understand why
> SQL would be causing a bottleneck. I'd look into what is causing the
> delay before looking for an alternative solution.


I definitely concur with analysing before fixing.

However, it is very likely that a suitably buffered flat file writer will
be faster than committing individual log entries to a database.

Each thread could have its own buffered logger which only flushed to a common
synchronized disk file logger when the buffer is full.
If log entries are timestamped, the ordering of log messages are not important.

Of course, this does run the risk of data loss in case of system failure.

If it is undesirable to have even small waits when flushing to the synchronized
logger, the logger could also be buffered with a separate thread to flush
it to disk.

MSMQ does seem like a bit overkill here, unlesss you really need transactional
support, distribution etc.

XML is not a very suitable format for log files, since they require all content
to be wrapped in a single element. It would, however,
be possible to have a log file containing lots of xml fragments.

Good luck!

Niels




All times are GMT. The time now is 06:31 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.