Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > any datatype which can handle GB's of Data.

Reply
Thread Tools

any datatype which can handle GB's of Data.

 
 
Garg
Guest
Posts: n/a
 
      04-23-2007
Hi All,

Is there any variable in java like Stringbuffer which can handle GB's
of char data.

As far as i know every variable has a limit is there any variable
which can handle unlimited data.

it sounds little bit stupid but which data type or design pattern can
solve this problem.

Thanks
Taurn Garg

 
Reply With Quote
 
 
 
 
Daniel Pitts
Guest
Posts: n/a
 
      04-23-2007
On Apr 23, 6:03 am, Garg <(E-Mail Removed)> wrote:
> Hi All,
>
> Is there any variable in java like Stringbuffer which can handle GB's
> of char data.
>
> As far as i know every variable has a limit is there any variable
> which can handle unlimited data.


No *computer* can handle unlimitted data, so how could any type in
Java do so?

In general, you are limitted in any programming language by how much
memory (virtual or otherwise) you have. Sometimes there are other
limits, but most people don't have enough memory to hit those other
limits.

Its generally better to accept this, and use algorithms that support
some sort of streaming or subset processing.
>
> it sounds little bit stupid but which data type or design pattern can
> solve this problem.


Its not a data type or design pattern. In this case, its an algorithm
or data structure.


Hope this helps,
Daniel.

 
Reply With Quote
 
 
 
 
Robert Klemme
Guest
Posts: n/a
 
      04-23-2007
On 23.04.2007 15:03, Garg wrote:
> Is there any variable in java like Stringbuffer which can handle GB's
> of char data.


In memory? On a 32 bit Windows system a JRE can use approx. 1.5GB heap.

> As far as i know every variable has a limit is there any variable
> which can handle unlimited data.
>
> it sounds little bit stupid but which data type or design pattern can
> solve this problem.


The question is: what do you want to do with that data? Chances are
that you can only deal with this data on disk or in a database.

robert
 
Reply With Quote
 
David Zimmerman
Guest
Posts: n/a
 
      04-23-2007


Daniel Pitts wrote:
> On Apr 23, 6:03 am, Garg <(E-Mail Removed)> wrote:
>> Hi All,
>>
>> Is there any variable in java like Stringbuffer which can handle GB's
>> of char data.
>>

> No *computer* can handle unlimitted data, so how could any type in
> Java do so?
>


Some objects such as java.net.Socket or java.io.OutputStream can handle
unlimited data...

But that's probably not what the OP meant
 
Reply With Quote
 
Mark Rafn
Guest
Posts: n/a
 
      04-23-2007
Garg <(E-Mail Removed)> wrote:
>Is there any variable in java like Stringbuffer which can handle GB's
>of char data.


StringBuffer is a class, not a variable. I don't mean this just as a nitpick;
the way you describe a problem has a very large affect on the types of
answers you'll get.

If you're imprecise in describing what you're doing, and unclear in your
description of the Java language, you'll get general answers that assume
you're a fairly inexperienced developer. Which may be what you're looking
for

>As far as i know every variable has a limit is there any variable
>which can handle unlimited data.


Every computer has a limit. AFAIK, Java's limits for this kind of class are
based on available memory and the fact that arrays must be indexable by a
32-bit signed integer. I'll be surprised if someone says there's no JVM on
any hardware that can handle 1.8 billion characters in a StringBuilder.

But then you get into WHY you want to do such a thing.

>it sounds little bit stupid but which data type or design pattern can
>solve this problem.


Generally, look for ways to avoid having it all in memory at the same time.
If possible, a streaming approach is good: read from a file/socket a little at
a time, process it, and write to a file/socket. Alternately,
java.io.RandomAccessFile lets you seek() to the section you want to work on.
In both cases, you'll need to be careful about the distinction between bytes
and characters.
--
Mark Rafn http://www.velocityreviews.com/forums/(E-Mail Removed) <http://www.dagon.net/>

 
Reply With Quote
 
AdrianWilsonTS@gmail.com
Guest
Posts: n/a
 
      04-23-2007
On Apr 23, 9:03 am, Garg <(E-Mail Removed)> wrote:
> Hi All,
>
> Is there any variable in java like Stringbuffer which can handle GB's
> of char data.
>
> As far as i know every variable has a limit is there any variable
> which can handle unlimited data.
>
> it sounds little bit stupid but which data type or design pattern can
> solve this problem.
>
> Thanks
> Taurn Garg



I can create such a variable for you - for a slight fee!
Seriously, you can use a File (or even memory mapped file) and just
put a delegate of some sort. E.g.

public class BigBuf {
// herein are the public external methods that look just like a
StringBuffer or what have you
// herein is the underlieing implementation that actually write/reads
to/from a file.
}

 
Reply With Quote
 
Garg
Guest
Posts: n/a
 
      04-24-2007
thanks for reply,

What i really need to do is that that i have to take the data from
database and then send it to one JSP page where javascript will handle
that data to show to the user.
the reading from database and sending to jsp or gui is been done ny
serlvet.

so what i am doing is like i am reading the data from database one by
one record and then creating a table and puting that in the
StringBuffer. but as data can be of any size. String buffer will not
be able to handle that. so in this case what should i do.

thanks
tarun


 
Reply With Quote
 
Lew
Guest
Posts: n/a
 
      04-24-2007
Garg wrote:
> thanks for reply,
>
> What i really need to do is that that i have to take the data from
> database and then send it to one JSP page where javascript will handle
> that data to show to the user.
> the reading from database and sending to jsp or gui is been done ny
> serlvet.
>
> so what i am doing is like i am reading the data from database one by
> one record and then creating a table and puting that in the
> StringBuffer. but as data can be of any size. String buffer will not
> be able to handle that. so in this case what should i do.


Don't.

Separate String (or StringBuilder or other type such as java.sql.Date) for
each column in each row (not record). Don't show all gazillion rows (not
records) at once - use some paging.

The datatype to hold everything at once is the "RDBMS" type. Everything else
should only hold a view into that. Certainly even if your server could suck
the whole data store into memory at once, there's no way someone's browser can.

--
Lew
 
Reply With Quote
 
Michael
Guest
Posts: n/a
 
      04-24-2007
On Apr 23, 9:03 am, Garg <(E-Mail Removed)> wrote:
> Hi All,
>
> Is there any variable in java like Stringbuffer which can handle GB's
> of char data.
>
> As far as i know every variable has a limit is there any variable
> which can handle unlimited data.
>
> it sounds little bit stupid but which data type or design pattern can
> solve this problem.
>
> Thanks
> Taurn Garg


Research the java.nio.ByteBuffer* classes such - specifically the
MemoryMapped (google is your friend). Basically disk-backed buffers
that you can perform many convenient slicing on which is more
efficient than sequentially navigating a file. If you've got the RAM,
then the OS should properly cache the contents of the file.


 
Reply With Quote
 
Daniel Pitts
Guest
Posts: n/a
 
      04-25-2007
On Apr 24, 4:42 am, Garg <(E-Mail Removed)> wrote:
> thanks for reply,
>
> What i really need to do is that that i have to take the data from
> database and then send it to one JSP page where javascript will handle
> that data to show to the user.
> the reading from database and sending to jsp or gui is been done ny
> serlvet.
>
> so what i am doing is like i am reading the data from database one by
> one record and then creating a table and puting that in the
> StringBuffer. but as data can be of any size. String buffer will not
> be able to handle that. so in this case what should i do.
>
> thanks
> tarun


First, you should use StringBuilder over StringBuffer in most cases.
Second, What makes you think StringBuilder can't handle it? It can
handle a surprising amount of data. You don't want to load all of
your database at once though. Try displaying a row at a time.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
database datatype bit what is the C# datatype? Jeff ASP .Net 2 04-16-2007 08:26 PM
incrementing HH:MM:SS information: any method or datatype to save me time in .NET? ASP .Net 3 02-13-2007 11:00 PM
501 PIX "deny any any" "allow any any" Any Anybody? Networking Student Cisco 4 11-16-2006 10:40 PM
How does union datatype handle leading/trailing white space? Leo XML 0 08-31-2004 03:26 PM
Can the XML <any> element restrict what datatype it stores? Meredith XML 0 11-07-2003 08:32 PM



Advertisments