Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > reading text-file with very long lines

Reply
Thread Tools

reading text-file with very long lines

 
 
McGregor
Guest
Posts: n/a
 
      01-29-2009
Hello!

I'm trying to read a file line by line with following code-snippet:

-----------------------------**-------------------------------
BufferedReader test = new BufferedReader(new FileReader(datei));
String tmp = "";
Vector data = new Vector();
boolean first = true;
int breaker = 0;
int position = 0;
while ((tmp = test.readLine()) != null) {
if (!first) data.addElement(tmp); else first = false;
if (tmp.contains("%%EOF")) breaker = position;
position++;
}
while (data.size() > breaker) data.remove(data.size()-1);
-----------------------------**-------------------------------

there are many "%%EOF" - lines within this file. I need to get the
last one and this is my solution to get the whole data from the second
line to the last line with "%%EOF"

But I encountered some problems. Some lines are very long, i.e. more
than 8000 characters (varied, depends on the file). When reading the
file using the above code I only get aprox. 1520 characters and the
remaing characters are lost and reading stops with no error at all.
So I need some other solution to read my file. Keep in mind, that I
need to compare strings...

Does anyone know any (nice) solution?
Thanks for your help.
McGregor
 
Reply With Quote
 
 
 
 
Stefan Ram
Guest
Posts: n/a
 
      01-29-2009
McGregor <(E-Mail Removed)> writes:
>file using the above code I only get aprox. 1520 characters and the


If replying to my post, please do not quote my complete
post, but only those parts you directly refer to.
Especially, please do not quote the complete source code.

public class Main
{
static final int NUMBER_OF_LINES = 100;
static final int MAXIMUM_LINE_LENGTH = 64000;
static final int MINIMUM_LINE_LENGTH = 1521;

public static void write( final java.lang.String[] args )
throws java.lang.Throwable
{
final java.io.PrintWriter out = new java.io.PrintWriter
( new java.io.FileOutputStream( "tmp.txt" ), true );
for( int l = 0; l < NUMBER_OF_LINES; ++l )
{ java.lang.System.err.println( "writing line " + l );
out.print( l );
out.print( ' ' );
if( java.lang.Math.random() < 0.1 )out.print( "%%EOF" );
final int length =MINIMUM_LINE_LENGTH +
( int )
( java.lang.Math.random() *
( 1 + MAXIMUM_LINE_LENGTH - MINIMUM_LINE_LENGTH ));
for( int p = 0; p < length; ++p )out.print( "x" );
out.print( "\n" ); }}

public static void read( final java.lang.String[] args )
throws java.lang.Throwable
{
final java.util.Scanner scanner = new java.util.Scanner
( new java.io.File( "tmp.txt" ));
java.lang.String result = null;
int l = 0;
while( scanner.hasNextLine() )
{ java.lang.System.err.println( "reading line " + l++ );
final java.lang.String line = scanner.nextLine();
if( line.contains( "%%EOF" ))result = line; }
if( result != null )
{ java.lang.System.out.printf( "%.78s%n", result );
java.lang.System.out.println( result.length() ); }
else
{ java.lang.System.out.println( result ); }}

public static void main( final java.lang.String[] args )
throws java.lang.Throwable
{
write( args );
read( args ); }}

94 %%EOFxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxx
9877

 
Reply With Quote
 
 
 
 
Tom Anderson
Guest
Posts: n/a
 
      01-29-2009
On Thu, 29 Jan 2009, McGregor wrote:

> I'm trying to read a file line by line with following code-snippet:
>
> -----------------------------**-------------------------------
> BufferedReader test = new BufferedReader(new FileReader(datei));
> String tmp = "";
> Vector data = new Vector();
> boolean first = true;
> int breaker = 0;
> int position = 0;
> while ((tmp = test.readLine()) != null) {
> if (!first) data.addElement(tmp); else first = false;
> if (tmp.contains("%%EOF")) breaker = position;
> position++;
> }
> while (data.size() > breaker) data.remove(data.size()-1);
> -----------------------------**-------------------------------
>
> there are many "%%EOF" - lines within this file. I need to get the
> last one and this is my solution to get the whole data from the second
> line to the last line with "%%EOF"
>
> But I encountered some problems. Some lines are very long, i.e. more
> than 8000 characters (varied, depends on the file). When reading the
> file using the above code I only get aprox. 1520 characters and the
> remaing characters are lost and reading stops with no error at all.


Odd. Could you try running this test program:

http://urchin.earth.li/~twic/tmp/LongLines.java

?

If there are any problems with reading lines up to 8000 characters long,
it should fail with an IOException. On my machine (java 1.5.0_16), it
completes successfully.

The thing is, your code looks like it should work to me. It's not quite
how i would have written it, but it ought to work.

> So I need some other solution to read my file. Keep in mind, that I
> need to compare strings...


Here's how i'd do it:

public static List<String> readToLastMarker(BufferedReader in, String marker) throws IOException {
List<String> acceptedLines = new ArrayList<String>();
List<String> candidateLines = new ArrayList<String>();
String firstLine = in.readLine(); // ignore the first line
String line;
while ((line = in.readLine()) != null) {
if (line.contains(marker)) {
acceptedLines.addAll(candidateLines);
candidateLines.clear();
candidateLines.add(line); // do you really want to do this?
}
else {
candidateLines.add(line);
}
}
return acceptedLines;
}

tom

--
I am the best at what i do.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Having compilation error: no match for call to (const __gnu_cxx::hash<long long int>) (const long long int&) veryhotsausage C++ 1 07-04-2008 05:41 PM
m// on very long lines leaks memory ShaunJ Perl Misc 5 03-13-2008 11:24 PM
very very very long integer shanx__=|;- C Programming 19 10-19-2004 03:55 PM
very very very long integer Abhishek Jha C Programming 4 10-17-2004 08:19 AM
Quick Book file access very very very slow Thomas Reed Computer Support 7 04-09-2004 08:09 PM



Advertisments