reading text-file with very long lines
 

Hello!

I'm trying to read a file line by line with following code-snippet:

-----------------------------**-------------------------------
BufferedReader test = new BufferedReader(new FileReader(datei));
String tmp = "";
Vector data = new Vector();
boolean first = true;
int breaker = 0;
int position = 0;
while ((tmp = test.readLine()) != null) {
if (!first) data.addElement(tmp); else first = false;
if (tmp.contains("%%EOF")) breaker = position;
position++;
}
while (data.size() > breaker) data.remove(data.size()-1);
-----------------------------**-------------------------------

there are many "%%EOF" - lines within this file. I need to get the
last one and this is my solution to get the whole data from the second
line to the last line with "%%EOF"

But I encountered some problems. Some lines are very long, i.e. more
than 8000 characters (varied, depends on the file). When reading the
file using the above code I only get aprox. 1520 characters and the
remaing characters are lost and reading stops with no error at all.
So I need some other solution to read my file. Keep in mind, that I
need to compare strings...

Does anyone know any (nice) solution?
Thanks for your help.
McGregor

Re: reading text-file with very long lines
 

McGregor <j.koerte@odd.de> writes:
>file using the above code I only get aprox. 1520 characters and the

If replying to my post, please do not quote my complete
post, but only those parts you directly refer to.
Especially, please do not quote the complete source code.

public class Main
{
static final int NUMBER_OF_LINES = 100;
static final int MAXIMUM_LINE_LENGTH = 64000;
static final int MINIMUM_LINE_LENGTH = 1521;

public static void write( final java.lang.String[] args )
throws java.lang.Throwable
{
final java.io.PrintWriter out = new java.io.PrintWriter
( new java.io.FileOutputStream( "tmp.txt" ), true );
for( int l = 0; l < NUMBER_OF_LINES; ++l )
{ java.lang.System.err.println( "writing line " + l );
out.print( l );
out.print( ' ' );
if( java.lang.Math.random() < 0.1 )out.print( "%%EOF" );
final int length =MINIMUM_LINE_LENGTH +
( int )
( java.lang.Math.random() *
( 1 + MAXIMUM_LINE_LENGTH - MINIMUM_LINE_LENGTH ));
for( int p = 0; p < length; ++p )out.print( "x" );
out.print( "\n" ); }}

public static void read( final java.lang.String[] args )
throws java.lang.Throwable
{
final java.util.Scanner scanner = new java.util.Scanner
( new java.io.File( "tmp.txt" ));
java.lang.String result = null;
int l = 0;
while( scanner.hasNextLine() )
{ java.lang.System.err.println( "reading line " + l++ );
final java.lang.String line = scanner.nextLine();
if( line.contains( "%%EOF" ))result = line; }
if( result != null )
{ java.lang.System.out.printf( "%.78s%n", result );
java.lang.System.out.println( result.length() ); }
else
{ java.lang.System.out.println( result ); }}

public static void main( final java.lang.String[] args )
throws java.lang.Throwable
{
write( args );
read( args ); }}

94 %%EOFxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxx
9877

Re: reading text-file with very long lines
 

On Thu, 29 Jan 2009, McGregor wrote:

> I'm trying to read a file line by line with following code-snippet:
>
> -----------------------------**-------------------------------
> BufferedReader test = new BufferedReader(new FileReader(datei));
> String tmp = "";
> Vector data = new Vector();
> boolean first = true;
> int breaker = 0;
> int position = 0;
> while ((tmp = test.readLine()) != null) {
> if (!first) data.addElement(tmp); else first = false;
> if (tmp.contains("%%EOF")) breaker = position;
> position++;
> }
> while (data.size() > breaker) data.remove(data.size()-1);
> -----------------------------**-------------------------------
>
> there are many "%%EOF" - lines within this file. I need to get the
> last one and this is my solution to get the whole data from the second
> line to the last line with "%%EOF"
>
> But I encountered some problems. Some lines are very long, i.e. more
> than 8000 characters (varied, depends on the file). When reading the
> file using the above code I only get aprox. 1520 characters and the
> remaing characters are lost and reading stops with no error at all.

Odd. Could you try running this test program:

http://urchin.earth.li/~twic/tmp/LongLines.java

?

If there are any problems with reading lines up to 8000 characters long,
it should fail with an IOException. On my machine (java 1.5.0_16), it
completes successfully.

The thing is, your code looks like it should work to me. It's not quite
how i would have written it, but it ought to work.

> So I need some other solution to read my file. Keep in mind, that I
> need to compare strings...

Here's how i'd do it:

public static List<String> readToLastMarker(BufferedReader in, String marker) throws IOException {
List<String> acceptedLines = new ArrayList<String>();
List<String> candidateLines = new ArrayList<String>();
String firstLine = in.readLine(); // ignore the first line
String line;
while ((line = in.readLine()) != null) {
if (line.contains(marker)) {
acceptedLines.addAll(candidateLines);
candidateLines.clear();
candidateLines.add(line); // do you really want to do this?
}
else {
candidateLines.add(line);
}
}
return acceptedLines;
}

tom

--
I am the best at what i do.