Jump to content

  • Log In with Google      Sign In   
  • Create Account


Exception in thread "Thread-4" java.lang.OutOfMemoryError: Java heap space


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
2 replies to this topic

#1 Jarwulf   Members   -  Reputation: 222

Like
0Likes
Like

Posted 08 November 2012 - 04:57 PM

Hi, I'm getting a crash in Java. Basically a txt file circa 150mb is being processed


fileHandler.shrinkFile();


goes into



public void shrinkFile() throws FileNotFoundException, IOException, FileFormatException{
		BufferedWriter outputTempFileWriter = null;
		BufferedReader inputFileReader = null;
	  
		  
			for(int i=0;i<choseFilesVector.size();i++){
				String inputFile = choseFilesVector.get(i).toString();  
				inputFileReader = new BufferedReader(new FileReader( inputFile ), readBufferSize);
			  
				txtArea.append("  Processing "+inputFile+"\n");
				txtArea.setCaretPosition(txtArea.getText().length());
							  
				//decide format
				String strLine = null;
				String format = null;
				do{

//CRASH HAPPENS HERE
					strLine = inputFileReader.readLine();	  
//CRASH HAPPENS HERE
			  
					if(strLine!=null && (strLine.split("\t").length>=4)){  
						String [] info = strLine.split("\t");
						if((info[0].trim().toLowerCase().matches("chr(x|y|[0-9]+)"))&&(info[1].trim().matches("[0-9]+"))&&(info[2].trim().matches("[0-9]+"))){
							//data of tagAlign
							format = "tagAlign";

It gets into the above loop and then simply crashes on line
strLine = inputFileReader.readLine();


the first time through giving me the error

Exception in thread "Thread-4" java.lang.OutOfMemoryError: Java heap space
	at java.util.Arrays.copyOf(Unknown Source)
	at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)

etc etc. Maybe the file I'm processing is too big but the program has happened gigantic files before the only difference this time is that the files being processed were autoconverted from a different format.

Sponsor:

#2 Bubsy   Members   -  Reputation: 407

Like
1Likes
Like

Posted 09 November 2012 - 06:52 AM

Actually its not because the file size, but I bet this new file has a gigantic line on it.

The readLine() method will try and allocate the full line as a string in memory. If the line of text gets to big, you'll eventually run out of heap.

There's two courses of action that you can take :
  • Raise the amount of memory avaliable to the JVM (It'll work, but eventually you may run on OutOfMemory erros again)
  • Refactor the code without relying on the readLine() method. (Instead of reading the full string and spliting it on memory, you should create a FSM and parse the file on a more "streaming" fashion)


#3 Jarwulf   Members   -  Reputation: 222

Like
0Likes
Like

Posted 09 November 2012 - 07:30 PM

Thanks, I added a newline when file generating and it seems to have worked.




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS