Jump to content

  • Log In with Google      Sign In   
  • Create Account

Close file sooner or later?


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
13 replies to this topic

#1 ultramailman   Prime Members   -  Reputation: 1585

Like
0Likes
Like

Posted 22 February 2013 - 02:50 AM

When I need to use a file, I would open it, copy the contents into a string, and close the file. Then I feel like I can do whatever I want with the string.

What if the file is not closed until the very end? What advantage would that give? Will there be performance penalties?

Say I open this file in write mode, and not close it for some time. Would that lock the file so that the user cannot change the file with a text editor?

If you'd like, share your file handling habits too, I'd like to know how other people do it.

Sponsor:

#2 santa01   Members   -  Reputation: 307

Like
0Likes
Like

Posted 22 February 2013 - 04:52 AM

I'd recommend sooner, Damian Conway - Perl Best Practices also recommend such approach, I guess its valid for other languages too. The only option for not releasing any resource (after its not required any more) is when the release process itself consumes much resources. For instance any garbage collector delays memory releasing because it could be cpu and time consuming.

 

 

What if the file is not closed until the very end?

 

for example any unhanded exception will probably halt your program and any buffered data won't be flushed to a file.


Edited by santa01, 22 February 2013 - 04:52 AM.


#3 markr   Crossbones+   -  Reputation: 1653

Like
0Likes
Like

Posted 22 February 2013 - 04:59 AM

You asked "would this lock the file"? You're referring to mandatory locking which is enforced on some OSs in some cases (mainly DOS / Windows). It is not a good idea to use this as a "feature", in my opinion.

 

The only benefit of leaving a file open is that you don't have to open it again if you need it later. If this doesn't fit your use-case, then you probably want to close it as soon as possible.

 

It's normally easier to close files as soon as possible, because you don't end up with bad "side effects" - what if someone instantiates several of your objects at once? What happens 1 million get created at once?

 

The OS can run out of file handles, or you could "forget" to close them and create a leak. You probably normally want to close the file when you're finished.

 

Closing and opening files is normally a fairly efficient operation, particularly if the file has been accessed recently, and its metadata are in an OS cache.



#4 swiftcoder   Senior Moderators   -  Reputation: 10364

Like
0Likes
Like

Posted 22 February 2013 - 09:22 AM

What if the file is not closed until the very end?

If you open enough files, you run out of file handles.

 

Certain systems set this fairly low (default config for Mac OS X tops out at ~10,000 file handles per process), though older OS were typically for more restrictive (sometimes as low as 256).


Tristam MacDonald - Software Engineer @Amazon - [swiftcoding]


#5 Bubsy   Members   -  Reputation: 407

Like
3Likes
Like

Posted 22 February 2013 - 09:27 AM

@swiftcoder Fairly high you mean wink.png

 

You should close as soon as possible. I guess it's best to use the RAII pattern to manage the file handler lifecycle, 



#6 ultramailman   Prime Members   -  Reputation: 1585

Like
0Likes
Like

Posted 22 February 2013 - 11:39 AM

You asked "would this lock the file"? You're referring to mandatory locking which is enforced on some OSs in some cases (mainly DOS / Windows). It is not a good idea to use this as a "feature", in my opinion.

 

Oh, so it's not portable "feature" then. I thought it would be a nice "feature" for some important files, for example the settings file, or map file, etc.

 

How about using abort() ? If I recall correctly, abort will not cause open and written files to be saved. Can that also be another "feature" to be used?

 

Seems like everyone is leaning towards closing it as soon as possible. Then I have to ask, is it common to parse a file while reading it? I've always copied files to strings first, but that just seems like one wasted pass.



#7 swiftcoder   Senior Moderators   -  Reputation: 10364

Like
1Likes
Like

Posted 22 February 2013 - 12:17 PM

Seems like everyone is leaning towards closing it as soon as possible. Then I have to ask, is it common to parse a file while reading it? I've always copied files to strings first, but that just seems like one wasted pass.

As soon as possible means "as soon as you are done reading/writing". No one is suggesting that your modify your reading routines to speed up how soon you close a file...


Tristam MacDonald - Software Engineer @Amazon - [swiftcoding]


#8 ultramailman   Prime Members   -  Reputation: 1585

Like
0Likes
Like

Posted 22 February 2013 - 12:46 PM

... suggesting that your modify your...

 

Oh no, I didn't mean to imply that. I am honestly just wondering how others parse their files, either parse while reading or parse after copying to string and closing file.



#9 swiftcoder   Senior Moderators   -  Reputation: 10364

Like
2Likes
Like

Posted 22 February 2013 - 12:56 PM

Oh no, I didn't mean to imply that. I am honestly just wondering how others parse their files, either parse while reading or parse after copying to string and closing file.

It really depends on the size of the file.
 
If I am reading a 1GB+ file, I read it very differently than I would a 100kb file.


Tristam MacDonald - Software Engineer @Amazon - [swiftcoding]


#10 Bubsy   Members   -  Reputation: 407

Like
1Likes
Like

Posted 22 February 2013 - 07:15 PM

Besides the size issue, it depends on how you can process the file's data. If you can process the data in an event fashion (much like a SAX parser parsing XML data), I'd tell you to go with a stream based approach instead of loading all data into memory.

 

I think this approach will lead to less memory usage and less heap fragmentation (or lesser stack overflows occurrences, if you're allocating your buffers on the stack)



#11 wintertime   Members   -  Reputation: 1867

Like
1Likes
Like

Posted 23 February 2013 - 04:50 PM

If its not a huge file I tend more and more to open, get size, alloc exactly enough memory, read whole thing at once, close file, do parsing. The reason is the single responsibility principle, its just so much easier to not have some intermingled read/parse/askformore/read a bit more/manually buffer cause not enough data/talk back/read more stuff/deal with possible read error in the middle/whatever going on.

A library also gets more flexible if it not insists on wanting a filename, but accepts in memory buffers, which you can get from anywhere you like.



#12 Flimflam   Members   -  Reputation: 657

Like
1Likes
Like

Posted 23 February 2013 - 05:57 PM

... suggesting that your modify your...

 

Oh no, I didn't mean to imply that. I am honestly just wondering how others parse their files, either parse while reading or parse after copying to string and closing file.

 

If you're working with large files then it would make sense to parse the data as you read it, otherwise you could run into memory problems and the like. For small files, this isn't really something you would generally need to concern yourself with, so the standard open -> read -> close -> parse would make perfect sense. 



#13 Ectara   Crossbones+   -  Reputation: 3057

Like
0Likes
Like

Posted 23 February 2013 - 06:14 PM

Following your example on opening it in write mode (I'm not sure if all you're doing is reading; if so, this won't be too relevant), close the file as soon as possible after writing, if there is no more in the immediate future. The last thing you need is to write an important file, then the program terminates unexpectedly without flushing the buffer to disk, leaving the file in an invalid state.

Sometimes, the OS won't flush as part of it's cleanup if the application terminates before the file closes. I spent several minutes trying to figure out why, in a program I wrote to create plaintext worksheets for practicing writing Japanese, the file was 0 bytes in length after running the application. The thing was, the file was closed at the end after you confirm that there are no more sections to be added to the sheet, and when it asked for confirmation while I was rapidly testing small changes, I sent a signal to the process in the terminal to exit the program in fewer keystrokes. This resulted in it not flushing the file buffer, making zero changes.

My point is, you could write it so that it does everything right, and something outside of your control could cause the program to ungracefully terminate, leaving the data your program thought was written still in memory, with no obligation to write it to disk.


Edited by Ectara, 23 February 2013 - 09:38 PM.


#14 wintertime   Members   -  Reputation: 1867

Like
0Likes
Like

Posted 23 February 2013 - 06:34 PM

Yeah even if you want to keep a file open you should explicitly flush when you stop writing for the foreseeable future as to not loose data.






Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS