#### Archived

This topic is now archived and is closed to further replies.

# file io functionality (...speed)

This topic is 5152 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Hey guys, gals, etc. My Problem: - I am using the C Low Level IO routines . These functions invoke the operating system directly for lower-level operation than that provided by stream I/O. Low-level input and output calls do not buffer or format data. - I have to parse text files of undetermined size. - I have both singleton struct and virtual class implementations (both are equally excruciatingly slow ). The Fact: - It more than likely that the speed is because of poor search routines on my part, but.... My Question: - Can anyone recommend some fast text searching algorithms or web sites devoted to such as I''ve found none. - Also, are the cpp iostream classes noticably faster? What about FILE *p''s? -Any help would be appreciated!

##### Share on other sites
Hello,

My suggestion would be to load the whole file into memory then parse it from there. This is what I have done. You might want to use linked lists for this. I heard std::list is good although I have never made a jump to it since I wrote my own linked list implementation a couple of years before STL was the standard.

Hope that helps

##### Share on other sites
The fastest way is indeed to first fully load the file into memory, no matter how large it is, and then use that memory. Iostream etc will be slower, it introduces overhead, so if you really need speed, use a FILE and read it in fully in one swift fread

##### Share on other sites
quote:
Original post by m_e_my_self
The fastest way is indeed to first fully load the file into memory, no matter how large it is, and then use that memory. Iostream etc will be slower, it introduces overhead, so if you really need speed, use a FILE and read it in fully in one swift fread

iostream use at least the same low-level routines I presume...probably you forget read() method in ifstream
std and stl are better !!!

If you want to process BIG text file I suggest you to use ''ropes'' or list of string.

list<string> text_file_lines;

If you want to load at max speed you can use a strstream and load it with your ifstream or create your own buffer.
But you need probably to split your file in lines to process them efficiently with std::string methods.
But I dont know what you want to do...

PS: I wrote tons of template list,array,string...when I learned STL I simply forget about them..and FILE* too !!!

##### Share on other sites
" My suggestion would be to load the whole
file into memory then parse it from there"

Well, not quite but close. Use memory mapping instead of loading the file into virtual space (otherwise you run the risk of having it getting swapped out to disk again if it is too big. DOH!).

• ### Forum Statistics

• Total Topics
628694
• Total Posts
2984253

• 16
• 13
• 13
• 10
• 10