Hey guys, gals, etc. My Problem: - I am using the C Low Level IO routines . These functions invoke the operating system directly for lower-level operation than that provided by stream I/O. Low-level input and output calls do not buffer or format data. - I have to parse text files of undetermined size. - I have both singleton struct and virtual class implementations (both are equally excruciatingly slow ). The Fact: - It more than likely that the speed is because of poor search routines on my part, but.... My Question: - Can anyone recommend some fast text searching algorithms or web sites devoted to such as I''ve found none. - Also, are the cpp iostream classes noticably faster? What about FILE *p''s? -Any help would be appreciated!

Hello,

My suggestion would be to load the whole file into memory then parse it from there. This is what I have done. You might want to use linked lists for this. I heard std::list is good although I have never made a jump to it since I wrote my own linked list implementation a couple of years before STL was the standard.

Hope that helps

The fastest way is indeed to first fully load the file into memory, no matter how large it is, and then use that memory. Iostream etc will be slower, it introduces overhead, so if you really need speed, use a FILE and read it in fully in one swift fread

quote:Original post by m_e_my_self
The fastest way is indeed to first fully load the file into memory, no matter how large it is, and then use that memory. Iostream etc will be slower, it introduces overhead, so if you really need speed, use a FILE and read it in fully in one swift fread

iostream use at least the same low-level routines I presume...probably you forget read() method in ifstream
std and stl are better !!!

If you want to process BIG text file I suggest you to use ''ropes'' or list of string.

list<string> text_file_lines;

Then you can read lines and put them in your list.

If you want to load at max speed you can use a strstream and load it with your ifstream or create your own buffer.
But you need probably to split your file in lines to process them efficiently with std::string methods.
But I dont know what you want to do...

PS: I wrote tons of template list,array,string...when I learned STL I simply forget about them..and FILE* too !!!

" My suggestion would be to load the whole
file into memory then parse it from there"

Well, not quite but close. Use memory mapping instead of loading the file into virtual space (otherwise you run the risk of having it getting swapped out to disk again if it is too big. DOH!).