Hello,

I am parsing TGA files for my textures and everything works fine.

The issue is when I get to 4096*4096 or 2048*2048 resolution on those images it is very very very slow !

I went into the code and optimized the memory allocation (now there is no waste) but the loading as fast at it can be.

This is my question : Should I read all the file at once in a char array and then parse the array (how can I do it when the TGA is compressed ?) ? Because as of yet I read the header at once and then when it comes to the image's data I read pixel by pixel, I guess this is what makes it slow...

I don't read all the file at once because if there is a lot (lot) of elements to load it would take a lot of memory (at worst twice).

Just a simple question

rXp

Sorry about the "parsing", I am reading a TGA file (exported from gimp or photoshop).

About the compression I know the header is never compressed but the header doesn't say how big is the "data image" part of the file.

What I do now : I get the header and prepare a unsigned char vector for X pixels. I load the data image into a temporary buffer and go trough that buffer to get the data. It is faster but on one PC takes 30sec to load a 4096*4096 file and on the other is takes 1 sec. The difference is that on the second PC there is 2 HDD and the files are on the second unused HDD.

Could it slow the process that much ?

About the compression I know the header is never compressed but the header doesn't say how big is the "data image" part of the file.

The size of the image data is the size of the file minus the size of the header.

Even if it were an uncompressed 24-bit bitmap image, it shouldn't take anywhere near 30 seconds to load a 4096x4096 image on any remotely modern computer. Can you post the code that loads the image data from file?

About the compression I know the header is never compressed but the header doesn't say how big is the "data image" part of the file.

The size of the image data is the size of the file minus the size of the header.

Really ? Because I based myself on the complete specifications and there is the color map, the ID, the extra information that could be added by the person making the image.

And yes this is the code, since I come from java I am still learning the good practice of C++. DO not hesitate to correct me on every little details. Always happy to learn

void OBJParser::parseTGA(const char* file, TGAData* const tga)
{
	boost::filesystem::path path(file);
	if (boost::filesystem::exists(path))
	{
		char header[18];
		unsigned short colorMapLength = 0;
		std::ifstream infile;
		infile.open(file, std::ifstream::binary | std::ifstream::in);
		//read header
		infile.read(header, 18);

		//read id length
		if (header[0] > 0)tga->data.imageID = (char*)malloc(header[0]);

		//read colormap type
		tga->header.isColorMap = (header[1] == 1);

		//read image type
		if (header[2] == (char)0)tga->header.imageType = TGAData::NONE;
		else if (header[2] == (char)1)tga->header.imageType = TGAData::UNCOMPRESSED_COLOR_MAP;
		else if (header[2] == (char)2)tga->header.imageType = TGAData::UNCOMPRESSED_TRUE_COLOR;
		else if (header[2] == (char)3)tga->header.imageType = TGAData::UCOMPRESSED_BLACK_AND_WHITE;
		else if (header[2] == (char)9)tga->header.imageType = TGAData::RUN_LENGTH_COLOR_MAP;
		else if (header[2] == (char)10)tga->header.imageType = TGAData::RUN_LENGTH_TRUE_COLOR;
		else tga->header.imageType = TGAData::NONE;

		//if there is a color map
		if (tga->header.isColorMap)
		{
			//colormap length to know how to skip it
		}

		//image specification
		char size[2];
		tga->header.xOrigin = *(static_cast<unsigned short*>(static_cast<void*>(&header[8]))); //read x origin
		tga->header.yOrigin = *(static_cast<unsigned short*>(static_cast<void*>(&header[10])));//read y origin
		size[0] = header[12];
		size[1] = header[13];
		tga->header.width = *(static_cast<unsigned short*>(static_cast<void*>(&header[12])));//read width
		tga->header.height = *(static_cast<unsigned short*>(static_cast<void*>(&header[14])));//read height
		tga->header.pixelDepth = header[16];
		tga->header.imageDescriptor = header[17];


		//read image
		//format : NUMBER[R G B  & A ssi pixeldepth > 0]
		const size_t pixelDepth = tga->header.pixelDepth / 8;
		const size_t totalPixel = tga->header.height*tga->header.width;
		tga->data.parsedImageData.reserve(totalPixel * 4);
		unsigned int counter = 0;
		char* chunkSize = (char*)malloc(1);
		char* rgba = (char*)malloc(pixelDepth);
		unsigned int bufferSize = boost::filesystem::file_size(file) - 18;
		char* bufferRestOfFile = (char*)malloc(bufferSize);//minus header
		infile.read(bufferRestOfFile, bufferSize);
		unsigned int index = 0;
		switch (tga->header.imageType)
		{
		case TGAData::RUN_LENGTH_TRUE_COLOR:
		{
			const char maskMSBToZero = 0x7F; //01111111
			while (counter < totalPixel)
			{
				unsigned char unsignedChunkSize = *static_cast<unsigned char*>(static_cast<void *>(&bufferRestOfFile[index]));
				index++;
				unsigned char nb = 0;

				if (unsignedChunkSize < 128) //raw chunk -> X next pixel
				{
					nb = unsignedChunkSize + 1;//number of same pixels in a row
					for (short i = 0; i<nb; i++)
					{
						tga->data.parsedImageData.push_back((unsigned char)bufferRestOfFile[index + 2]);
						tga->data.parsedImageData.push_back((unsigned char)bufferRestOfFile[index + 1]);
						tga->data.parsedImageData.push_back((unsigned char)bufferRestOfFile[index]);
						if (pixelDepth > 3)
						{
							tga->data.parsedImageData.push_back((unsigned char)bufferRestOfFile[index + 3]);
							index += 4;
						}
						else
						{
							tga->data.parsedImageData.push_back(255);
							index += 3;
						}

						counter++;
					}
				}
				else //X * COLOR
				{
					nb = (unsignedChunkSize & maskMSBToZero) + 1;
					for (short i = 0; i<nb; i++)
					{
						tga->data.parsedImageData.push_back((unsigned char)bufferRestOfFile[index + 2]);
						tga->data.parsedImageData.push_back((unsigned char)bufferRestOfFile[index + 1]);
						tga->data.parsedImageData.push_back((unsigned char)bufferRestOfFile[index]);
						if (pixelDepth > 3)
							tga->data.parsedImageData.push_back((unsigned char)bufferRestOfFile[index + 3]);
						else
							tga->data.parsedImageData.push_back(255);

						counter++;
					}
					if (pixelDepth > 3)	index += 4;
					else index += 3;
				}
			}

		}
			break;
		case TGAData::UNCOMPRESSED_TRUE_COLOR:
		{
			while (counter<totalPixel)
			{
				tga->data.parsedImageData.push_back((unsigned char)bufferRestOfFile[index + 2]);
				tga->data.parsedImageData.push_back((unsigned char)bufferRestOfFile[index + 1]);
				tga->data.parsedImageData.push_back((unsigned char)bufferRestOfFile[index]);
				if (pixelDepth > 3)
				{
					tga->data.parsedImageData.push_back((unsigned char)bufferRestOfFile[index + 3]);
					index += 4;
				}
				else
				{
					tga->data.parsedImageData.push_back(255);
					index += 3;
				}

				counter++;
			}
		}
			break;
		case TGAData::UCOMPRESSED_BLACK_AND_WHITE: //not tested at all
		{
			while (counter < totalPixel)
			{
				tga->data.parsedImageData.push_back((unsigned char)bufferRestOfFile[index]);
				tga->data.parsedImageData.push_back((unsigned char)bufferRestOfFile[index]);
				tga->data.parsedImageData.push_back((unsigned char)bufferRestOfFile[index]);

				tga->data.parsedImageData.push_back(255);
				counter++;
				index++;
			}
		}
			break;
		}
		free(rgba);
		free(chunkSize);
		infile.close();
	}
}

Maybe you wonder why I use a vector and not an array, well it is because a lot of people told me to so I wouldn't need to worry too much about the memory allocation.

I don't think free is a big deal but at first I wanted something that worked, not I want something that work well. (do then optimize)

About the compression I know the header is never compressed but the header doesn't say how big is the "data image" part of the file.

The size of the image data is the size of the file minus the size of the header.

Really ? Because I based myself on the complete specifications and there is the color map, the ID, the extra information that could be added by the person making the image.

If you don't consider those as part of the header, then just parse their size and subtract from the total. It doesn't change the idea that you derive the size of the remaining image data from the file size.

infile.read(bufferRestOfFile, sizeof(bufferRestOfFile));

infile.read(bufferRestOfFile, sizeof(bufferRestOfFile));

I don't think this line does what you think it does. sizeof gives the static size of the pointer, not the dynamic size of what it points to.

Sorry that was an old version. I actually use the size of the fil minus the 18byte of header now. It works fine. (by that I mean that my texture works)

The post has been updated with the correct code.