🔒 Locked

load jpg from memory

Started by Filami Jul 14, 2005 at 2:58 PM 9 replies 3.4k views

Original Post

Jul 14, 2005 at 2:58 PM

Hello ppl. I have a simple question to make. I know how to load jpg images from files, but, I need to load them from memory instead. Any help?

SiCrane

Jul 14, 2005 at 3:00 PM

How are you loading them from file? Most image loaders have support for loading from memory.

Filami

Jul 14, 2005 at 6:16 PM

I am using the jpeg sdk and I'm not using any image loader like devIL or direct3DX. In fact, I'm developing my own image loader for my engine.

template<typename T>inline void SWAP(T& a, T& b){	T t = a;	a = b;	b = t;}struct tImage{	int channels;	int sizeX;	int sizeY;	unsigned char *data;};GLuint LoadTextureJPG(const char* path){	struct jpeg_decompress_struct cinfo;	tImage pImageData;	FILE *pFile;			//carregar imagem	pFile = fopen(path, "rb");	if(!pFile)return 0;		jpeg_error_mgr jerr;	cinfo.err = jpeg_std_error(&jerr);	jpeg_create_decompress(&cinfo);	jpeg_stdio_src(&cinfo, pFile);		jpeg_read_header(&cinfo, TRUE);	jpeg_start_decompress(&cinfo);	pImageData.channels = cinfo.num_components;	pImageData.sizeX    = cinfo.image_width;	pImageData.sizeY    = cinfo.image_height;	int rowSpan = cinfo.image_width * cinfo.num_components;	pImageData.data = ((unsigned char*)malloc(sizeof(unsigned char)*rowSpan*pImageData.sizeY));	unsigned char** rowPtr = new unsigned char*[pImageData.sizeY];	for (int i = 0; i < pImageData.sizeY; i++)		rowPtr = &(pImageData.data[i * rowSpan]);	int rowsRead = 0;	while (cinfo.output_scanline < cinfo.output_height)		rowsRead += jpeg_read_scanlines(&cinfo, &rowPtr[rowsRead], cinfo.output_height - rowsRead);	delete [] rowPtr;	jpeg_finish_decompress(&cinfo);	jpeg_destroy_decompress(&cinfo);	fclose(pFile);/*	for(int y=0; y < pImageData.sizeY/2; y++){		for(int x=0; x < pImageData.sizeX; x++){			int y1 = y;			int y2 = pImageData.sizeY - y - 1;			SWAP(pImageData.data[(x+y2*pImageData.sizeX)*3+0], pImageData.data[(x+y1*pImageData.sizeX)*3+0]);			SWAP(pImageData.data[(x+y2*pImageData.sizeX)*3+1], pImageData.data[(x+y1*pImageData.sizeX)*3+1]);			SWAP(pImageData.data[(x+y2*pImageData.sizeX)*3+2], pImageData.data[(x+y1*pImageData.sizeX)*3+2]);		}	}*/	//criar textura	GLuint tex;	glGenTextures(1, &tex);	glPixelStorei (GL_UNPACK_ALIGNMENT, 1);	glBindTexture(GL_TEXTURE_2D, tex);	int textureType = GL_RGB;	gluBuild2DMipmaps(GL_TEXTURE_2D, pImageData.channels, pImageData.sizeX, pImageData.sizeY, textureType, GL_UNSIGNED_BYTE, pImageData.data);	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_LINEAR);	return tex;}

This code was not taken in my engine since I did not implemented for it yet. It was taken from other application done by me that was using openGL. What I want to do is to copy this code to my engine to use in my own texture interface (thatis API independent). I need to load from memory and not from a file to be able to me more flexible. I'm thincking to integrate a file packer in my engine and I want to support loading from memory, form the package or from a file directly and the better way to do this is to load them all by memory. This way I only need to open the entire file to the memory and then load the texture.

But as you can see, the data source in this case is only a FILE* and I want it to read from a const char* ou from a void*.

Any help?

coelurus

Jul 15, 2005 at 11:45 AM

Got some full code for this, but it's infested with code for streaming and distributed work so I'll just give you the important bits:

/* Whatever floats your boat */#define JPEG_BUFFER_SIZE (8 << 10)static char jpegBuffer[JPEG_BUFFER_SIZE];typedef struct {    struct jpeg_source_mgr  pub;    /* I call this 'file' even though it's in memory */    objptr                  file;} JPEGSource;static void init_sourceFunc(j_decompress_ptr cinfo) {    ((JPEGSource*)cinfo->src)->pub.bytes_in_buffer = 0;}static boolean fill_input_bufferFunc(j_decompress_ptr cinfo) {    JPEGSource  *src = (JPEGSource*)cinfo->src;    /* "somehow" is obviously your way of reading data from some source (you could just point jpegBuffer to the proper place I guess) */    somehow_read(jpegBuffer, JPEG_BUFFER_SIZE, src->file);    src->pub.next_input_byte = jpegBuffer;    src->pub.bytes_in_buffer = JPEG_BUFFER_SIZE;    return TRUE;}void skip_input_dataFunc(j_decompress_ptr cinfo, long num_bytes) {    JPEGSource  *src = (JPEGSource*)cinfo->src;    if (num_bytes > 0) {        somehow_seek(src->file, FILE_SEEK_CUR, num_bytes);        if (num_bytes > src->pub.bytes_in_buffer) src->pub.bytes_in_buffer = 0;        else {            src->pub.next_input_byte += num_bytes;            src->pub.bytes_in_buffer -= num_bytes;        }    }}void term_sourceFunc(j_decompress_ptr cinfo) {}static JPEGSource   jpegSource = {                                    .pub.init_source = init_sourceFunc,                                    .pub.fill_input_buffer = fill_input_bufferFunc,                                    .pub.skip_input_data = skip_input_dataFunc,                                    .pub.resync_to_restart = jpeg_resync_to_restart,                                    .pub.term_source = term_sourceFunc                                };int jpeg_loader_thingy() {    ...    jpegInfo.src = (struct jpeg_source_mgr*)&jpegSource    jpegSource.file = file;    jpegSource.pub.next_input_byte = NULL;    jpegSource.pub.bytes_in_buffer = 0;    ... the usual stuff    return ret_code;}

There's info on how this works in the JPEG "SDK" as you called it, look for a txt-file called libjpeg.doc.gz or similar. I could've missed a few things so you better get hold of that doc and compare :)

Filami

Jul 15, 2005 at 1:36 PM

thank you

I have tried that code but I had some problems. I have adapted your code to mine and I have got this:

#include "jpeglib.h"#pragma comment (lib, "jpeg.lib")void somehow_read(char* buffer, int maxSize){	FILE* file = fopen("teste.jpg", "rb");	fseek(file, SEEK_END, 0);	long s = ftell(file);	fseek(file, SEEK_SET, 0);	fread(buffer, 1, s, file);	fclose(file);}struct tImage{	int channels;	int sizeX;	int sizeY;	unsigned char *data;};/* Whatever floats your boat */#define JPEG_BUFFER_SIZE (8 << 10)static char jpegBuffer[JPEG_BUFFER_SIZE];typedef struct {    struct jpeg_source_mgr  pub;    /* I call this 'file' even though it's in memory *///    objptr                  file;} JPEGSource;static void init_sourceFunc(j_decompress_ptr cinfo) {    ((JPEGSource*)cinfo->src)->pub.bytes_in_buffer = 0;}static boolean fill_input_bufferFunc(j_decompress_ptr cinfo) {    JPEGSource  *src = (JPEGSource*)cinfo->src;    /* "somehow" is obviously your way of reading data from some source (you could just point jpegBuffer to the proper place I guess) */    somehow_read(jpegBuffer, JPEG_BUFFER_SIZE /*, src->file*/);    src->pub.next_input_byte = (const unsigned char*)jpegBuffer;    src->pub.bytes_in_buffer = JPEG_BUFFER_SIZE;    return TRUE;}void skip_input_dataFunc(j_decompress_ptr cinfo, long num_bytes) {    JPEGSource  *src = (JPEGSource*)cinfo->src;    if (num_bytes > 0) {//        somehow_seek(src->file, FILE_SEEK_CUR, num_bytes);        if (num_bytes > src->pub.bytes_in_buffer) src->pub.bytes_in_buffer = 0;        else {            src->pub.next_input_byte += num_bytes;            src->pub.bytes_in_buffer -= num_bytes;        }    }}void term_sourceFunc(j_decompress_ptr cinfo) {}static JPEGSource   jpegSource;int jpeg_loader_thingy() {	struct jpeg_decompress_struct cinfo;	jpeg_error_mgr jerr;	cinfo.err = jpeg_std_error(&jerr);	jpeg_create_decompress(&cinfo);	jpegSource.pub.init_source = init_sourceFunc;	jpegSource.pub.fill_input_buffer = fill_input_bufferFunc;	jpegSource.pub.skip_input_data = skip_input_dataFunc;	jpegSource.pub.resync_to_restart = jpeg_resync_to_restart;	jpegSource.pub.term_source = term_sourceFunc;	jpegSource.pub.next_input_byte = NULL;    jpegSource.pub.bytes_in_buffer = 0;    cinfo.src = (struct jpeg_source_mgr*)&jpegSource		tImage pImageData;	jpeg_read_header(&cinfo, TRUE);	jpeg_start_decompress(&cinfo);	pImageData.channels = cinfo.num_components;	pImageData.sizeX    = cinfo.image_width;	pImageData.sizeY    = cinfo.image_height;	int rowSpan = cinfo.image_width * cinfo.num_components;	pImageData.data = ((unsigned char*)malloc(sizeof(unsigned char)*rowSpan*pImageData.sizeY));	unsigned char** rowPtr = new unsigned char*[pImageData.sizeY];	for (int i = 0; i < pImageData.sizeY; i++)		rowPtr = &(pImageData.data[i * rowSpan]);	int rowsRead = 0;	while (cinfo.output_scanline < cinfo.output_height)		rowsRead += jpeg_read_scanlines(&cinfo, &rowPtr[rowsRead], cinfo.output_height - rowsRead);	delete [] rowPtr;	jpeg_finish_decompress(&cinfo);	jpeg_destroy_decompress(&cinfo);	return true;}int main(){	jpeg_loader_thingy();	return 0;}

But this do not work preperly. The function 'fill_input_bufferFunc' is never called and when I pass the function 'jpeg_read_header(&cinfo, TRUE);' no data is filled in the image_width and height. The program then exists with code 1 in the 'jpeg_start_decompress(&cinfo);'. Any help?

coelurus

Jul 15, 2005 at 5:37 PM

Your code seemed to work for me (I ran it with no image and it crashed and gdb told me 'fill_input_bufferFunc' was called), but you had some rather risky things in there so here's my take on it:

#include <stdlib.h>#include <stdio.h>#include "jpeglib.h"#define JPEG_BUFFER_SIZE (8 << 10)static char jpegBuffer[JPEG_BUFFER_SIZE];static FILE *src;static int  srcLen;int source_init(const char *filename) {    src = fopen(filename, "rb");    if (src == NULL) return 0;    fseek(src, 0, SEEK_END);    srcLen = ftell(src);    fseek(src, 0, SEEK_SET);    return 1;}void source_close() {    fclose(src);}int source_read(char* buffer) {    int len = JPEG_BUFFER_SIZE;    if (len > srcLen) len = srcLen;    srcLen -= len;    fread(buffer, len, 1, src);    return len;}void source_seek(int num) {    fseek(src, num, SEEK_CUR);}struct tImage{    int channels;    int sizeX;    int sizeY;    unsigned char *data;};/* Slick... */typedef struct {    struct jpeg_source_mgr  pub;} JPEGSource;static void init_sourceFunc(j_decompress_ptr cinfo) {    ((JPEGSource*)cinfo->src)->pub.bytes_in_buffer = 0;}static boolean fill_input_bufferFunc(j_decompress_ptr cinfo) {    JPEGSource  *src = (JPEGSource*)cinfo->src;    src->pub.bytes_in_buffer = source_read(jpegBuffer);    src->pub.next_input_byte = (const unsigned char*)jpegBuffer;    return TRUE;}void skip_input_dataFunc(j_decompress_ptr cinfo, long num_bytes) {    JPEGSource  *src = (JPEGSource*)cinfo->src;        if (num_bytes > 0) {        source_seek(num_bytes);        if (num_bytes > src->pub.bytes_in_buffer) src->pub.bytes_in_buffer = 0;        else {            src->pub.next_input_byte += num_bytes;            src->pub.bytes_in_buffer -= num_bytes;        }    }}void term_sourceFunc(j_decompress_ptr cinfo) {}static JPEGSource   jpegSource;int jpeg_loader_thingy() {    struct jpeg_decompress_struct cinfo;    jpeg_error_mgr jerr;    if (!source_init("tjof.jpg")) return 0;    cinfo.err = jpeg_std_error(&jerr);    jpeg_create_decompress(&cinfo);    jpegSource.pub.init_source = init_sourceFunc;    jpegSource.pub.fill_input_buffer = fill_input_bufferFunc;    jpegSource.pub.skip_input_data = skip_input_dataFunc;    jpegSource.pub.resync_to_restart = jpeg_resync_to_restart;    jpegSource.pub.term_source = term_sourceFunc;    jpegSource.pub.next_input_byte = NULL;    jpegSource.pub.bytes_in_buffer = 0;    cinfo.src = (struct jpeg_source_mgr*)&jpegSource    tImage pImageData;    jpeg_read_header(&cinfo, TRUE);    jpeg_start_decompress(&cinfo);    pImageData.channels = cinfo.num_components;    pImageData.sizeX    = cinfo.image_width;    pImageData.sizeY    = cinfo.image_height;    printf("%d %d\n", pImageData.sizeX, pImageData.sizeY);    int rowSpan = cinfo.image_width * cinfo.num_components;    pImageData.data = ((unsigned char*)malloc(sizeof(unsigned char)*rowSpan*pImageData.sizeY));    unsigned char** rowPtr = new unsigned char*[pImageData.sizeY];    for (int i = 0; i < pImageData.sizeY; i++)        rowPtr = &(pImageData.data[i * rowSpan]);    int rowsRead = 0;    while (cinfo.output_scanline < cinfo.output_height)        rowsRead += jpeg_read_scanlines(&cinfo, &rowPtr[rowsRead], cinfo.output_height - rowsRead);    delete [] rowPtr;    free(pImageData.data);    jpeg_finish_decompress(&cinfo);    jpeg_destroy_decompress(&cinfo);    source_close();    return true;}int main(){    printf("%d\n", jpeg_loader_thingy());    return 0;}

Probably not fool-proof, just some minute-scale addendums. I threw in a "source-module" near the top which could act as any source for your data later on. I tried with one image and it reported the proper dimensions, didn't bother to see what was loaded though.

I would strongly advice you to read libjpeg.doc.gz about the JPEG buffer-thingy, it's simply a middle-hand between the source data and the JPEG library and it must be continuously filled until the file ends.

And I do hope you plan on replacing the error-handlers [smile]

Filami

Jul 15, 2005 at 9:10 PM

I have used the code you posted as is and it didn't worked :(
Its strange because I thinck it should work right now.

coelurus

Jul 16, 2005 at 6:01 AM

Have you read libjpeg.doc.gz yet?

A little more information from some debugging-efforts would be helpful too. Considering the pragma-lib thingies you had first, it looks as if you're using vc++ which I don't so I can't test the sample in your place.

mike74

Jul 16, 2005 at 7:45 AM

I would use boutell.com's gd library:

http://www.boutell.com/gd/manual2.0.33.html

I used it today, and it works great. I'm quite pleased.

By the way, you want this function:

gdImageCreateFromJpegPtr(int size, void *data) (FUNCTION)

Mike C.
http://www.coolgroups.com/zoomer

raydog

Jul 16, 2005 at 4:24 PM

As I remember it, the jpeg library from Independent JPEG Group has no functions
for loading jpegs from memory, just from file. You might have to hack the function
called fill_input_buffer() in jdatasrc.c to get it to work.

You might do better with the jpeg library from Intel if you want a straight-forward
approach.

coelurus

Jul 16, 2005 at 4:59 PM

Clarification: The libjpeg package includes official information on how to load images from memory and sample code demonstrating it has been posted.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.