Sign in to follow this  
cozzie

Passing unknown block of data with it's size?

Recommended Posts

Hi,
I was wondering if it's possible to be able to pass different sized "blocks" of data to 1 function.
An example:

struct objOne
{
float x, y, z;
};

struct objTwo
{
float a, b, c;
std::string someString;
};

The function:
void ProcessData(sometype pData, const uint pSizeOfObjectInMemory)

And within this function it would like to handle the data (for creating or updating a d3d11 buffer), using the memory size of the object (basically sizeof(...). I want to be able to pass either an object of objOne to the function, or objTwo. Would this be possible by somehow passing a void* to the object's memory or something like that?

Any input is appreciated.

Share this post


Link to post
Share on other sites
Yes, it's possible, exactly as you describe. In fact, a lot of standard functions in OS APIs like Windows' or POSIX' do exactly that: they take a pointer to a struct and a size field, using the latter to determine the version of the struct passed in (e.g., an older app might have been compiled when the struct was 8 bytes, but then the struct is extended and apps compiled against newer headers would have a 12 byte size instead).

That said, don't. Use a union if you must, but you don't want that other. Use a std::variant if your compiler is (very) new if you still need a single function. Or use overloading like Apoch recommended, or some polymorphic behavior, etc.

You might be better off asking us about your real problem. This somewhat smells like an XY Problem to me.

Share this post


Link to post
Share on other sites

Or a template, maybe?

template<typename Type>
void SetConstants(const Type&& data)
{
     static_assert(std::is_trivially_copyable<Type>::value, "Type must be a trivially copyable value."); // we don't want to copy vtables, etc...

     SetConstants((const float*)&data, sizeof(Type));
}

Shouldn't be too bad even if you try to avoid templates for compile time, if you can pack the functions implementation into a generic version.

Edited by Juliean

Share this post


Link to post
Share on other sites

Thanks guys.

- In my initial attempt I did have overloaded function depending on the 'struct' I'm passing

- the reason why I'm looking for a more flexible way, has to do with the context:

-- it's a ConstantBuffer manager

-- currently I just have 2 possible structs (CB per frame, CB per object), but in the future I expect to have more different struct that need to be passed (updated in the CB)

- I'm looking for a way to pass the data independent of the used struct, to prevent having to add overloaded functions for each new struct I want to use later on

(because the map/unmap etc. functions just needs a block of data, there's no other reason to 'know' which struct it is, other then it's size, for the memory width of the ID3D11Buffer)

 

@SeanMiddleTech: do you have an example how this would work?

- how do I convert my struct to a void * (in the called function), or do I need another type?)

 

@Juliean: I've never used templates so far (not avoided them perse, but no need yet to use them). Maybe this is a use-case where they have to come in. But for now I'm leaning towards the "block of data through a void *" or using function overloads (using the same function names, but with different parameter for the to be passed struct).

Edited by cozzie

Share this post


Link to post
Share on other sites

But for now I'm leaning towards the "block of data through a void *" or using operator overloads.

 

If you go for overloading (doesn't have to do with operators though), than thats fine.

If you go for void*, you really should be using a template, if you can. Templates are a much type-safer and hard-to-screw up replacement for pretty much anything you would use void* in C-code (or old-style C++).

Share this post


Link to post
Share on other sites
Are you actually trying to bind data to callback functions? There are all kinds of idiomatic ways to implement that in C++, ranging from using lambdas in current versions of the language to using std::bind in your grandma's compiler. Consider these in addition to falling back to 1968-era C skills.

Share this post


Link to post
Share on other sites

Something like this should be a starter:

 

https://www.justsoftwaresolutions.co.uk/articles/intrototemplates.pdf

 

With that in mind, look at the example I gave you. Template boils down to having a function with different possible input-vales, so instead of void* you have T*, where T resolves to what you actually put into the function at this point. You should've already worked with classes that use this concept, like vector<> or similiar, right? So this should be a good starting point for writing template code yourself, which is something that really comes in handy at some point or another anyways :)

Share this post


Link to post
Share on other sites

NSFW:

 

struct s1

{

// members

};

 

struct s2

{

// members

};

 

 

void do_something(void *p,unsigned size)

{

// do something with the data pointed to by p

}

 

void main

{

st1  joe;

st2 fred;

do_something(  (void*)(&joe), sizeof(st1) );

do_something(  (void*)(&fred), sizeof(st2) );

}
 
this code is not "stupid programmer proof"  but is all that's technically required to do the job.
 
be sure to check the return type of sizeof() i'm guessing its unsigned.
 
note that a template is the preferred method these days.
Edited by Norman Barrows

Share this post


Link to post
Share on other sites

be sure to check the return type of sizeof() i'm guessing its unsigned.

 

Its size_t to be precise.

 

this code is not "stupid programmer proof" but is all that's technically required to do the job.

 

I'm going to shit bricks if even the best programmer won't at least once in a while get that code wrong, I can just see the copy-paste fuckups resulting in black-screens before my eyes :D

 

Srysl, just use templates, if not for the safety factor, then to save the extra typing (putting (void*)& and sizeof(Struct) would totally annoy me in the long run) :)

Share this post


Link to post
Share on other sites
I still think I have to pass the sizeof struct to know what struct I'm passing to the template function (I think)

 

You shouldn't have to. The template resolves to the type of the struct you passed, you when you have template<typename T>, you can then just use sizeof(T). If you go over the article I linked, you should get a good feel for it, its really easy once you get a hang of it.

EDIT: I pretty much posted the correct implementation of the function in my first post already, you can take that as a reference :)

Edited by Juliean

Share this post


Link to post
Share on other sites

NSFW:
 
struct s1
{
// members
};
 
struct s2
{
// members
};
 
 
void do_something(void *p,unsigned size)
{
// do something with the data pointed to by p
}
 
void main
{
st1  joe;
st2 fred;
do_something(  (void*)(&joe), sizeof(st1) );
do_something(  (void*)(&fred), sizeof(st2) );
}
 
this code is not "stupid programmer proof"  but is all that's technically required to do the job.
 
be sure to check the return type of sizeof() i'm guessing its unsigned.
 
note that a template is the preferred method these days.

EWWW my eyes, my eyes! The code, it burns! :(

Share this post


Link to post
Share on other sites

@SeanMiddleTech: do you have an example how this would work? - how do I convert my struct to a void * (in the called function), or do I need another type?)


It's Middle*ditch*. Yes, my name is just as stupid-sounding as it looks. Thanks, parents! :P

In any case, I'm not sure what you're asking about specifically. You can convert any struct into a void*.

Your use case of constant buffers still isn't quite at the root. If you just want to memcpy a struct into a mapped buffer, you don't need to know which struct is passed in. Just take the void* address of the struct and its size, since that's all memcpy itself requires. The template approach others advocated helps to remove programmer error, e.g.:

template <typename T>
auto simple_update_buffer(ConstantBuffer* buffer, T const& value) -> enable_if_t<is_object_v<T>>>
{
  void* address = buffer->map();
  memcpy(address, &value, sizeof(value));
  buffer->unmap();
}
The advantage being that you can't accidentally pass in the wrong size. The enable_if bit is an extra simple-ish check to make sure you don't accidentally pass in a pointer (because otherwise the function would bind to a reference-to-pointer).

Though in this case, I might argue that if you have different kinds of constant buffers that you should use the type system and make actually different constant buffer types, to ensure that you create your buffers with the right size and only update them with the proper values, e.g.

// a constant buffer that holds a copy of struct T
template <typename T>
class ConstantBuffer
{
  Buffer* _buffer = nullptr;

public:
  void allocate()
  {
    // guarantee that the buffer is allocated for the size of T
    _buffer = device.CreateConstantBuffer(sizeof(T));
  }

  void update(T const& value)
  {
    // guarantee that the buffer can only be updated with a copy of T
    void* mapped = _buffer->map();
    memcpy(mapper, &value, sizeof(value));
    _buffer->unmap();
  }
};

...

ConstantBuffer<PerFrameData> perFrameCB;
ConstantBuffer<PerModelData> modelCB;

PerFrameData data;

perFrameCB.update(data); // OK
modelCB.update(data); // COMPILE ERROR - saved you from a stupid mistake

The above of course is a simplistic example. You can make the wrapper support arrays of the given type with the proper size calculations, partial updates, etc. if required (e.g. for updating individual instances of a streaming per-instance buffer), to more intelligently use mapped addresses, etc.

The point isn't to use templates to do fancy meta-programming tricks (which are, 90% of the time, a stupid idea - they just make your code harder to read, harder to debug, and slower to compile) but to augment the types in your code so that the compiler enforces rules like "only the per-frame structure can be copied into the per-frame constant buffer."

Use the type system to make your life easier and less bug-prone.

(And as an aside, I'm not saying that this is the best way to do a constant buffer. It's just one of many ways to do things.)

Share this post


Link to post
Share on other sites
Thanks to all the input, I've managed to get it up and running using templates (yiha, my first template :cool:).
There's one thing I'd like to improve:

- some way of making sure that no 'rubbish' is being passed to the update function
-- for example like SeanMiddleDitch mentioned above
-- I also thought of storing the original sizeof(T)'s when creating the buffers, and then compare them when Update is called. That wouldn't assure the correct/ same struct/ set of data, but it will be a preventing of the user passing a wrong CBuffer struct to update one.
 
What would be my (other) options do add a check like that with the existing approach/code?
 
Note: I know the way of returning errors (true/false) and using the handle/ID's can be arguable/ done otherwise, but let's park that outside the primary goal (learning and applying templates :))
// DEFINITION

#ifndef CD3DCBUFFER_MGR_H
#define CD3DCBUFFER_MGR_H

#include <d3d11.h>
#include <vector>
#include <atlbase.h>


/**************************************************************************************/
/***							CD3D CBUFFER MGR CLASS								***/
/*** Manages creation and usage (lifetime) of Constant Buffers, D3D11				***/
/**************************************************************************************/

class CD3dCBufferMgr 
{
public:
	CD3dCBufferMgr(ID3D11Device *pDevice);
	~CD3dCBufferMgr();

	CD3dCBufferMgr(const CD3dCBufferMgr& other) = delete;				// copy constructor: not allowed
	CD3dCBufferMgr(CD3dCBufferMgr&& other) = delete;					// move constructor: not allowed
	CD3dCBufferMgr& operator=(const CD3dCBufferMgr& other) = delete;	// copy assignment: not allowed
	CD3dCBufferMgr& operator=(CD3dCBufferMgr&& other) = delete;			// move assignment: not allowed

	ID3D11Buffer* GetCBufferPtr(const unsigned int pHandle)		const;
	
	template<typename T>bool Add(const T *pBufferData, unsigned int *pIdCreated)
	{
		unsigned int newId = 0;
	
		// find empty spot
		for(size_t i=0;newId == 0 && i<mCBuffers.size();++i)
		{
			if(mCBuffers[i] == nullptr)	newId = i;
		}

		if(newId == 0)
		{
			newId = mCBuffers.size();
			mCBuffers.emplace_back();
		}

		// Create the constant buffer
		D3D11_BUFFER_DESC cBufferDesc;
		cBufferDesc.Usage = D3D11_USAGE_DYNAMIC;
		cBufferDesc.BindFlags = D3D11_BIND_CONSTANT_BUFFER;
		cBufferDesc.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE;
		cBufferDesc.MiscFlags = 0;
		cBufferDesc.StructureByteStride = 0;
		cBufferDesc.ByteWidth = sizeof(T);

		// pointer to initial data
		D3D11_SUBRESOURCE_DATA initData;
		initData.SysMemPitch = 0;
		initData.SysMemSlicePitch = 0;
		initData.pSysMem = pBufferData;

		if(FAILED(mDevice->CreateBuffer(&cBufferDesc, &initData, &mCBuffers[newId]))) 
		{
			// writte debug logging here, not in this small test appl
			return false;
		}

		*pIdCreated = newId;
		return true;
	}

	template<typename T>bool Update(ID3D11DeviceContext *pDeviceContext, unsigned int pHandle, const T *pBufferData)
	{
		if(pHandle >= mCBuffers.size()) return false;

		D3D11_MAPPED_SUBRESOURCE mappedRes;
		if(FAILED(pDeviceContext->Map(mCBuffers[pHandle], 0, D3D11_MAP_WRITE_DISCARD, 0, &mappedRes)))
		{
			// writte debug logging here, not in this small test appl
			return false;
		}

		T *dataPtr = (T*)mappedRes.pData;
		*dataPtr = *pBufferData;
		pDeviceContext->Unmap(mCBuffers[pHandle], 0);

		return true;
	}

	bool Release(const unsigned int pHandle);
		
private:
	std::vector<CComPtr<ID3D11Buffer>>	mCBuffers;
	
	CComPtr<ID3D11Device>			mDevice;
};

// DECLARATION

#include "cd3dcbuffermgr.h"


/**************************************************************************************/
/***								CONSTRUCTOR										***/
/*** ==> usage: when creating a CD3dCBufferMgr object 								***/
/*** ==> sets all variables in the class object to initial/ passed values			***/
/**************************************************************************************/

CD3dCBufferMgr::CD3dCBufferMgr(ID3D11Device *pDevice) : mDevice(pDevice)
{	
	// nothing yet
}

/**************************************************************************************/
/***								DESTRUCTOR										***/
/*** ==> usage: when CD3dCBufferMgr object is not needed anymore					***/
/*** ==> releases stuff, COM objects, memory etc. 									***/
/**************************************************************************************/

CD3dCBufferMgr::~CD3dCBufferMgr()
{
	// nothing yet
}

/**************************************************************************************/
/***									RELEASE										***/
/*** ==> usage: to free up a CB and make room for another one						***/
/*** ==> sets the requested CB to nullptr for reuse									***/
/**************************************************************************************/

bool CD3dCBufferMgr::Release(const unsigned int pHandle)
{
	if(pHandle >= mCBuffers.size()) return false;

	mCBuffers[pHandle] = nullptr;

	return true;
}




/**************************************************************************************/
/***								GET CBUFFER PTR							  CONST	***/
/*** ==> usage: to retrieve a pointer to a specific Constant Buffer					***/
/*** ==> returns the (const) pointer									 			***/
/**************************************************************************************/

ID3D11Buffer* CD3dCBufferMgr::GetCBufferPtr(const unsigned int pHandle) const
{
	if(pHandle >= mCBuffers.size()) return nullptr;
	return mCBuffers[pHandle];
}

Share this post


Link to post
Share on other sites
[source]
typedef unsigned int BufferHandle;

class D3dCBufferMgr
{
public:
D3dCBufferMgr(ID3D11Device *pDevice);
~D3dCBufferMgr();

D3dCBufferMgr(D3dCBufferMgr&& other) = delete; // move constructor: not allowed
D3dCBufferMgr(const D3dCBufferMgr& other) = delete; // copy constructor: not allowed
D3dCBufferMgr& operator=(const D3dCBufferMgr& other) = delete; // copy assignment: not allowed
D3dCBufferMgr& operator=(D3dCBufferMgr&& other) = delete; // move assignment: not allowed

ID3D11Buffer* GetConstantBufferPtr(BufferHandle pHandle) const;

template<typename T>bool TryRegisterBuffer(BufferHandle &createdID)
{
return Add<T>(nullptr, createdID);
}

BufferHandle GetNewHandle()
{
if(mFreeBuffers.size() > 0 )
{
BufferHandle id = *mFreeBuffers.back();
mFreeBuffers.pop_back();
return id;
}

BufferHandle id = mCBuffers.size();
mCBuffers.emplace_back();
return id;
}

template<typename T>bool TryRegisterBuffer(const T *pBufferData, BufferHandle &IdCreated)
{
BufferHandle IdCreated = GetNewHandle();

// Create the constant buffer
D3D11_BUFFER_DESC cBufferDesc;
cBufferDesc.Usage = D3D11_USAGE_DYNAMIC;
cBufferDesc.BindFlags = D3D11_BIND_CONSTANT_BUFFER;
cBufferDesc.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE;
cBufferDesc.MiscFlags = 0;
cBufferDesc.StructureByteStride = 0;
cBufferDesc.ByteWidth = sizeof(T);

// pointer to initial data
D3D11_SUBRESOURCE_DATA initData;
initData.SysMemPitch = 0;
initData.SysMemSlicePitch = 0;
initData.pSysMem = pBufferData;

if(FAILED(mDevice->CreateBuffer(&cBufferDesc, &initData, &mCBuffers[IdCreated])))
{
// write debug logging here, not in this small test appl
Release(IdCreated);
return false;
}

return true;
}

template<typename T>bool TryUpdate(ID3D11DeviceContext *pDeviceContext, BufferHandle handle, const T *pBufferData)
{
if(handle >= mCBuffers.size()) return false;

D3D11_MAPPED_SUBRESOURCE mappedRes;
if(FAILED(pDeviceContext->Map(mCBuffers[pHandle], 0, D3D11_MAP_WRITE_DISCARD, 0, &mappedRes)))
{
// write debug logging here, not in this small test appl
return false;
}

memcpy(mappedRes.pData, pBufferData, sizeof(T)); // you should only have POD so this will be fine
pDeviceContext->Unmap(mCBuffers[pHandle], 0);

return true;
}

bool Release(BufferHandle handle)
{
if(handle >= mCBuffers.size()) return false;

mFreeBuffers.emplace_back(handle);
mCBuffers[handle] = nullptr;

return true;
}

private:
std::vector<CComPtr<ID3D11Buffer>> mCBuffers;
std::vector<int> mFreeBuffers;

CComPtr<ID3D11Device> mDevice;
};
[/source]

Some notes;

- C-prefix on classes remains bullshit
- No need to linear scan when allocating buffers; maintain a free list
- Added a null-data buffer to create without initial data
- You missed cleaning up when failing to create buffer so added that
- 'const' on pass by value types is just redundant
- memcpy is a better copy solution - buffers should only contain POD so a copy operation would be insane
- just use a reference when you need to return a second value rather than thinking this is still C and passing a pointer to a thing - that only makes sense if the thing is optional, which is clearly isn't here.
-- a 'pair' return might be saner anyway... more so with C++17's structured bindings coming down the line

Also, much like Sean proposes I wouldn't do it like this anyway - looking in to a central manager when updating things feels wrong; let the owning objects deal with it directly that way you can't get the wrong data being copied in to the wrong buffer plus you can save dicking off to find things in a vector to update.
(You can still have a central common manager if you want to share named global bufer

Right now when you are trying to update your code has;
- do a function call
- check if a value is in range (99.9999% of time it will be)
- look up that buffer in the array
- map
- copy the thing
- unmap it
- return

Maintaining locally means;
- do a function call
- map
- copy
- unmap
- return

5 operations vs 7 with redundant 'if's and the like in there with the added fun of having to go looking far away for the data too.

Edit:
Ugh... 'source' tag destroy formatting... well.. that's useless... *slowclap* Edited by phantom

Share this post


Link to post
Share on other sites

I don't know why, but up until now I've always read it as SeanMiddleTech as well... brain farts.

 
i did at first too.


You'd both be very surprised how common this exact mistake is. I still get called Middletech by a few friends who legitimately thought that was my name for years before I bothered to correct them. :)
 

BTW, Sean, do you work for a middleware / engine company?


I work at Wargaming's Seattle studio and have since before Wargaming bought us (previously Gas Powered Games).

Share this post


Link to post
Share on other sites

@phantom: wow, thanks for taking the time to put through the improvements.

I'm gonna go through them one by one and adopt them (for sure :)).

 

Regarding your remarks, partially I'll probably use them, some points are basically done this way through the whole engine/ codebase. Not that this means I should improve on these points, but the impact's bigger.

Share this post


Link to post
Share on other sites
No problem - I've been boning up on my C++ again of late (for work reasons I kinda stopped paying attention, with C++17 around the corner I figure now is a good time to pick up the stuff I missed/haven't used in anger yet, if only for my home projects) so it was a nice exercise :)

I should probably note, I didn't run it by a compiler, so it _might_ have some issues, but everything passes my internal test ;)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this