Reusable function for adding values from a container

Started by
7 comments, last by l0calh05t 10 years, 2 months ago

Just wrote this. I wanted a way to add values from a member-variable of each element in a function.


struct MyStruct
{
	std::string otherData;
	int memberVariable;
};

std::vector<MyStruct> myContainer = {...};

//Sum the values of *a member variable* within each element of a container.
int total = SumValuesFrom(myContainer, memberVariable);

Note that 'memberVariable' isn't a variable. Or a variable type. Also note that it is 11:23 PM my time, and I got zero sleep last night.

So naturally, I'm at peak programming performance. tongue.png

Here's my functional, reusable, perfectly abusive of macros solution:


#define SumValuesFrom(container, member) priv_XValuesFrom_normal(container, member, [](const decltype(container.front().member) &valueA, const decltype(container.front().member) &valueB) { return (valueA + valueB); })

#define priv_XValuesFrom_normal(container, member, addingFunctor) priv_XValuesFrom<decltype(container.front().member)>(container, [](const decltype(container)::value_type &element) { return element.member; }, addingFunctor)

template<typename ValueType, typename ContainerType, typename ReadingFunctor, typename AddingFunctor>
ValueType priv_XValuesFrom(const ContainerType &container, ReadingFunctor getValue, AddingFunctor addValues, ValueType initialValue = ValueType())
{
	ValueType total = initialValue;

	for(const auto &element : container)
	{
		total = addValues(total, getValue(element));
	}

	return total;
}

There's even a version for if the elements are pointers (haven't tested that yet).

It compiles and runs: [Ideone.com]

The only thing my sleep-deprived mind is displeased with is the dependency on container.front().

Advertisement

This looks like a job for pointer to members:


template <typename T, typename U, typename Container>
T SumValuesFrom(Container c, T U::* member_ptr) {
  return std::accumulate(c.begin(), 
                         c.end(), 
                         T(), 
                         [=](T a, const U & b) {
                           return a + (b.*member_ptr);
                         });
}

int main(int, char **) {
  std::vector<MyStruct> myContainer = /* */;
  int total = SumValuesFrom(myContainer, &MyStruct::memberVariable);
}

Wow. Such C++. Many syntax. Very reuse.

Now profile both and see if pointer-to-member and macro magic compile to similar assembly laugh.png

This looks like a job for pointer to members:

blink.png

Well, that makes it a whole lot easier. Thank you! I've never used pointer-to-member before.

I couldn't think of any way to make it template only - I briefly considered using a byte 'stride'-type function, casting to char*, but other than that horrible idea I was drawing a blank.

if your already storing the values in std containers, woudn't using for_each and a lamba be easier?:



int Total = 0;
std::for_each(in.begin(), in.end(), [&Total](MyStruct x){ Total+=x.memberVariable;});
std::cout << "Total: " << Total << std::endl;
c++11 only obviously.
Check out https://www.facebook.com/LiquidGames for some great games made by me on the Playstation Mobile market.

Oh, that brings a tear to my eye.

With all the time I spent on optimizing, that is usually the first kind of code to get eliminated. Even a small container, perhaps with 20 items, could suffer a penalty of several nanoseconds in accumulated overhead. If you had thousands or tens of thousands of items, the result could approach microseconds in overhead.

Hodgman is absolutely right about that. Congratulations on writing reusable code. Now someone like me will probably need to replace it with a (usually much simpler) piece of specialized code. The Lambda function version is pretty good assuming it gets inlined, but if it doesn't, well, it will probably look like a traditional simple for loop with simple member access. Exactly like the kind of thing that was trying to be avoided in the first place.

I focus on readability-first, when possible. I can replace 'SumValuesFrom()' with a specialized function (or inline code) if needed - but until it's actually needed, isn't that premature optimization?

I imagine things are different for 3D graphically intensive AAA games... but for my 2D indie game, I can always optimize one function, or write a specialized function, when needed.

I imagine things are different for 3D graphically intensive AAA games... but for my 2D indie game, I can always optimize one function, or write a specialized function, when needed.

That is the great difficulty about optimization.

Every programmer, including me on many occasions, has followed that advice. It is often good advice because most code isn't a major bottleneck. "We don't need to specialize the functionality for performance, so this time we'll use the generic version. If this time is particularly bad we can optimize it later." Less work up front, probably never an issue.

The closest common expression is "death by a thousand cuts".

It isn't the individual slowness that is the problem.

Using a function to calculate it instead of a bit of direct code to calculate. That's fine, it adds a few nanoseconds, but understandable for clarity...

Using accessor functions rather than reading the element directly. That's also fine, it adds a few nanoseconds, but understandable for reuse...

Using virtual functions to allow classes to be customized. Sure it has a cost of a few nanoseconds, but if every subclass must follow different behavior it can be cheaper than the comparisons involved on a piecemeal solution...

Using a function to combine the funcationality to a single place. That's fine, it adds a few nanoseconds, but understandable for readability...

Putting all that in a loop. That's fine, we need to loop, it is understandable because there are more than one...

Copying the pattern into several systems. Well, okay, just be aware that the tiny inefficiencies are multiplying...

When I run a profiler and see a giant function sucking down microseconds, I love it. Easiest fix ever.

When I run a profiler and see a small function being called many thousand times, I enjoy it. I can isolate it, do some search-and-replace, maybe move something around, maybe inline something. Not hard, do the work and call the job done.

When I run a profiler and see a common pattern is being wasteful, I can figure out a fix, email the team to watch for it, hunt down bad offenders, and spend a few days trying to clean up. It is work, but part of the job description; not every day is unicorns and rainbows.

When I run a profiler and see nothing that stands out, but there are thousands of tiny inefficiencies that each require individual assessment, I start having nightmares and consider following a career advancement opportunity elsewhere.

if your already storing the values in std containers, woudn't using for_each and a lamba be easier?:



int Total = 0;
std::for_each(in.begin(), in.end(), [&Total](MyStruct x){ Total+=x.memberVariable;});
std::cout << "Total: " << Total << std::endl;
c++11 only obviously.

why so complicated?


int Total = 0;
for(auto&& x : in)
    Total += x.memberVariable;
std::cout << "Total: " << Total << std::endl;

No need for for_each or lambdas.

This topic is closed to new replies.

Advertisement