I call this function like this:int total = 0; cout << score(sol, total) << "/" << total;
But it keeps displaying the total as 0.
Your recursive function is probably working fine: this line of code is the problem.
The order of evaluation for parameters passed to cout using the << operator is effectively undefined. For various reasons, it is most likely evaluating them in the opposite order to that which you expect - it's printing the value of total first, (which is initially zero) and THEN calling the score function.
Note in your test program, you are outputting the return value of the function, and the value returned by reference in different cout calls. This works exactly as you expect.
Try using separate cout calls or call the score function outside of cout, e,g:
int total = 0; int currentScore = score(sol, total); cout << currentScore << "/" total;