Hello all. Working on a large refactor as usual...
One of the biggest issues I deal with is what I call software rot.

Software with rot tends to be riddled with cross class dependencies,
too many [potentially stale] internal and external couplings.
Inconsistent data structures and threading models. Odd uses of friend
classes, etc.

It all comes to down to too much coupling. Personally I want to blame
logical coupling; however, that is just my bias against the current codebase
I'm working against...

We have a core set of classes that we reuse. For that matter... those classes
are contained in a few components that all our projects reuse. As we add
more abilities the classes tend to get larger and their concepts get muddled.

It finally gets to a point where responsibilities shift and an entire section
must be refactored. This is where I feel the rot comes in. One area gets
reconstructed with the abilities and paradigms--but it must maintain
compatibility with the rest of the code.

Internally we have to start building facades or end up with base classes
with dozens of methods and delegates; and/or, the logical dependencies
cause rippling bugs.

Each time a class/component is reused... the more the rot mutates it into
something other than what it was originally intended to be.

Obviously part of this comes down to the granularity of the classes. Small
classes tend to stay stabler longer; however, as each generation of the
software is developed--even the small classes tend to grow and grow.

One could make base classes so small that their abilities never change;
however, that only moves the problem. You would still end up with aggregate
classes either inherit or encapsulate dozens and dozens of children. This
itself adds coupling and complexity and ends up being a lot of typing to
tie the interfaces together.

I would like to start a discussion of general ways of coping with this
problem: anticipating and preventing rot in new projects, coping with
debugging and extending large systems already infected with rot, etc.

[Edited by - Mathucub on November 26, 2010 2:26:54 PM]

Why not make a copy of the classes for new projects and leave the old classes unchanged?

Quote:Original post by CzarKirk
Why not make a copy of the classes for new projects and leave the old classes unchanged?

That can work, but it seems to only move the problem. Say there is a bug
in a class does "X". You figure out there is a logical error in "X", or
you have to upgrade it. Then you have to find each of the places where it "X" is
implemented and fix it.

Which is worse? Tracking interface changes or all the places where implementations
must be maintained.

Quote:Original post by KulSeran
For starters, I'd say try to focus the developers.
I'd like to claim that a lot of the mutation style rot comes from several people doing a design-as-you-go set of changes to one item.
If you end up having several people working on item X, make sure they see the whole picture. The task assignment shouldn't be "go make a gun" if the real task is "go make a gun, in the future it will support these 6 mods". If that whole-picture isn't in everyone's head, then you'll get a lot of code in place that feels hackish, as it was added for the "now" instead of being future proof around the whole design.
A lot of the post-mortums on GamaSutra mention stuff like this. Teams get a lot of mileage from having only one person working on AI, having a dedicated artist tools guy, or having "agile" type small teams of a programer+artist+designer. The mileage usually comes more from the small task forces having full knowledge of the task, and not necessarily from one person just being godly at some task.

That does seem like a good way to manage a project. Especially if a project has a
finite lifetime. I wonder how we could adapt that into an unbounded project; or
a "rotten" project whose codebase is inherited.

When working on toolkit style libraries, core abilities can be added for years.
The one I'm working on is 10 years old. I still have stubbed out interfaces for
old DX6. [We actually still had a client requirement to support it until last
year.]

As an over the top example; "a gun with 6 mods" would be implemented very
differently if it were a 2D spite vs. a 3D model.

Say one had two perfect "guns" that ran perfectly in their respective domains;
it seems that the prevention of rot would have to focus on strong encapsulation
between the cases.

Quote:Original post by Mathucub
Which is worse? Tracking interface changes or all the places where implementations
must be maintained.

Interface does not change. Once the interface has been released, it's fixed. Precisely due to the cost of upgrading.

Secondly, lay off the OO. Not everything needs to be an object. Class invariants need to be wrapped into objects. The rest are free functions. If using one of strict OO languages, then IoC solves many coupling problems.

Quote:We have a core set of classes that we reuse. For that matter... those classes
are contained in a few components that all our projects reuse. As we add
more abilities the classes tend to get larger and their concepts get muddled.

What you have is a big ball of mud that everyone messes with. This is not reuse, it's the anti-thesis of reuse, the worst of practices. Spaghetti code, bent into whatever shape it currently needs to be.

If code needs to be modified so frequently, then it's not reusable in the first place.


When thinking about reuse and core constructs, think Facebook. They use LAMP. They do not modify MySQL, PHP or Apache. They may tweak them from time to time, but the interfaces between those are fixed.

Quote:This itself adds coupling and complexity and ends up being a lot of typing to
tie the interfaces together.

Complexity is inherent to problem being solved.

To paraphrase: "When tackling some difficult problem, people say: I'll make this class take on more responsibility. Now they have two problems.".

Quote:Internally we have to start building facades or end up with base classes
with dozens of methods and delegates

If you sweep shit under the carpet, the room will look cleaner, but will still stink.

How many methods a class has does not matter. It's the responsibilities it has. Does it have more than one? Split it.

The OO approach to design wants to have designers provide these idiot-proof baby-could-use class interfaces. Unfortunately, people designing them have no clue on how to solve the actual problem. So instead of trying to fix that, they pile on abstraction over abstraction, until original problem is buried deep enough to no longer matter.

There is no real benefit in hiding everything into classes. By covering your eyes, complexity does not go away.

Large scale, underdefined and generally fluid projects often benefit from IoC. That one decouples data from logic, making each of those independently extensible. It's no different from C's design - it doesn't need member functions to remain explicit. And yet all OSes are written using it. And those are *much* more complex problems than developers ever encounter.


But the underlying issue described here is the usual lack of any kind of organization and management. The spaghetti-style development, where everything is touched by everyone as needed, no forethought is given into anything, solutions are half-baked and rushed.

Adopting TDD would force lower coupling due to more testing.

The second problem is the OO overload. "Everything is a class, everything is private, everything is encapsulated". In C++, only class invariants should be encapsulated. The rest are free functions. In C# or Java, as mentioned, IoC, either manual or automatic. Equivalent to free functions, as far as language design allows.


These are proven methods. Think about it:
- when some class X lacks functionality for project P, rather than breaking class X, a free function is added in P's filesystem. If this function later needs to be reused, it's promoted to P's parent. But the interface remains fixed.
- If X is no longer sufficient, there are two choices. X is extended without changing class invariants or X gains another class invariant (watch the single responsibility). Either way, from outside interface to other projects, these changes are idempotent. They do not affect functioning of anything else.
- If X is no longer sufficient, but extending it would violate invariants - then X is fine, add Y. Don't use a bigger hammer until round peg fits into square hole.

The above are very trivial rules which, when enforced through automated testing, ensure forward and backward non-breaking compatibility.

Quote:ends up being a lot of typing to tie the interfaces together.

This reminds me of a saying: "We need to save, no matter the cost". By trying to save typing through exposing everything via .OneMasterMethodToDoEverything(), you are incurring prohibitive downstream costs.

Programmers type, cooks cut, welders weld, sportsmen run... Yet by doing more or less of it is not what makes them good or bad at it. It is how, where, when and why they do it.

In general I'd say there's no magic bullet (so I'm just going to ramble on a bit here). Clear requirements and design up-front helps a lot though.

Quote:As we add
more abilities the classes tend to get larger and their concepts get muddled.

It finally gets to a point where responsibilities shift and an entire section
must be refactored.

Ideally each class should have a single responsibility, if it doesn't split it into another class (which sounds like what you do do) edit: or split it into another function as Antheus metions.

Quote:
This is where I feel the rot comes in. One area gets
reconstructed with the abilities and paradigms--but it must maintain
compatibility with the rest of the code.

If the interface of the class hasn't changed, it shouldn't break anything. If the interface of the class has changed, then you may have to update dependant code which could introduce a new bug.
Good software interface design is very important, here's an interesting paper/mp3 about API design (it's even more important for APIs to have well designed interfaces but a lot of the same rules apply).

Nothing can really prevent the introduction of bugs apart from being a careful programmer and doing testing - such as doing unit tests (possibly utilizing dependency injection so objects that are more complex than your base classes can be tested) and also having proper testers/QA.
The testing process is important.

Before going in and refactoring software I think benefits and costs need to be weighed in every case - if a component has been tested and it works - and is complete is refactoring necessary? If bugs are present why not simply just fix the bugs without changing any interface?
If you are adding new functionality then try to decide if the implementation should be part of an existing class or it's own separate class (see SRP again).