Jump to content

  • Log In with Google      Sign In   
  • Create Account


Should I break_the_build?

  • You cannot reply to this topic
13 replies to this topic

#1 Lith   Members   -  Reputation: 322

Like
2Likes
Like

Posted 23 June 2014 - 01:53 PM

I'm going to preface this topic with the fact that I've only every worked alone on projects.

 

In the project I'm working on, I have got to a stage where I've finally decided to refactor one of the systems that a lot of the total code depends on. Refactoring this would probably mean refactoring a lot of other code afterwards. Now I realise that I've got to this scenario through a horrible combination of bad design choices and lack of forethought, but my question is what to do when I find myself in this situation.

 

Because I'm working alone on this project, it doesn't really matter if I just make a new branch, completely refactor the system and then all of the other code afterwards. This could mean that the project won't actually build for days. Again, not too much of a problem considering that I'm working by my self.

 

What would be your advice if I encounter this when I'm working with a team of programmers? Assuming that the refactoring work HAS to be done, how can I approach this problem so that I cause minimum disruption to the other programmers?

 

Thanks for your time.


Edited by Lith, 23 June 2014 - 02:51 PM.


Sponsor:

#2 dmatter   Crossbones+   -  Reputation: 2975

Like
10Likes
Like

Posted 23 June 2014 - 02:17 PM

Disruption just has to be managed. Usually there are a few tricks available that can be juggled during the disruptive period.

Private branches allow you to keep the mainline stable for other devs.

Communication is important, you can warn people to steer clear of the code you are changing, on pain of a hellish merge otherwise. If people need to work on the same code at the same time then taking small steps with frequent published commits works well, that way everyone can keep synchronised.

You can also usually achieve partial refactorings, where temporary shim layers and hacks at the seam between new and old soon-to-be-dead code can allow the program to work as normal.

Different shops take different views on this, but it may be perfectly tolerable to enter a period of instability or even brokenness during a radical change - so long as it's planned for and communicated.

#3 Lith   Members   -  Reputation: 322

Like
0Likes
Like

Posted 23 June 2014 - 04:47 PM

Thanks dmatter, good info.

 

You can also usually achieve partial refactorings, where temporary shim layers and hacks at the seam between new and old soon-to-be-dead code can allow the program to work as normal.

This seems like a very useful skill to learn. I'm going to try and challenge myself to keep the project working as it was previously at every commit, while still slowly changing things around and refactoring.



#4 Hodgman   Moderators   -  Reputation: 27586

Like
9Likes
Like

Posted 23 June 2014 - 05:18 PM

Yeah this is what branches are for :)
The master branch should always be in a working condition, but your own WIP branches can be broken, as long as no one else on the team expects them to be working.

At my last job, it was almost impossible to break the master branch. Any commits to it were queued up in a staging area, where a build machine would automatically merge/rebase them, compile for every platform/configuration, rebuild any data files if you'd changed the tools, then run the game through a series of tests and AI vs AI matches on every platform. If anything went wrong, your commit would be rejected (and you'd be emailed the error log).
It meant that committing to master took 20mins, but also that the build was never broken. Best system I've ever worked with :D

In other jobs, sometimes a broken build will cause other people to waste half an hour of their time. When you're on a team of 50 people, that's actually a rediculously large amount of money in wasted wages!

#5 SeraphLance   Members   -  Reputation: 1277

Like
2Likes
Like

Posted 23 June 2014 - 11:19 PM

Yeah this is what branches are for smile.png
The master branch should always be in a working condition, but your own WIP branches can be broken, as long as no one else on the team expects them to be working.

At my last job, it was almost impossible to break the master branch. Any commits to it were queued up in a staging area, where a build machine would automatically merge/rebase them, compile for every platform/configuration, rebuild any data files if you'd changed the tools, then run the game through a series of tests and AI vs AI matches on every platform. If anything went wrong, your commit would be rejected (and you'd be emailed the error log).
It meant that committing to master took 20mins, but also that the build was never broken. Best system I've ever worked with biggrin.png

In other jobs, sometimes a broken build will cause other people to waste half an hour of their time. When you're on a team of 50 people, that's actually a rediculously large amount of money in wasted wages!

 

We have that all the way down to the "rejection" part -- bad commits just break the continuous builder, which seems to me to kind of defeat the purpose of the entire pipeline.

 

That said, yes, in a professional environment, the less downtime in the build, the better.  If you need to fix bad architecture, then make a branch (or if you're using something like hg/git, just have a local repo).  Even with a totally screwed up arch, you can usually fix each component piecemeal.

 

EDIT:  Honestly, I think this applies to hobby stuff too.  The more I treat my hobby work like "professional" work, the happier I am.  A lot of work in professional infrastructure goes into reducing headaches and downtime, and I like not having headaches and downtime.  I don't think I'll ever make a continuous build server for my side project, but things like keeping the build unbroken, data-driving the game, and pushing what you can into runtime make the process a lot more enjoyable in the long run.


Edited by SeraphLance, 23 June 2014 - 11:22 PM.


#6 frob   Moderators   -  Reputation: 18836

Like
1Likes
Like

Posted 24 June 2014 - 07:48 AM


Refactoring this would probably mean refactoring a lot of other code afterwards.

You keep using that word. I do not think it means what you think it means.

 

"Refactoring is the process of changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure." -- Martin Fowler (one of the small group of authors who put the term into popular use)

 

When you refactor a chunk of code it should break exactly nothing. 

 

If you are rewriting methods or rewriting your API, then that can break things.

 

 

I recommend constantly refactoring your code. Refactor aggressively.

 

I like to think of it as you would work in a shop. Refactoring is much like sweeping up the scrap bits and sawdust. You need to clean up the code every time you work with it. Failure to do so and you will quickly find yourself ankle-deep in sawdust and unused scrap bits.


Check out my personal indie blog at bryanwagstaff.com.

#7 zee_ola05   Members   -  Reputation: 313

Like
1Likes
Like

Posted 24 June 2014 - 11:29 AM

"Refactoring is the process of changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure." -- Martin Fowler (one of the small group of authors who put the term into popular use)

 

 

What do you call

If you are rewriting methods or rewriting your API, then that can break things.

 

Can't we call this refactoring too?

 

And I agree, we should refactor aggressively. :)



#8 zee_ola05   Members   -  Reputation: 313

Like
1Likes
Like

Posted 24 June 2014 - 11:32 AM

 

"Refactoring is the process of changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure." -- Martin Fowler (one of the small group of authors who put the term into popular use)

 

 

What do you call

If you are rewriting methods or rewriting your API, then that can break things.

 

Can't we call this refactoring too?

 

And I agree, we should refactor aggressively. smile.png

 

Redesign, maybe? I'm asking because I usually misuse the word "refactor" too.



#9 frob   Moderators   -  Reputation: 18836

Like
0Likes
Like

Posted 24 June 2014 - 11:45 AM

It is all about the interface, or the API, or the contracts that you give the user of your library.

 

It might be that you are the only user of the library. It might be that millions of people use your library. Either way you are providing an interface, a contract, that they can call the function and get reliable results.

 

Rewrite generally implies that you are throwing it away and starting over from scratch. The entire interface will be different.

Redesign generally implies that pieces will change but much will remain. Some of the interface will be different.

Revision generally implies new features are added. Interfaces will change slightly to accommodate new features.

Refactor means internal changes only. The interface will remain unchanged.


Check out my personal indie blog at bryanwagstaff.com.

#10 Lith   Members   -  Reputation: 322

Like
0Likes
Like

Posted 24 June 2014 - 12:26 PM

Rewrite generally implies that you are throwing it away and starting over from scratch. The entire interface will be different.

Redesign generally implies that pieces will change but much will remain. Some of the interface will be different.

Revision generally implies new features are added. Interfaces will change slightly to accommodate new features.

Refactor means internal changes only. The interface will remain unchanged.

 

Ah my bad. Thanks for pointing that out Frob.

 

I guess what I'm trying to do is redesign without breaking anything.



#11 King Mir   Members   -  Reputation: 1909

Like
0Likes
Like

Posted 24 June 2014 - 12:33 PM


Refactor means internal changes only. The interface will remain unchanged.
That's still too broad. If you change a back end algorithm, that's not a refactor. A refactor only changes structure.

 

I'd say the archetypal refactor is moving common code to a common interface, like an inheritance hierarchy. You're not adding new functionality, but you're changing the code to make code reuse easier in the future. You wouldn't do this for public facing code, because that would change the API and ABI, so I guess the observation that a refactor is internal only is correct,



#12 Kian   Members   -  Reputation: 236

Like
0Likes
Like

Posted 25 June 2014 - 06:23 AM

That break up works for libraries, but when you're talking about a program, the "external behavior" is what the user sees. You could make major modifications to your code (adding or removing libraries, adjusting inheritance structure, etc), and so long as the program behaves the same I think calling it a refactoring would be accurate. It follows from the definition; the internal structure is whatever code you have in your program, the external behavior is what the user sees.

#13 Eck   Members   -  Reputation: 1712

Like
0Likes
Like

Posted 25 June 2014 - 08:07 AM

Like others have said, long refactors (or redesigns or rewrites or whatever you want to call them) have to be coordinated with your team like others have said. Try to minimize working in the same areas. And if there is a hellish merge that is necessary, make sure the person responsible can handle it. Don't hand it off to a mouth-breathing intern "because it's just grunt work."

 

And don't feel bad for getting yourself into this situation. It happens to everyone, even high-paid architects whose job it is to make sure it doesn't happen. You can lessen the impact of bad design choices by spending a little time each week looking at your pain points.  What new features were harder than they should have been? What existing code is a pain to work with? What code do you dread touching for fear of it breaking something? And then, once you have your pain point identified, spend time cleaning up the mess. When you don't do this on a periodic basis, the mess gets bigger and bigger until it gets overwhelming to fix the problem. So you put it off longer, which makes the mess bigger, which makes you put it off longer... etc.

 

So do yourself a favor and clean up your code every so often or you'll find yourself scrapping the whole damn project and starting over.

 


At my last job, it was almost impossible to break the master branch.

...

Best system I've ever worked with 

 

Nice Hodgman. We had a similar system at my last company with single-button packaging, and single button deployment. Our details sounded a little less sophisticated than yours but I still miss it. We even had our overall software development process nailed down to something that made everyone happy (devs, project managers, qa, and the client). Then the money-grubbing execs messed everything up by being uber-jerks and the team scattered to the four winds.

 

- Eck



#14 zee_ola05   Members   -  Reputation: 313

Like
0Likes
Like

Posted 25 June 2014 - 12:59 PM

Since this is related, I'd like to share the process we adapted in my previous work - Git Flow. This is a nice model to follow if we want our master/develop branch to be always working. It is pretty simple too.

 

http://nvie.com/posts/a-successful-git-branching-model/







PARTNERS