Radioteeth, your condition has been know even before I started programming and I'm old
In DonaldKnuth's paper "StructuredProgrammingWithGoToStatements", he wrote: "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%."
So, learn to not worry too much and care about your code architecture and data handling at the start of a project.
Modern profilers do a great job of showing the real bottlenecks later on.