I would not fret of remembering a syntax of any given language. With anything you don't use on a regular basis, it will fade.

The crucial part here is to learn the design principles and patterns that cross over into any language. Just because the implementation of the observer, factory, or insert any other pattern may vary from language to language, the goal and purpose of that pattern remains the same and this is where the biggest value comes from. Once you recognize that and recognize the importance of Single Responsibility and Separation of Concerns -- there really isn't any problem too big you can't solve regardless of the language's syntax.

Languages are overestimated.

Of course, C++ has its specific features (and problems). But after all, it's just a tool. Most programming and design principles are very similar to other C-ish languages, like C# or Java.

Sure, you will lose some skill if you don't practice - same as for any other type of ability.

So if you can land the job you're after - do it. If you need, you can continue with C++ any time later.