I'm a programmer,but you could respond for designers too. Sure,a person who doesn't play games could make a game,but I think it might not be what we expect. I mean we,the gamers,are used to a certain style of gameplay. FPS,RPG,MMORPG whatever,in fps we are used with the feel of the camera,the weapons etc, a non gamer might design a game which might feel uncomfortable to us. For example,let's take a small ideea: most games use a fov of about 60 degrees,a non gamer might use a different value,since he's not used to seeing games like we do(yes,its a forced example,since everyone could probably view that the image looks kinda distorted,but if you search deeper,there are many places where a non gamer dev might do mistakes).
Personally,I think games are like movies,one should definitely watch great movies in order to make great movies.