I can't speak with JPanel experience but language agnostic generality is that the OS sends keydown events with that delay.
Hmm.. yeah, that could work, I'll give it a shot. Thanks!
Depending on what you're doing that can either be good or bad, for something like a GUI you'd want to forward the keypress events to your gui and basically treat it normally. For something like character movement you usually want to create a keystate class and have a class repesenting each key and it's state, then update that based on OS events.
I.e. I press letter w down then set the keydown bool for w to true, then I keep it that way until a key up event is received from the listener, then you reverse it. Using that methodology you can just poll your keystate class to see if a key is still down at the moment you check it.
Essentially a long winded explanation of what neonic said.