It sounds like he's a good partner for you, and those aren't too easy to come by! If you enjoy dancing with him and get along well (especially when one of you makes a mistake), that's the most important thing.
As a beginner, you don't really need to worry about competition judges--plenty of time for that in the future, if and when you ever decide to compete.
When do you feel like you are going to fall over? All the time, or just on moves, like underarm turns, that require him to reach over your head?
If it's all the time, it probably means both of you need to work on your balance a bit, so you both maintain your own balance independently. And regardless of his height, he needs to lead gently enough that he isn't pulling you over.
If it's during turns, that can be a problem. Your instructor may be able to give him some tips on how to give you more room. And you can soften into your knees on turns more than you might with another partner. (Don't duck if you can help it--it just doesn't look good.) Or if he just can't reach, there are alternatives he could use, like leading a turn with your shoulder rather than your hand. If he's on the short side, he might want to learn some of those, so he can dance with any taller woman.