Should I give up my dream of becoming a teacher if I'm not good at math?

I was never good at Math. I get nervous going to a cl that deals with numbers. I get embarrased when were in groups and I'm the only one who doesn't know what to do. I go to college but still don't know most of my multiplication tables. Maybe because during elementary when my cl was learning them I was always sent to ESL cl, and I never got to catch up. Recently my Math Proffesor told me maybe I was in the wrong cl, insulting but what can I do? Since I have to take math cles to get a degree, I'm thinking about just giving up.