Karma is not something that happens to you. Karma is something that happens through you. And you can change the momentum of your karma by consciously making a new choice - again and again.
You have that much freedom to make your life what you want. When the yogis say that your life is your karma, what they are saying is that your life experience is entirely up to you. How you experience anything is 100% your making.