Racism = Prejudice + Power. While racism can and does occur at the interpersonal level (using a slur, telling a joke, denying service, etc.), its more insidious and dangerous forms are institutional and ideological. Check out The Four I's of Oppression.
But I'm not a racist! Of course you don't do racist things like use derogatory slurs, but people with racial privilege benefit from systemic racism whether they like it or not. Even if you grew up poor, disabled, or suffered in other ways, you didn't have the added burden of racism. Check out the Privilege Checklist and see how it makes you feel. When we feel defensive about our racial privilege, it's often referred to as White Fragility.