Why feel any guilt over having prejudices and maybe some residual guilt? I am sure I am racist and likely sexist if bias alone makes one racist. Therefore, I feel as a growing individual, I feel an obligation to be aware of my prejudices and treat every individual I meet with fairness and without bias.
We may not be able to entirely change our viewpoints, but we can change our behaviors as part of our human responsibilities. Changing one's behavior can slowly lead to one's changing his or her viewpoints. I mentioned this viewpoint of mine to friends who are from minority groups; they don't see this viewpoint of mine as "racist", but as one having many natural biasies as do they.