So what they give to charities? Jim Jones killed his followers and Sai Baba sexually abused many of his.
That makes them very bad people in the eyes of society and actually, criminals.
Why do you need to rationalize your involvement like that? It's okay to admit you followed a deviant pervert who has no redeeming qualities. It isn't your fault he is sexually abusing pervert, either. You are not him. All of the bad he has done far outweighs any good and he belongs in a prison because of it.
You can call me a black and white thinker Andries, but I grew up with a very bad person and nothing he ever did that can be judged as good (and there are many things that could be placed in that category) can ever justify the horrendous bad he did to me and my four sisters for the first twenty years or so of our lives. Actually, my father has never stopped trying to abuse and control my family for our entire lives, but that's another forum altogether and definitely a book.
Look, you don't have to take on and feel the shame that belongs to Sai Baba, any more than I have to take on the shame of being my biological father, or Prem Rawat, for that matter.