Freeman was doing the job like mythbusters, revealing how magicians trick/fool the people. And no he was not working for the bad guys as he turned down the offer of the insurance guy of millions of dollars twice (first not taking the offer of not exposing), second time to put down the 4 horseman. he could have taken it but indirectly he helped the 4 horseman by not taking up the offer.
as for ruffalo holding freeman responsible for his father's death. that was absurd logic portrayed in the film. coz his father worked out a new trick in order to outdo freeman since he had exposed his previous tricks. but the new trick didn't worked out prolly coz he was not so good a magician.
But who in their right mind would plot the revenge for 18 yrs, live an undercover life of an fbi agent (when he could have bumped off freeman in other ways) just to avenge death of his father's own mistake. not to mention how he accumulated the resources to form himself as the eye of the horus.