I think there is a big difference between the fear caused by the two scenarios you describe. I fear governmental misconduct much more than private violence. If a private person attacks my wife or me, I can fight back. If a police officer attacks one's wife (as the LAPD is prone to do the poor or black), the only remedy is to endure it and then file a lawsuit - at which point the officer will lie and then seek retaliation. The same is true with an evening intruder. If armed men (robbers) enter my home, I can fight back - or at least try. If armed men (the police) enter my home because they are too stupid to properly fill out a search warrant application, they will kill me and get away with it. The list can go on ad infinitum. The IRS going after my money causes me much more concern than you going after my money.

So while both situations cause much fear, the rape of your wife by a government actor is much greater cause for concern. I think this holds true for anyone who has been on the receiving end of unconstitutonal state action.