Very well said by Mark Dever in The Deliberate Church: Building Your Ministry on the Gospel (note: this might be slightly paraphrased, because it’s from my notes and I don’t have the actual copy of the book with me right now)
The difference between Christians and non-Christians is not that non-Christians sins whereas Christians don’t. The difference is found in what side we take in the battle. Christians take God’s side against sin, whereas non-Christians take sin’s side against God.
A Christian will sin, but turn to God again and his word and say “help me fight my sin.” A non-Christian, even if he recognizes his sin, effectively responds, “I want my sin more than God.”