Psa 141:3-5 Set a guard over my mouth, O LORD, keep watch over the door of my lips! Incline not my heart to any evil, to busy myself with wicked deeds in company with men who work iniquity; and let me not eat of their dainties! Let a good man strike or rebuke me in kindness, but let the oil of the wicked never anoint my head; for my prayer is continually against their wicked deeds.
The Bible often talks about the heart, but it does not mean the physical muscle that resides in our chest in these instances. It represents our life, our will, our intellect, our desires, etc. Everything that should be pure for God comes from our heart.