Being a false prophet, she capitalizes on peoples' devotion to God as a means to manipulate them into acts of depravity by naming pre-selected "sinners" in their ranks.
As Leah, she claims to be a prophet, and "foretells" where the next demon attack is going to be. She even gives them an "exorcism" that expels the demons, which is in Enochian. However, she is the one controlling the demon attacks and the exorcism actually means, as translated by Castiel, "You breed with the mouth of a goat."
Upon discovering her true identity, Dean Winchester kills her using a stake made from the branch of a Cypress tree. He was able to successfully kill her due to his being Michael's true vessel, and therefore a true servant of Heaven, the only thing said to be able to kill her.