There are two kinds of impediments to devotional service

According to Śrīla Viśvanātha Cakravartī Ṭhākura, there are two kinds of impediments to devotional service. The first is an offense at the lotus feet of a Vaiṣṇava. This is called vaiṣṇava-aparādha. Śrī Caitanya Mahāprabhu warned His devotees not to commit vaiṣṇava-aparādha, which He described as the mad elephant offense. When a mad elephant enters a beautiful garden, it destroys everything, leaving a barren field. Similarly, the power of vaiṣṇava-aparādha is so great that even an advanced devotee becomes almost devoid of his spiritual assets if he commits it. Since Kṛṣṇa consciousness is eternal, it cannot be destroyed altogether, but advancement may be checked for the time being. Thus vaiṣṇava-aparādha is one kind of impediment to devotional service. Sometimes, however, the Supreme Personality of Godhead or His devotee desires to impede one's devotional service. For example, Hiraṇyakaśipu and Hiraṇyākṣa were formerly Jaya and Vijaya, the gatekeepers in Vaikuṇṭha, but by the desire of the Lord, they became His enemies for three lives. Thus the desire of the Lord is another kind of impediment. But in both cases, the pure devotee, once advanced in Kṛṣṇa consciousness, cannot be lost. Following the orders of his superiors (Svāyambhuva and Lord Brahmā), Priyavrata accepted family life, but this did not mean he lost his position in devotional service. Kṛṣṇa consciousness is perfect and eternal, and therefore it cannot be lost under any circumstances. Because the material world is full of obstructions to advancement in Kṛṣṇa consciousness, there may appear to be many impediments, yet Kṛṣṇa, the Supreme Personality of Godhead, declares in Bhagavad-gītā (9.31), kaunteya pratijānīhi na me bhaktaḥ praṇaśyati: once one has taken shelter at the lotus feet of the Lord, he cannot be lost.

Śrīmad-Bhāgavatam 5.1.5