Various accounts exist of the origination of Durga puja in the state of Orissa, and all historical accounts agree that important influences came from many regions outside of Orissa state. Durga Puja is a festival that is observed for 10 days, and in two different ways in Orissa. In Shakti peethas (temples of the goddess) the Durga Puja is observed with proper rituals for 10 to 16 days, known as Shodasa Upachara. Goddess Durga is also worshiped across the state by devotees who erect beautifully decorated pandals bearing Durga Ma in her various Deity forms.
According to Markandeya Purana, the King of Chedi dynasty, Suratha, started rituals of Durga Puja during 300 B.C. The Chedi dynasty belongs to Kalinga (modern Orissa). Durga Puja has different names in different Puranas and Sastras. In Devi Purana and the Kalika Purana it is called Vijaya Dashami. In the Devi Mahatmya it is called Mahaparbana, and in Markandeya Purana it’s called Duseehera.
The present form of Durga worship in Orissa is that of earthen idols, which began during the reign of Ganga King Chodaganga Dev in the 11th century at Puri. The earthen idol of Mahishamardini Durga is known as Gosani, and the Dussehra fesival is also called the Gosani Yatra. It is noteworthy that the co-worship of Mahisamardini Durga with Madhava (Lord Jagannath) was prevalent from the 11th century, Ganga period in Puri.
More recently, in the period 1512 to 1517, Caitanya Mahaprabhu paid a visit to Cuttack, the capital of the Gajapati empire of Orissa. The then emperor of Orissa, Gajapati Pratap Rudra Dev, known as King Prataprudra, received him at Gadagadia Ghata, situated near the river bank of Mahanadi. Gadagadia Ghata is very close to the king’s Palace, which is popularly known as Barabati Fort. At that time, it is said that Sri Caitanya Deva started Durga Puja at the Binod Bihari temple, which is presently known as Balu Bazar. 
In his paper, Life of Sri Chaitanya Mahaprabhu, Srila Bhaktivinoda Thakur described Lord Caitanya’s travels to Cuttack:
“He traveled along the side of the Bhagirathi as far as Chhatrabhog situated now in Thana Mathurapur Diamond Harbour, 24 Parganas. There He took a boat and went as far as Prayag Ghat in the Midnapore District. Thence He walked through Balasore and Cuttack to Puri, seeing the temple of Bhubaneswar on His way. Upon His arrival at Puri He saw Lord Jagannath in the temple and put up with Sarvabhauma at the request of the latter.”
Gajapati Pratap Rudra Dev was very strongly influenced by Sri Caitanya’s uddanda kirtan during his visit to Puri. King Prataprudra surrendered to Caitanya Mahaprabhu as his Spiritual Master, and took diksa initiation from him. From this point onward, Lord Jagannath became more widely known as Sri Krishna. By the mercy of Mahaprabhu’s visit to Puri Dhama, the traditional observance of Chandana Yatra, Janmastami, and Radhastami, were popularly celebrated at Sri Mandir from the 16th Century AD onward.
It is also said that when Caitanya Mahaprabhu went to Orissa, an ancestor of Shri Akshya Kumar Banarjee, who is presently engaged as a priest in the puja performed at Balu Bazar pandal, was personally in the retinue of Lord Caitanya.  That ancestor established worship of Binod Bihari and Banka Bihari in the temple at Balu Bazar, or Binod Bihari temple. He is also credited with starting Saradiya Puja of Devi Durga, in the form of Ghata Puja.
We find very few instances in Gaudiya Vaisnava literature where references are made to Lord Caitanya and Durga worship. In Bhagavad-gita As It Is, and throughout Srila Prabhupada’s teachings, we are instructed not to engage in demigod worship, but to simply worship Sri Sri Radha-Krsna. Durga-puja and Dussehra are not observed in ISKCON temples (or should not be, according to Srila Prabhupada’s standards). Srila Bhaktisiddhanta Saraswati Thakur once said that ‘Durga Puja is not dancing for Hindi songs, taking intoxication and having fun’, as we typically find in puja pandals.  Srila Prabhupada said that such worship of Durga is done by fools.
In Sri Caitanya-caritamrta, Adi-lila 17.37-38 we find an interesting passage regarding the mood around Durga worship amongst Gaudiya Vaisnavas:
“One night while kirtana was going on inside Srivasa Thakura’s house, a brahmana named Gopala Capala, the chief of the nonbelievers, who was talkative and very rough in his speech, placed all the paraphernalia for worshiping the goddess Durga outside Srivasa Thakura’s door.
This brahmana, Gopala Capala, wanted to defame Srivasa Thakura by proving that he was actually a sakta, or a worshiper of Bhavani, the goddess Durga, but was externally posing as a Vaisnava. In Bengal there is perpetual competition between the devotees of Goddess Kali and the devotees of Lord Krsna. Generally Bengalis, especially those who are meat-eaters and drunkards, are very much attached to worshiping the goddesses Durga, Kali, Sitala and Candi. Such devotees, who are known as saktas, or worshipers of the sakti-tattva, are always envious of Vaisnavas. Since Srivasa Thakura was a well-known and respected Vaisnava in Navadvipa, Gopala Capala wanted to reduce his prestige by bringing him down to the platform of the saktas. Therefore outside Srivasa Thakura’s door he placed various paraphernalia for worshiping Bhavani, the wife of Lord Siva, such as a red flower, a plantain leaf, a pot of wine, and reddish sandalwood paste. In the morning, when Srivasa Thakura saw all this paraphernalia in front of his door, he called for the respectable gentlemen of the neighborhood and showed them that at night he was worshiping Bhavani. Very much sorry, these gentlemen called for a sweeper to cleanse the place and purify it by sprinkling water and cow dung there. This incident concerning Gopala Capala is not mentioned in the Caitanya-bhagavata.”
While Durga worship is not recommended by our Sampradaya Acaryas, we understand that Durga Ma is the Lord’s devoted servant, and she will serve Him diligently as she takes charge of the material body of each one of us, at the time of our death. In Caitanya-caritamrta, Madhya lila 9, we read about the many holy places Mahaprabhu visited during His manifest lila. During His travels in South India, at Sri Saila-parvata, Sri Caitanya personally met Lord Siva and his wife Durga, who were in the dress of a brahmana and brahmani.
In another of Lord Caitanya’s pastimes at the home of Candrasekhara Prabhu, in what is now known as Vrajapattana (Caitanya-caritamrta Adi lila 10.13), all the devotees put on a drama. Sri Caitanya Mahaprabhu came onstage, but no one recognized Him. Through the Yogamaya potency, the Lord revealed His female forms – all the saktis, Rukmini, Durga, Laksmi devi, and so many others, one after another.
Considering Durga Ma’s transcendental relationship as the Lord’s servitor, we can better understand Lord Caitanya’s pastimes in Cuttack, establishing Durga Puja at the Balu Bazar.