Sri Krsna and Sri Caitanya only appear once on a day of Brahma. Once in 4.32 billion years.
Some say that Sri Caitanya appears more often in other Kali Yugas, but not in His full form.
Either way, it's a long time. Half a millennium or several millennia apart. There are no historical records.
What exists is an inspired Sadhu who claims that this pastime took place here in this place. And it is established that it is in this location.
It is not just the birthplace of Mahaprabhu. In several other places there is no agreement. Some say it's in a location. Others say it's somewhere else.
There are no historical records to prove things that happened millennia and millions of years ago.
The exact location really isn't very important. The important thing is to celebrate the pastime.