The Great Lakes were once the valleys of a massive river system that drained into the Atlantic Ocean. Repeated periods of glaciations gouged out the valley, creating large basins that soon filled with the melted ice water of the receding ice.
The Bluffs were formed by a glacial drumlin, or pile of ground up mud, sand, and stones pushed along by the glacier as it scoured the land. Over time, Lake Ontario grew and began eroding away at the northern end of the drumlin, exposing the cliffs. Wind, rain, and snow melt-water continue to eat away at the cliffs, reshaping the Bluffs and providing us with a dynamic landscape that can be vastly different from year to year.
Drumlins are common for Western and Central NY, but one that is sliced in half like this is pretty rare. There are only three large examples of this in the region, with the Chimney Bluffs being both the largest and the most accessible. Other bluffs along Lake Ontario can be found just to the east near Port Bay and then at Fair Haven Beach State Park.