The 3-story native sandstone Castle was the first home of Baker University, Kansas' oldest 4-year college. Listed on the National Register of Historic Places , it was built in 1857 and 1858, at a cost of $3,200 and includes native wood - mostly walnut - woodwork. It provided a "sight to behold" for pioneers traveling the historic Santa Fe Trail during the 1850s and 1860s who could see it for miles over the treeless prairie before their rumbling wagons actually came alongside it.
It now houses an outstanding museum of the University and Baldwin as well as the Santa Fe Trail. The third floor, originally donated by the Palmyra Masonic Lodge, has been designated as the state's Masonic Historic Lodge #1.
Also on the Complex grounds is the Palmyra Post Office and the Kibbee Cabin.