The history of the park is both colorful and varied. The park is named for Thaddeus Stevens' charcoal iron furnace which began operation in 1837. The Honorable Thaddeus Stevens was born in Caledonia County, Vermont and became a famous abolitionist, statesman and father of the public school system in Pennsylvania.

During the Civil War, the area was visited by Confederate troops. Because of the conflict that was in progress between the North and South and in part due to Mr. Stevens views on freeing those in slavery, the furnace was destroyed in June of 1863 by Confederate Cavalrymen under the command of General J. A. Early. The scenic watercourse along PA Route 233 was once the millrace that operated the furnace waterwheel.

Pastures were used as field hospitals for the wounded taken from the Gettysburg Battlefield. Today they are playfields for visiting children.

In 1927, the Pennsylvania Alpine Club reconstructed the stack of the old furnace as a reduced scale monument. The reconstructed stack and blacksmith shop are the only visible reminders of the early iron works.

The park is one of the oldest Pennsylvania state parks and is also one of the most popular in the Commonwealth. The park was first operated privately, then later by transportation companies that promoted it as an amusement park.

Years ago, the area was the scene of carnivals, balloon ascensions, family picnics and any other event that with fanfare and promotion served to attract large crowds.

Caledonia is the name given to ancient Scotland by the invading Romans.