U.S. Geological Survey FAQ and U.S. Geological Survey page on sinkholes:

"A sinkhole is a depression in the ground that has no natural external surface drainage. Basically this means that when it rains, all of the water stays inside the sinkhole and typically drains into the subsurface.

Sinkholes are most common in what geologists call, “karst terrain.” These are regions where the types of rock below the land surface can naturally be dissolved by groundwater circulating through them. Soluble rocks include salt beds and domes, gypsum, and limestone and other carbonate rock. Florida, for instance, is an area largely underlain by limestone and is highly susceptible to sinkholes.

When water from rainfall moves down through the soil, these types of rock begin to dissolve. This creates underground spaces and caverns.

Sinkholes are dramatic because the land usually stays intact for a period of time until the underground spaces just get too big. If there is not enough support for the land above the spaces, then a sudden collapse of the land surface can occur."

"The most damage from sinkholes tends to occur in Florida, Texas, Alabama, Missouri, Kentucky, Tennessee, and Pennsylvania."

Learn More:

• Living with Karst (Booklet), American Geosciences Institute
  This booklet defines "karst" and why karst areas are important. It also presents karst-related environmental and engineering concerns, guidelines for living with karst, a map of karst-prone areas, and sources of additional information.
   
• Sinkholes (Webpage), U.S. Geological Survey
  Overview of sinkhole formation and types of sinkholes from the USGS Water Science School
   
• List of Sinkhole Maps by State (Maps), American Geosciences Institute
  AGI's Maps and Visualizations database provides links to maps containing information about sinkholes in Florida, Kentucky, Minnesota, Missouri, and Virginia.

• Digital Engineering Aspects of Karst Map (Map), U.S. Geological Survey
  Map (including downloadable GIS data) showing the distribution of karst (and thus sinkhole vulnerability) in the United States. This map does not include information about areas where large underground cavities, fissures, tunnels, or other karst-like features have developed due to mining or other human activity.