Neogene climates appear(ed) to be suitable to study the upper limit of climate change. The time interval 4-5 Ma is the warmest time interval with persisting "El-Nino"-type temperature distributions. Coupling this ocean with an atmospheric GCM yields non-reconstructable parameters such as wind-fields and snow-fall. The resulting snow-rich and cold winters on the Northern Hemisphere (NH) continents are consistent with observed ice rafted debris (IRD) during the Pliocene in the Norwegian Sea (ODP site 642B) and respective sites in the North Pacific. The IRD is data that is independent of SST reconstructions and modelings. Thus the Pliocene had colder aspects than generally thought. Other reconstructed Neogene time intervals, such as 5-6 Ma are in northern higher latitudes, using the same reconstruction method, considerably colder, even though in the tropical oceans high SSTs occur. Side aspect: The pattern "high tropical SSTs in coexistence with present Arctic sea ice and observed high snowfall in subsequent winter on NH continents appears to be comparable to Neogene environments (possibly an early indicator of one possible trajectories of climate change?). Thus Neogene and Pleistocene environments might appear useful as proven scenarios for future climate change: But: All Pleistocene warm time intervals are characterized by the co-occurrence of temperatures of 3 degrees C above preindustrial levels and IRD (e.g. ice floes) in the Norwegian Sea and the North Pacific. High Pleistocene "greenhouse"-temperatures did not prevent IRD in the Norwegian Sea and Arctic Sea ice further north. Thus in the Neogene an ice covered Arctic ocean coexisted with high tropical SSTs. Today these Pleistocene high temperatures are still not reached. The temperature levels of warm Pleistocene times--with IRD in the Norwegian Sea--appeared in the past as one upper limit of climate change. Today, at least in the Norwegian Sea at ODP site 642B, no IRD occurs, neither in summer nor in winter. Thus, data based, one boundary condition of the climate system is today different from the Pleistocene and Pliocene. Thus the Holocene appears not be one of several interglacials any more. In this case IPCC scenarios of future climates, that regard +6 degrees C, e.g. well above the +3 degrees C of Pleistocene warm times, as possible, might not be excluded. In this case Neogene climates might only partially be a key to the future. Alternatively, which needs to be tested, high SSTs in coexistence with Arctic Sea ice generate conditions (wind fields during NH winter, ice floes) that lead to IRD in the Norwegian Sea. To test this, long transient runs of coupled atmosphere/ocean models are needed. If the climate fluctuations of the past are met (overall) correct, the used boundary conditions apply. If the boundary conditions need to be modified until the model reproduces the known climate fluctuations testing by subsequent drilling provides the data.