Stephen Hawking, one of the greatest physicists of his generation (wheelchair, computer voice system and all), began his 1988 book "A Brief History of Time" with an old, no doubt apocryphal, tale of turtles. The tale has many versions, sometimes involving Hindu myth, often naming various 19th Century scientists. Basically, after a lecture on the nature of the solar system and the galaxy, a woman stands up to object that the world is a flat disc. The lecturer asked what holds up the disc. "A turtle," she says. And what holds up the turtle? "It's turtles all the way down."
I personally like a version that I'm told (by Wikipedia) was aired in a Season 5 episode of the sitcom "The Office." In it, an employee wants to organize a birthday surprise for her boss. She suggests a stripper jumping out of a cake, holding a smaller cake, out of which a smaller stripper jumps, holding a still smaller cake. And so on. When asked what the next stripper will be holding, she says: "A cupcake. It's cupcakes and strippers, all the way down."
For some reason, this "infinite regression" problem (call it the Turtle Problem) came to me while reading about an early Ionian school of "science" that believed the world could be reduced to principles that can be understood. Thales of Miletus (about 624 to 546 BC) is credited with predicting the first solar eclipse. Empedocles (roughly 490 to 430 BC) discovered there was a thing we call air. Democritus (roughly 460 to 370 BC) came up with atoms. Through careful measurements, Aristarchus (roughly 310 to 230 BC) concluded that the sun must be hugely larger than the earth, becoming the first to argue that the earth must not be the center of the solar system, but orbits the sun.
This Ionian thinking lost out (Aristotle, for instance, rejected atomism because he couldn't stomach the idea that humans are made of soulless bits) and it wasn't revived for about 2,000 years. So, during all that time, it basically was turtles (or cupcakes and strippers) all the way down.