The idea of an infinite ensemble of universes has arisen in many different contexts in physics, from quantum mechanics (the many worlds interpretation) to certain cosmological models (eternal inflation models).
In a purely instrumental approach (accept only that what you can in principle measure in experiments), one should always replace a multiverse theory by a single universe theory. However, this may not be a very natural thing to do.
Another motivation for the multiverse is this:
Physics, as it is practiced now, is necessarily incomplete. A fundamental theory needs to postulate certain fundamental physical quantities like the physical universe, spacetime, fields (or strings) etc. and fundamental laws of physics. These quantities are themselves beyond the realm of explanation. If they can be explained then that would only mean that there exists a more fundamental theory, with less fundamental quantities.
Clearly, the only possible way in which everything can (in principle) be explained is if there are no fundamental physical quantities at all. Suppose that there doesn't exist a physical universe. It could be that only mathematical worlds exist. That would mean that all possible mathematical worlds are ''as real'' as this one.
