This is mostly speculation, but I would say there are 3 major reasons:
- Cost. Adding any sort of custom feature is usually quite costly, and the way homes are sold/built, most people just don't care enough. I'm currently building a house, and I would guess that adding a built-in shelf like that from a builder would run somewhere in the $1000-$5000 range, depending on size and how elaborate you'd want it. A book shelf at Ikea costs <$100.
- Most people don't really have much of a vision of how exactly they're going to use a space until they start moving their stuff into it. If you have built-in shelves all over the place, it really limits where you can place other furniture.
- More minor, but there's all kinds of things running through walls, like insulation, plumbing, and electrical. This can of course be routed differently, but that adds complexity/cost.
Unless you're dealing with a load bearing wall, there are no structural reasons to have the wall in the first place, so you can pretty much do with it what you wish. This could even be done in a load-bearing wall. Most outside walls are load-bearing, and they have windows and doors in them, which is essentially the same concept as the built-in shelf. It just adds more complexity again, but it's entire possible.
I think today, you would see this kind of thing much more in custom / higher-end houses, versus cookie-cutter suburban ones - mostly for the reasons I mentioned above. It's interesting that this seems to be more common in older homes. The phone shelf was ubiquitous, and other built-ins are really common in old houses as well compared to today. Maybe this is due to these houses being "higher end" in their day, and therefore they are the ones more likely to survive into today? More of a question for an architecture historian I guess.