I see some stores selling sand specifically for leveling the ground, e.g. before building a stone sidewalk or a brick wall. The local rock quarry even sells some sand for this purpose. What is so special about this? Can I just use regular dirt from my yard that is unneeded to level the ground before building a stone wall?
It is just sand, to the best of my knowledge, other than the price they apply to it. Otherwise it can be a coarse sand, fine sand, washed sand, even stone dust. The benefit of using sand is, it is easy to grade out and readily packs tight to reduce settling after the pavers are set. Which is another big plus.
I have done what you suggest with large pieces of flagstone, not brick. The disadvantage using dirt under brick is that it will settle differently in places, so eventually the brick may not be as flat as you first laid them. If you can live with that, them go for it. It is what was done back in the early days. When I was setting the flagstone, I wish I had sand to use, it would have been much easier, but I wanted to save a buck.
About using flagstone, large pieces, 18"X18" and larger, the rough ground under the flagstone is "smoothed out" by the stone spanning over the high spots. I did what I could to get even bearing under each stone. The downside on what I was using was the material was not uniform in thickness, much unlike brick. If you could scrape the ground evenly, you may have a chance to set them right on grade. As soon as roots or grass roots enter the equation, like my situation did, forget easy, uniform scraping.