I worked for a place that sold limestone base, and am familiar with DOT specs for roads and parking lots. I have two concerns with your proposal:
4" of sub+base is not enough if the soil doesn't drain well; you will get potholes within a couple years,especially if you park about the same place each day. 6" of limestone should handle cars, 8" should handle heavy trucks. A thick base might seem like a lot of extra up-front cost, but a strong foundation will not only eliminate future maintenance, it will result in a flatter less puddle/ice prone surface.
Use limestone, 100%. No contest. Chert isn't even anything particular; it could be a wide variety of minerals with no composition guarantees. Certain minerals called chert, like jasper, fracture smoothly and result in a weak matrix. Other chert minerals like flint are quite brittle and suffer from compaction. Limestone on the other hand is mostly calcium, one of the strongest and stablest minerals. Oyster shells are perhaps the best base material, but limestone is also mostly calcium and MUCH cheaper.
If you use properly-compacted limestone, in most climates, you can just hotmix the top and it will last 20+ years. Asphalt's bad rep is because people skimp on base. If you want concrete for aesthetics and don't have trucks, you won't need 4 full inches of it if you have a good base (6"+), so you can recover some cost there.
bottom line: I've seen parking lots fail before the painted lines wore off because of bad and insufficient base, resulting in the whole thing being redone. Compared to that kind of cost, "splurging" on 6" of the best base available is a long-term bargain. If you do go with chert, make sure it's very well compacted and use at least 8" before surfacing. Since you have to use less limestone to achieve the same level of quality, it might be more competitive than unit costs alone suggest.