Since this is a moving target question, let's address that target.
Personally, I'd suggest that you just assume it was asbestos. Asbestos is NOT extremely dangerous, as some seem to think. it isn't radioactive, or a deadly toxin, or even worth laying awake at night and worrying about. It's a material composed of very light and very sharp little fibers. It's about 10-20x Worse than fiberglass to breathe or handle.
There are various methods of testing for it, none of which require the presence of actual scientists, an exotic chemical lab or anything more sophisticated than a decent microscope or high powered magnifying glass and a rudimentary understanding of what the word fiber means. This method might give a high level of false positives, but that's OK.
Wear a face mask. Taking a very small sample of the material (pea sized), soak it in water for a few minutes and then crush it with pliers to expose fresh surface area.
Note: You have just rendered the material friable, which (according to certain java programmers who have never actually done asbestos abatement) means that you have put yourself, your dog and anyone within a 50 block radius in risk of imminent cancer and possible jail time. It's not true, but try not to breathe the stuff. If we were doing an STM asbestos composition test, we'd need a lot of time and extra equipment while we analyzed the actual composition.
Place the crushed sample on the slide and illuminate it with a strong light from the side so that light reflects into the tube. It may take some finesse. Really, you should use a polarization kit, but that would take two or three paragraphs to explain. Most "contractors that know asbestos" just look at the stuff with a magnifying glass, and they do OK. You probably can't tell mica from mohair, so err on the safe side by assuming that anything with small flakes or is fibrous might contain asbestos.
As the water evaporates, you will see the composition of the material in finer detail. Look for fibrous material that reflects the light slightly, as if it were made of small bits of fish scale. If the material appears to be very large - like a bit of thread, it probably isn't asbestos. However, if it appears to be made of very fine threads or flakes, it might be.
You can look at a crushed bit of drywall and a crushed bit of cement for comparison. Don't forget that you could pay $30-$100 locally to get this done, or even purchase a kit from amazon to send it into a lab. It's an excellent way to drag the project out for weeks and pay several hundred dollars to a licensed contractor for removal, which always makes professionals happy
Asbestos is not uranium. You have to work pretty hard to get it to kill you, and that means getting a substantial amount into your body. I won't address your body's immune system response, but I'll explain why asbestos can mess you up. It's pointy and sharp, and living cells hate that!
There are six different minerals (amosite, chrysotile, crocidolite, and fibrous varieties of tremolite, actinolite, and anthophyllite) that are usually called asbestos. It occurs naturally in the environment, and the odds are good that you have quite a bit of it in your yard right now. It's fibrous, and that means it disintegrates into little threads that you might not even be able to see.
Even if you get enough of it into your body, it can take years before bad things happen enough to be noticeable. Your body will defend against these particles pretty well by trying to encapsulate and expel them. Since the particles are very sharp and light, your body won't do the best job of cleaning you out, and thus a certain amount of the substance will stick around and interfere with your immune system.
The CDC doesn't know if it causes birth defects, as there is no proof of this, anecdotal or otherwise. There are two types of cancer caused by exposure to asbestos: lung cancer and mesothelioma. You can get these from breathing dust and fiber of any kind, not just from asbestos. Smoking will guarantee it.
Also, Asbestos must be rendered “friable” to be considered hazardous. Friable asbestos fibers are those which can potentially be released into the air. Busting up backer board could do this, but not much. Try to take it out in big pieces and keep it wet, wet, wet until it's in the bag. The risk is close to nil if you wear protective clothing and keep it wet.
We don't know how much you need to accumulate in your lungs to cause long term problems. It appears to be much more than a day or two of having the dust around, but all of the linkage studies involved people who had many months of continuous exposure.
There are three ways to start accumulating it: You can ingest it, you can get it into your epidermis or other membranes and worst of all, you can breath it and get it into your lungs. As long as you address all three of these routes of contact (and arrange for proper disposal of the removed waste) you have little to worry about.
I'll assume you're bright enough to realize that you should wear protective clothes made of closely woven materials and gloves, and that you'll put cloth booties on, and remember to cover up your face, ears, and neck.
Asbestos fibers do not evaporate into air or dissolve in water, and Asbestos fibers and particles may remain suspended in the air for a long time. this is why I add soap to my spray bottles - the soap solution will coat and cling to the fibers so that after the water evaporates, the fibers are heavier than before and will not float for as long.
In many cases, the trained professionals you want to rely on are $10 an hour off the street temp labor with 30 minutes of training, a $15 disposable plastic suit with hood (you can buy these at any pro paint store), a pair of booties and a cartridge type face mask. There is no secret to this process - just don't breathe it, get it on your eyes or skin, or actually swallow it. And they'll use blunt force (a term normally reserved for human injuries by the way) and chisel it off your wall, just like you would if you did it yourself.
All of the horror stories about asbestos are from people who had long term exposure - mechanics, workers, miners and people who lived in places where asbestos blows naturally in the air, or had been flaking from tiles or pipes for many years such as school kids and teachers.
This is important to know - because plaster dust (drywall, plaster walls), rock wool/insulation materials and concrete/stone dust is just as bad for you and your lungs as asbestos fibers are. For more than a hundred years people have done this kind of demolition (and suffered similar effects from inhaling similarly shaped fiber particulates) and yet no one bothers to call in the professionals for these things - even though you should be taking the exact same precautions.
Here's the money shot: Not a single case of cancer has been attributed to someone who pulled out two 4x8' pieces of cementious backer board from a wood stove installation over a couple of days. Not one. I suppose if you crumbled it up into gravel and left it laying around in a pile or scattered it throughout your house, you might have a problem.
If you were removing asbestos wraps from pipe or doing demolition of a sizable area, it might be a big deal. Pulling two small sheets of circa 1980s cement backer board down, (even if the manufacturer added a bit of asbestos to the mix) isn't going to be even close to risky - and certainly not compared to pulling wallboard and/or mortar down without a mask.
OSHA allows - actually states it won't be concerned about - up to 100,000 fibers per meter of air at a workplace before they get concerned. The EPA says that you can drink water with 7 million fibers per liter, but currently allows much more in residential sources.
The prudent course of action is to assume that it is asbestos containing material, unless you have access to a polarized light microscope and/or want to send it off to a lab to be tested. Even if it isn't, you should take the same precautions. This is a two hour job. Put on your protective gear, cover up the place, wet it and get rid of it. When you're done, wet wipe everything once again with soapy water, then with clean water. Then forget about it.
But seriously, wear a mask. And some gloves.