Here is the issue. I can use the toggle bolts, but it looks like drywall has lost strength. It can be seen in the pictures that drywall around the place of the bracket holes is swelled and about to come off.
As you have found out, curtain rod supports do not do well hanging from plaster anchors. They really should be attached to studs, but where that is impossible, toggle anchors give you a chance at success. I prefer the solid bar type that pivots after insertion.
The drywall in the immediate vicinity is beyond saving. Even toggles will not have much solid to support them. You need to move the bracket well away from the damaged area (to a stud if you can), or you need to build up the area to support it.
One way to reinforce the area is to create a wooden backing behind the drywall to serve as a support. Find a piece of wood at least 1 1/2 inches wider than the space between the screws. Cut it to about 6 inches in length.
Cut away the damaged section of drywall making a hole a bit wider than the board and about 3 inches high. You are going to insert the board vertically into this hole and attach it to the back of the drywall.
An easy way to do this is to drill a small hole in the center of the board and tie a string through the hole. You can then insert the board and use the string to prevent it from falling and to manipulate it into place.
Screw through the drywall into the board using drywall screws. Use two screws top and two on the bottom. This locks your new framing piece in.
Make a patch of drywall and fill the hole. Tape and patch with joint compound. Lightly sand and prime. Paint now or later.
Now, when you reattach the bracket, you will be screwing into the wood which is supported by many square inches of drywall. I would predrill the holes with a bit slightly smaller than the screw body to reduce the chance of splitting the wooden backer.
Forget the drywall. Go to a lumber supply and buy a pretty looking 1x4 or 1x6 board of generous length considering the width of the window or whatever. It is best if it is long enough to cover at least two studs, so for narrow windows, it might extended well beyond the window (consider making it full room length).
Route the board's edges (optional), sand (optional), stain (optional), and finish it (for sure) to be water resistant (Tung oil, varathane, etc.).
Take down the rest of the bracketing. Find the studs: drill small (1/16th inch) holes every one inch to find the first stud, then confirm stud spacing in multiples of 16 inch spacing. Screw or nail the board to the studs. Attach brackets to board using standard hardware.
You will be mounting your draperies on an "Escutcheon."
First, layout your drapery rod kit parts & check to see everything's there--that includes the instructions, too.
The screws provided are likely of plated zinc, a soft metal, which may be suitable for light curtains, however not so for draperies, especially those of some weight or those that will be drawn open and closed and their length may not be long enough to to go through just short of the escutcheon's thickness. So, you will need to substitute the same quantity and size in stainless steel wood screws of the length that the escutcheon requires.
You will also need 3 stainless steel wood mounting screws that are this long: the thickness of your escutcheon board + 1/2 inch for drywall + 2-1/2 inches into the wall stud. So, your shopping list should include 3 stainless steel screws 3-1/2 inches to 4 inches long. The size used for securely mounting exterior door to jamb (4 inches) works fine
Because wall framing doesn't take into account supporting of draperies, the goal here is to center your escutcheon over the window, by fastening both escutcheon ends to the furthermost studs possible, and also through a middle stud for added stability, but the middle stud may not actually fall at the centermost point of the escutcheon.
If you find routing to be fun, do it. But it's the long way: short way is, get the 3" wide board cut to desired length (the "escutcheon") ; mount to wall at 3 stud locations using the 3-1/2 to 4 inch screws (3 of them); then miter-cut trim moulding and mount trim edge against edges of the escutcheon (like is done for 2-piece colonial baseboard trim technique) using a pneumatic brad-nailer. Then paint the trimmed escutcheon to match your wall.
Finish the job by marking the escutcheon for placement of the drapery rod brackets, pre-drill holes; mount brackets using the screws that came with brackets ( or substitute stronger stainless steel screws if draperies are heavy) and install rod with draperies.
Or, if this is to be a 1-person job: prime & paint pieces first; when fully dry, lay out escutcheon flat on floor directly under window, measure on wall where mounting studs will be & transfer their positions onto escutcheon. Check layout and measurements carefully to ensure accuracy. When satisfied with accuracy of markings, using drill bit smaller than mounting screws, pre-drill escutcheon's stud mounting holes (3 of them ) through escutcheon and IF YOU ARE ABSOLUTELY SURE YOUR MEASUREMENTS ARE SPOT-ON, also through the located wall studs at the location that aligns with the escutcheon mounting holes. IF YOU'RE NOT SURE, THEN WAIT TO DRILL THROUGH THE DRYWALL UNTIL YOU PLACE THE ESCUTCHEON UP INTO PROPER POSITION AGAINST THE WALL & LEVEL IS CHECKED.
Mount the brackets while escutcheon is flat on floor, position brackets on it & mark holes, measure again to be sure of alignment & pre-drill with bit smaller than mounting screw and then mount brackets on it. Now raise & attach escutcheon into position on wall. Since this is being done without a helper, raise one end up into position on wall , check with level and mount escutcheon to the first end stud (either end will do) & using a dead-man prop to hold it into position, work your way across the length of the escutcheon attaching it to the other 2 identified studs. Hang rod & draperies.
-Wow, you're done! Good job! Now, clean up your mess.
Easiest way to handle this.
Buy decorative plinth - whatever size you want.
Glue it over the area, centered on where you want your rods.
The tension from the rods should make sure there is little to no pressure put on the glue on the plinth (you can use silicone or a variety of construction adhesives).
I used something very similar to this - https://www.craftmill.co.uk/casting-moulding-products/chromatic-moulding-alginate/square-plaque-plinth-wavy - on my last install. I put these up right away if there is no framing to attach to. I don't even bother with anchors as they will fail. If this fails, you just glue the plinth back on.