How to minimize ceiling “backwash” of air circulation in an attic HVAC system

Basically you are trying to balance your new ducts with respect to your old ducts by sizing the new ducts. If you make a mistake or if your relative cooling loads (office vs rest of the house) ever changes, your only recourse would be to replace the ducts.

My preference is to size the ducts as large as reasonable possible (without increasing cost too much) and use a damper to tune the airflow ratios. Since you have 12" duct and a place to attach them, just get a damper like this:

http://www.grainger.com/Grainger/DAYTON-Round-Balancing-Damper-2TFY1?Pid=search

An uninsulated unshaded west wall in Texas can probably generate a very high cooling load in the afternoon.

Given that this question is 3 years old you probably moved on... But I can make a couple of non-expert suggestions...

The first thing I would look at is shade. Can you add some foil into the windows to cut down the heat in the morning? Or some shade cloth on the outside? I have large sliding doors facing west and in the summer I span some 50% black shade cloth from the eave to the ground about 6' out. One can see through quite fine and it cuts down the heat load tremendously (even with the low-e windows I have). This saves a lot of money too...

WRT your specific question, here is what I would do. I you want to go to a ceiling register, check out https://www.hvacquick.com/products/residential/Grilles-Registers/Curved-Blade-Grilles/TRUaire-A303-Series-Triple-Deflection-Curved-Blade-Grilles Click on the links tab and then on the performance data link. You get a table that shows grille size on the left and then performance data across the row keyed by CFM. Take a 10x10 grille, assume you have 200CFM, that means the air "jet" will reach 12ft horizontally ("throw"), and the noise is "NC 30", which is low (there's no lower entry in that table, but other types of grilles/manufacturers go down to NC 20).

Now the other two interesting measurements are at the top of the table, for that register @200CFM you have a "back pressure" of 0.022 "WC and 600fpm face velocity. You probably have something like https://www.hvacquick.com/products/residential/Grilles-Registers/Grilles/TRUaire-210-Series-Steel-Single-Deflection-Adjustable-Bar-Grilles now, look at the performance info for the 10x6 you have. At 200CFM you have 15' throw (interpolating a bit), 0.026 "WC pressure loss, and 650fpm face velocity. So you would be reducing the pressure by a rather small amount, so you wouldn't get much more air.

Now, if you have an an anemometer to measure wind you could work your way backwards by measuring face velocity. I got a https://www.amazon.com/dp/B01L9SC36A but cheaper will work too. So measure the fpm you have now by holding the anemometer in front of the center of the grille, use the table for the grille you have now to get CFM, then use that to look up the hypothetical new grille to see throw distance and noise level. Note that for ceiling you want curved blades so the air mixes at ceiling level and down't blow straight down on you. You could also go with a round ceiling grille...

If the new grille has lower pressure loss, you will get more CFM, stealing from other outlets. WRT duct, 8" flex doing 200 CFM has a friction loss of 0.024 "WC (http://www.hartandcooley.com/tools/friction-loss-calculator-for-flexible-ducts). Of course I don't know how long your duct is, but it doesn't look like it's going to be a limiting factor, but it may well be in the same range as the grille.

OK, I'm not an expert and an expert would tell you you have to recalculate and rebalance the whole system from scratch if you make a change. And the expert would be right. But you asked how to "wing it, and it doesn't have to be perfect" and I hope that the above gives you a way to make some back-of-the-envelope calculations so you end up in the right ballpark.