I'm not sure if this is what you are after, but the simplest way to stop the math adjustments from showing up on the cards for your attack powers etc. is to simply equip the non-magical equivalent of the item, making sure to uncheck the boxes on the Equipment page in the 'Show on Power Cards' column for the magical versions.

That is, if they are to have a Magic Dagger of Whateverness, give them that magical dagger but equip the non-magical 'Dagger' item and ensure that only the 'Dagger' has a tick in the checkbox for 'Show on Power Cards'.

EDIT: The 'Show on Power Cards' option is found in the sixth column of the Inventory tab, which is the default first tab of the Equipment section. It may not show up unless you scroll a bit sideways, and only equippable items have checkboxes.

The items cards will still show their bonuses, but only on the actual items cards themselves - the power cards will show only the inherent bonuses. All special powers granted by the magical version will show up on the item cards themselves. The one exception are those items that grant class-related powers, such as the Tome of Summoning granting extra spells. The extra spells will not show up unless you equip the item, which forces the adjustments to show up on the power cards again. This can be worked around by using the 'House-rule' function on the 'Powers' page to simply manually grant the addtional powers separately from the items.

By RAW, no.

The Use Magic Device feature lists the specific attunement requirements that it circumvents, and, as the question states, "spellcaster" isn't one of them. There are other attunement requirements that Use Magic Device doesn't get around — for example, alignment restrictions exist for a Sword of Answering or a Talisman of Pure Good (or Evil).

If the intent was for Use Magic Device to get around all attunement restrictions, then it would say so; we can infer that the intent is that some such restrictions cannot be avoided.

It's possible that the intent of the designers was to allow rogues with Use Magic Device to be able to use the (mostly) wands with the 'spellcaster' restriction, but so far the wording of the rules doesn't support that.

But probably by RAI, yes.

In this Twitter post from October 2016, Jeremy Crawford was asked about how a Wand of Paralysis is used by a rogue, and he doesn't say that it can't be done. But the applicability of Use Magic Device to the spellcaster attunement requirement wasn't the subject of the question, either.

However, this doesn't seem to be a critical game balance issue, and given that Use Magic Device is intended to allow rogues to use spell scrolls, it doesn't seem unreasonable to extend it to the "spellcaster" items as well.