Skyrim does NOT like custom keyboard bindings. Especially if you changed M1.
To fix this, go into the controls and reset them to defaults, by pressing T.
How to fix (2013 update)
I've tried the bugged saved game file I initially posted in my question with the current latest versions of the official patch, Unofficial Skyrim Patch (USKP) and Skyrim Script Extender (SKSE), and the saved game is now Aspect of Peryite-free.
I've configured SKSE to use its ClearInvalidRegistrations console command on every saved game load, which removes invalid scripts left running by uninstalled mods. This feature was introduced in v1.6.7 of SKSE:
add console command ClearInvalidRegistrations to remove invalid
OnUpdate() registrations
This prevents orphaned OnUpdate() events and
the resulting bloated/broken saves when removing certain mods. When
applied to an already bloated save, it will stop growing further and
instead shrink over time as the game processes all queued events. This
may take hours depending on the amount of bloat.
To execute automatically after each reload, add this to \Data\SKSE\skse.ini:
[General]
ClearInvalidRegistrations=1
According* to a developer of the Unofficial Skyrim Patch (USKP), the underlying cause is because DA13 ('The Only Cure' quest) is never stopped.
A possible fix* for this:
In the meantime, load an affected save, then type stopquest DA13 into the console. That ought to put an end to that. Plus it’ll free up some unnecessarily leftover alias memory.
BTW, don’t use the console command on a save not bothered by it. Even though it may appear benign, console commands can have negative consequences you don’t know about.
If this doesn't work or if you experience issues like: NPCs not responding properly to conversation (not reply to you when you choose a certain conversation path - leaving you stuck), merchants not opening the buy/sell window, horses that can't be ridden, and other interaction issues, then the save game is most likely corrupted. Try the SKSE ClearInvalidRegistrations console command fix I've posted above, instead.
To prevent this from happening in the PC, get the latest version of the Unofficial Skyrim Patch. This has been fixed since version 1.1 of the USKP:
"The Only Cure" (DA13) was never stopped which would lead to the Aspect of Peryite randomly initiating conversation and being unable to exit.
* This info is from the old USKP bug tracker page at 16bugs.com, which is now not viewable. The new USKP bug tracker report page of this bug is at afkmods.iguanadons.net.
Best Answer
You could just map the movement to a key that you can toggle e.g. Caps lock, Num lock, Scroll lock, etc. That way you don't have to weigh anything down. And as far as I'm aware, there are no console commands in Skyrim that are able to produce this effect. In case there are any, you can try looking here.