I've been playing Skyrim Special Edition on Steam for a while now and here. Lately I've noticed my move speed slowing way down while walking.
There seem to be two stages to it (three if you count it not happening at all): at first it's hardly noticeable and I walk at maybe 80-90% speed, then, after playing for a seemingly random amount of time (ranging from a few minutes to hours), it drops without warning to like 10% of the regular speed.
Once a stage has been reached it is continuous and can't be reverted to the original speed.

I move at normal speed in the air while jumping. I'm not encumbered and can fast travel, but this is because I greatly increased my carry weight via console command at the start of the game (using player.modav carryweight 9999 to get it to persist), so I've got around 20,000 carry capacity.

I've tried a number of fixes which didn't solve my issue:

• Saving and reloading
• Saving and quitting and loading
• I tried the recommended console command from a steam thread player.setav speedmult 100
• I tried testing with tgm and tcl on, both separately and together
• I tried the fixes of sprinting and talking to someone, sprinting and sitting in a chair
• I've dumped enough stuff in chests and whatnot to be at around 4,000 of 20,000 carry weight, and it still didn't help.
• I've also tried emptying my inventory of everything but quest items (0 carry weight) and it still persisted.

I'm at my wit's end trying to figure this out.

Mod list:

- 100 percent max pickpocket chance
- AchievementsModsEnablerSSE
- Alchemist Compendium
- Andromeda - unique standing stones of Skyrim
- Angelic Halos and Demonic Horns
- Ars Metallica
- Ash Pile Expiration SE Installer
- Auto hide ammo
- Barenziah quest makers
- BetterConsoleV20
- Breezehome - Eli's Breezehome Overhaul
- Craftable unarmored and armored vampire gear
- Dark Brotherhood Reborn - Dawnstar Sanctuary
- Dark Envoy 
- Death Alternative - Your money or your life
- Disenchantable Dark Brotherhood gear
- DisplayEnemyLevelSE
- Dragon Wall Wisdom - Readable Dragon Walls
- Eli_Breezehome - Ars Metallica Patch
- Enchant and smithing potion duration
- Fangs and eyes - Red Combat
- Fast travel from interiors
- Fortify destruction damage
- Fortify enchanting
- Fully improvable Nightingale armor
- GIST Soul Trap
- Hearthfire Multiple adoptions and custom home support SE
- Hearthfire mod that makes my kids' last names Dovahkiir (can't find the exact name)
- Hide-SkyUI
- Holidays
- Hot springs of Skyrim
- Hunterborn
- Immersive Children SE - FOMOD installer
- Immersive citizens - AI overhaul
- Immersive speechcraft SE
- Jewelry limiter - another multiple rings and amulets mod
- Leveled vampire drain - legendary edition
- Marry me Serana
- Mirai the Girl with the Dragon Heart
- Modern Brawl Bug Fix
- Neko race - all in one installer
- No Camera Collision
- RaceCompatibility All-in-One Scripted Installer
- RaceMenu
- Raeza - Adoptable Plugin
- Raeza SE (main file)
- Readable Shadowmarks
- Resurrect
- Scaling Bound Weapons
- Sig. Equip.
- SkyUI
- SKSE64 (Skyrim Script Extender for Special Edition)
- Spell Crafting for Skyrim
- The Paarthurnax dilemma
- The Shrouded Lair
- The Shrouded Lair Expansion
- Thieves Guild Bounty Clearer
- Truly Light Glass Armor
- Truly Light Glass Armor no pauldrons (both the standalone version)
- UnenchantedItems
- Unlimited Training
- UNP Female Body Renewal
- Unread books glow SSE
- Vampire Lord no fall damage
- Vampire Lords can activate
- Vampires regen
- Viewable faction ranks
- Volkihar Knight SE
- White River Cottage SE
- Wintersun

Update:

It looks like the issue might have to do with my controller as I had an identical issue in another game as well (Killing Floor 2). Will test with a new controller soon and update with the results.