Do you have access to "Xanathar's Guide To Everything"? If so, you should also consider the College of Swords, which is another weapon-based bard college option. It offers the ability to use a melee weapon you are proficient in as a bardic spell casting focus, which works well for bards who want to use melee weapons in one or both of their hands.
However, keep in mind it does not offer the same "Battle Magic" feature as the College of Valor does, which means you will likely be casting and slashing on different turns.
Pure Vengeance at 13th level
Stats:
Str 20 [17 base + 1 racial + 2 ASI @ 8th level]
Dex 10 base
Con 18 [16 base + 2 ASI @ 12th level]
Int 9 base
Wis 16 base
Cha 20 [18 base + 2 racial]
Feats:
Great Weapon Master @ 4th-level ASI
Fighting Style:
Great Weapon Fighting
There's not many choices to make with this path, if you're set with Vengeance Paladin. Pick up Great Weapon Master as early as possible and use it only on foes when you have a 85% chance to hit, (at 13th-level, that's 14 AC). Other than that, the -5 to attack hurts you too much as you'll do a lot more damage with smite than with the +10 damage of GWM. This is backed up here, see Markovchain's answer:
Larger base damage = Smaller benefit
This is not surprising. If you can dish out 30 damage without GWM/SS, and you use GWM/SS, then you are also risking dealing no damage on a miss by taking a penalty.
The higher your base damage, the more you have to lose on a miss. Whereas, the additional 10 damage is always constant, so it is more significant if your base damage is small.
In the case of you, being a paladin, and able to dish out 49 damage on a hit with Divine Smite, GWM is not that important to you but you should still pick it up for the potential Bonus Action attack on a crit or kill.
If you're satisfied with the above, here's a little bonus:
10 Eldritch Knight Fighter / 3 Devotion Paladin
A.k.a The Smite Knight
It's one of the highest burst damage I've ever seen played, and have had problems challenging as a DM. If you say everything is "flexible" and if you don't mind deviating from your original Oath and Class, this build is pretty solid.
It focuses on smiting each time you hit, and using Action Surge to double your first turn (It gets even more ridiculous at 20th-level, as you can then Action Surge twice). You can also keep on using GWM when you Channel Sacred Weapon.
The con about this build is that it requires too many ridiculous stats to pull off. But don't worry about that, you have the ridiculous stats needed to pull it off. Happy Smiting!
Best Answer
Ability scores
You will need a good attack stat, good Cha, and everyone needs a decent Con, especially casters in the front line.
For multiclassing Str 13 is mandatory, after all this you can't afford a good Dex beside it, so you should be Str primary.
This also means you need Heavy Armor, so first level has to be Paladin.
Feats
Shield Master provides arguably the best at-will control, and having a good Str helps with that too.
Later you will get Expertise in Athletics, and become practically unbeatable.
With a shield in hand you can cast Paladin spells that require Somatic and Material components, but not Bard spells. Warcaster solves this, and helps with concentration as well.
Class options
For good DPR you should get Extra Attack as soon as possible. A Valor bard could provide it, but than the Martial Weapon and Shield proficiency is wasted from the Paladin, so I suggest Lore.
Vow of Enmity is so good I usually recommend Oath of Vengeance, but you get advantage from spells Master anyway. So I would go with Ancients, as I find the spell list better, and Aura of Warding is awesome. However, Conquest, Devotion and Crown is a valid choice too.
Fighting Style
Dueling for DPR, or Defense for survivability.
Race
Half-elf, at level 4 has the same ability scores and feats as a variant human, plus Darkvision and a skill.
Skills
Athletics is the center of this build, Perception is needed by everyone, and Stealth can offset the disadvantage of heavy armor.
Other than that, whatever you like.
Full build
Half-elf
Str: 15+1
Dex: 10
Con: 13+1
Int: 8
Wis: 12
Cha: 14+2