Stardew Valley – Does Saying No to Adopting a Child Affect Relationship Score?

It's really weird. short answer no, talking to villagers at festivals does not affect their affection (at least not during the egg festival I just checked. I'm guessing all the other festivals are similar)

Long answer, I made a backup of my save. I loaded up the game; I was on the day of the egg festival. I talked to Linus before the festivals. I went to the festival; I talked to as many people as I was able to. I won the festival. I left and I happened to find Pam and Sebastian to talk to. I saved and quit the game.

I looked at my save file and the changes were: Lewis, Sam, Wizard and Dwarf went up 8(?) points, Linues went up 30(?), Sebastian went up 20 and Pam went up 20. Everyone else went down 2 points.

This is of course as of March 17, 2016.

It is still possible to reach 10 hearts with all other Stardew Valley residents, which does not require giving them a bouquet. After marriage, your significant other will be able to have up to 13 of 12 hearts.

Oddly enough, you are still able to give bouquets to other marriageable candidates after marrying with no direct consequences that I've been able to observe.