First of all, it's not an unlimited resource; the Druid can only wild shape twice per short rest, limiting the amount of poison the Rogue can attempt to harvest and by RAW:
DMG p.258
Serpent Venom (Injury). This poison must be harvested from a dead or incapacitated poisonous snake.
Unless the Druid can retain his form after dying, the Rogue won't be able to farm Druid snake venom. Furthermore, the DMG says that harvesting poison requires a check:
DMG p.258
Crafting and Harvesting Poison
The creature must be incapacitated or dead, and the harvesting requires 1d6 minutes followed by a DC 20 Intelligence (Nature) check.
It goes on about how to add proficiency to it and what happens on a failure but the DMG at least says that harvesting parts from creatures is not an easy task (DC 20 is hard, after all)
The point I'm trying to make is that you should allow your players to do clever things but limit the powergamey-ness to a minimum level that doesn't completely break your game. How you do this is up to you, I personally find that, at my table, at least, that doing the "yes, but..." approach to GMing maximizes the fun. "Yes, but you have to make a check to see if you can get enough venom for a single dose (this is important, you can get the vial half full but that won't cut it!). The druid can help you by giving you Advantage on the check, but it's not automatic." Being flexible like this allows your PCs to at least attempt the thing they really wanna do, and fosters a fun game.
As for other possible attempts to exploit "infinite" things, allow and disallow at your own discretion. A good tip for this is to think about how overpowered it would be to allow it.
Take your infinite arrow feathers example, by pulling the feathers off the wild shaped druid (ouch!), he is able to save a whopping 1 GP (What a bargain!). Consequences may or may not exist, depending on the kind of game you're playing but I would personally rule that once the druid reverts, he finds that he lost some of his hair.
Rulings, Not Rules
Jeremy Crawford, the lead game designer, called the rules "intentionally silent on these corner cases":
Wild Shape can introduce wild situations. What happens when someone swallows a druid in a Tiny form? Is a druid fecund in beast form? The rules are intentionally silent on these corner cases, leaving adjudication to DMs. As always, I say go with what's best for your story.
The particular ruling (the druid can shapeshift into a specific animal) can lead to a very satisfactory in-game situation. Or it can be utterly boring and devastating, depending on the plot. It is the DM's job to make the right decision.
Aside from the combat, there will always be huge difference between classes' features, so you can't compare (or "balance") them. Some features will be much more useful than another ones in certain situations.
Following the rules-as-written as strict as possible won't help here. In the end of the day, making the game fun an engaging is not about the rules. For instance, if your game is all about picking locks and disabling traps, and you have only one rogue in the party, (s)he inevitably steals the spotlight. You, the DM, have to balance these things by your self — how exactly do class features work in order to not to spoil the fun.
You are the DM and you want to decide, how does magic (the Wild Shape, in particular) work in your world. For this job, what things should you consider in the first place — your own story, the fact if your players have fun, common sense, or nitpicking these minor semantic details in the rules (which are concise and not detailed enough)?
The Adventurer's League Guide describes the role of the DM the similar way:
As the Dungeon Master, the most important aspect of your role is facilitating the enjoyment of the game for the players. You help guide the narrative and bring the words on the pages of the adventure to life. The outcome of a fun game session often creates stories that live well beyond the play experience at the table. Always follow this golden rule when you DM for a group: Make decisions and adjudications that enhance the fun of the adventure when possible.
Wild Shape description is open-ended
PHB gives only basic restrictons of the Wild Shape:
Starting at 2nd level, you can use your action to magically assume the shape of a beast that you have seen before.
Your druid level determines the beasts you can transform into, as shown in the Beast Shapes table.
The only limitations it describes are the creature type, its CR and its flying/swimming speed — pure balancing ones, a sound base for DMs to build their own adventures. As a DM, you are free to apply all the necessary restrictions — the creature size, type, features or appearance. It would be reasonable to discuss this with the player beforehand, say, prepare a list of their wild shapes.
You don't have to though — if you are happy with the default restrictions, just say that all other things are allowed (dinosaurs included), unless it spoils the fun.
Best Answer
Maybe.
When wild shaped, "any action that requires hands is limited to the capabilities of your beast form" and you still have access to any features granted from your class, race, feats, and so on. (PHB, pg 67)
Pact of the Blade says "you can use your action to create a pact weapon in your empty hand." You are also proficient with it while you wield it. (PHB, pg 107)
That being said, if you wild shape into a beast that has hands, such as a gorilla, then you can use pact of the blade to create a pact weapon, and you will be proficient with it.
However, if you are an elk, you do not have hands, thus you cannot use the pact of the blade class feature.
If you wish to take the course of action described in your question, such as allowing an elk to gain extra "spikes" and deal extra damage with his natural attacks, you are stepping over the threshold into homebrew territory.