How to kill Mom

You can unlock Judas by first beating Mom, which will unlock two new dungeon levels called "The Womb". Then you have to beat The Womb, by killing the boss Mom's heart.

TL;DR: Yes, try every pill you find

In the game, according to the list near the bottom of the items page on the wiki, there are

8 pills with definitely good effects:

1. Bad Gas: poisons surrounding enemies
2. Balls of Steel: gives 2 soul hearts
3. Full Health
4. Health Up
5. Pretty Fly: adds a close orbiting fly
6. Range Up
7. Speed Up
8. Tears Up

4 pills with definitely bad effects:

1. Health Down
2. Range Down
3. Speed Down
4. Tears Down

And 5 pills with neutral effect or possibly good or bad effect:

1. Bad Trip: Take 1 damage if more than 1 heart left, otherwise grants full health
2. Bombs Are Key: Swaps bomb and key counts
3. Explosive Diarrhea!: Drops lit bombs while walking for a short time
4. I Found Pills: no important effect
5. Telepills: Teleport to a random room

If we assume that the set of pills for each playthrough is uniformly chosen from all of the pills, then it would be wise to take every unidentified pill because it is more likely to be good than bad, and even some of the neutral pills can be useful in the right circumstances.

If we assume a reasonable bad distribution, with 2 bad pills, 2 neutral pills, and 2 good pills (generally worse than random but possibly what actually happens), it may still be wise to take random pills. Some of the neutral pills have useful effects at times, so it would be best to try random pills with as many of the following conditions as possible

• You have 1 health (so Bad Trip heals you)
• You want to switch bombs and keys, or at least wouldn't care if they switch (Bombs are Key)
• You want to drop bombs in the room, possibly to break rocks or look for the secret room (Explosive Diarrhea!)
• You would be OK teleporting into any room in the level (Telepills)

So the absolute best time to test pills would be when you have defeated the boss and cleared the level and you have 1 heart and you still need to break a rock somewhere.

However, even discounting the possible benefit of neutral pills, I would say it is still good to test every pill because knowing what pill you have and what pills there are can be useful. If you know that you have an Explosive Diarrhea! pill for example, you can use it against a boss or a room full of enemies that are easier to kill with bombs.

Also, if you know what bad pills there are, you can more accurately determine whether to buy the PhD or Lucky Foot, because they transform bad pills into the corresponding good ones. So if the bad pills are Range Down and Speed Down, and you already have both of those at max, it is probably not worth it to buy the PhD because you will not get any new benefit from those previously bad pills.

Thus I conclude that unless we know that we can get very bad sets of pills, it is probably good to try unidentified pills.