Should a bathroom fan vent to a separate duct system

If there is no water source explaining the moisture in the closet, and your basement has a vapor barrier, it is possible that the moisture source is airborne.

The closet may be sufficiently cold and the air in the rest of your house may be sufficiently humid that psychrometrics causes the relative humidity in the closet to be high enough to grow mold.

if the cause is psychrometric...

Installing a continuous fan that exhausts outdoors would help only insofar as the increased airflow is able to maintain a higher closet air temperature than before. If the makeup air flowing into the closet is warm and dry enough, you might solve the problem.

Perhaps a better solution is to insulate the closet to bring it closer to the temperature in the rest of your home.

You probably should get some wet and dry bulb thermometers and learn how to read a psychrometric chart, then run some basic experiments. Once you wrap your mind around the psychrometrics at work in your case it should be obvious whether they are the cause, and if so, what you could do about it.

We have dry cold winters as well, -35C, but not arctic. We exhaust bath moisture outside, but our fuel costs are pretty cheap. It would be worth estimating just how much extra heating costs are incurred by exhausting outside, it may not be that bad. Or a good argument why not to go outside. Excessive fan use can be limited by installing timer or occupancy switches.

I suspect the air exchanger doesn't work well because it ices up. It's worth investigating the cause though, in case it is just a clogged filter. A direct exhaust ices up outside around the outlet instead of inside the exchanger. The ice build up can be dramatic after many weeks below freezing and no sun. Another reason against outside exhaust besides being inefficient.

I don't think exhausting from one small space to another is a good idea. If your idea is to work, it would need to exhaust into the largest space. Perhaps above a heat source. I think some will find this objectionable, it's a bit unsanitary sounding. Sort of like using your roommate's damp bath towel. Nothing really wrong with it, yet quite unappealing. Especially if someone used the fan to remove odors instead of moisture! Eww. It's also possible that despite the dryness, heavy moisture load may just be too much to be dispersed within the dwelling.

You should focus on a workable air exchanger solution first. It is the only solution that is both energy efficient and fully sanitary. A direct exhaust may turn out to not be overly expensive, but it's still inefficient and should be avoided. Moving the moisture around inside may work, but should be the last resort, it just sounds a bit unsanitary, and it may not even work in all cases.