Duct Design Question

Hi! I have two bathrooms that are next to each other, and I'm trying to install a bathroom ventilation fan in each. One bathroom needs a 150 CFM fan (which has a 6" duct), and the other a 50 CFM (4" duct). I'd like to have to make only one opening in the roof for the cap, so I'd like to connect the exhaust of the two fans with a y-connector in the attic, and then go through the roof opening. Is it better to use a 6" roof cap, and a 6"-4"-6" y-connector (6" and 4" into 6")? Or am I better off searching for a 150 CFM fan that also has a 4" duct, so I can connect them with a 4"-4"-4" y-connector and vent it out through a 4" roof opening? Thank you!