If its a question of balance, as stated, then the difficulty scalability should be more than capable of offsetting that imbalance.. and we could simply play with the knowledge that things may be marginally 'easier' - that is, if the team doesn't want to implement some sort of scaling difficulty based on party size.