You can join just one or both factions depending on what titles, cosmetics, housing furniture, and mounts you want.
If you decide to join both factions, you will gain +900 reputation for quests of that faction and then lose -500 reputation for quests of the opposing faction. Therefore, your net gain in general is +400 reputation if you do an equal amount of quests for both factions.
Both the [Badge of Taste] and the [Badge of Dishonour] let you barter 2 badges for 3 Festival Tokens at the Inn League and Ale Association Headquarters..
The Inn League drink deliveries are much easier because you can buy all the drinks ahead of time and keep them in your inventory bags. For the Ale Association, you will have to go to every pickup and every delivery location every time you do the quest.
However, the Ale Association’s daily challenge is easier because you only have to run around the town of Bree instead of running around the entire Shire for the Inn League daily trial.