We've run tournaments with five Masters. Not a problem. Delays are inevitable, which is why Speed Zendo is so appealing. Give the table a total of 5 minutes per level of rule difficulty, which is plenty, and if none of them can get it, they all lose.
-Jacob