It was periodic re-randomization. Without periodic re-randomization even if students submit attempts, I can still increase their max attempts globally.
But if periodic re-randomization is enabled then the global max attempt does not increase for those students who just even looked at the question. And for those even turning off periodic re-randomization does not help, since they have individual max attempts marked in the database. Only individual changes work for them.
Got it. Thanks.