Mid rally buzzer no replay

The coach asked for a challenge in mid rally. Today's regulations state that the rally will NOT be stopped, the challenge request will mark the action in the rally that is challenged, and at the end of the rally the challenged action will be reviewed. Once the challenge request arrived the scorer, the assistant scorer pushed the buzzer by mistake. The rally continued BUT blue jersey team players stopped playing for several seconds. This was not realized by the referees. The rally ended with loss of rally of the blue jersey team. The referees, once realizing the buzzer caused interference with the play, should have called for replay of the rally.