Electricity balancing is one of the main demanders of short-term flexibility. To improve its integration, the recent regulation of the European Union introduces a common standalone balancing energy market. It allows actors that have not participated or not been awarded in the preceding balancing capacity market to participate as voluntary bidders or ‘second-chance’ bidders. We investigate the effect of these changes on balancing market efficiency and on strategic behavior in particular, using a combination of agent-based modelling and reinforcement learning. This paper is the first to model agents' interdependent bidding strategies in the balancing capacity and energy markets with the help of two collaborative reinforcement learning algorithms. Results reveal considerable efficiency gains in the balancing energy market from the introduction of voluntary bids even in highly concentrated markets while offering a new value stream to providers of short-term flexibility. ‘Second-chance’ bidders further drive competition, reducing balancing energy costs. However, we warn that this design change is likely to shift some of the activation costs to the balancing capacity market where agents are prompted to bid more strategically in the view of lower profits from balancing energy. As it is unlikely that the balancing capacity market can be removed altogether, we recommend integrating European balancing capacity markets on par with balancing energy markets and easing prequalification requirements to ensure sufficient competition.