Quantum networks allow quantum processors to communicate over large distances. These networks often require simultaneously existing multiple entangled pairs of quantum bits (entangled links) as a fundamental resource for communication. Link generation is a sequential and probabilistic process, and successfully generated links are stored in a quantum memory. Links in memory are subject to noise that causes their quality to decay and become unusable. This paper uses reinforcement learning (RL) to investigate dynamic tuning of the entanglement generation protocol to minimise the time to generate multiple links. By comparing a fixed number of actions to a continuous action space, we analyse the importance of finer-grained tunings of the protocol. This is tested in simulated near-term and medium-term network abstractions. The results show that protocol tuning significantly reduces the mean time to generate entangled links, with finer tuning providing greater benefits up to a point. Furthermore, a heuristic is derived from the RL policies which matches and exceeds their performance. Future work can explore more advanced reinforcement learning algorithms to find better policies, as well as using different noise models to make more generally applicable policies.