To date, PMN J2134-0419 (at a redshift z = 4.33) is the second most distant quasar known with a milliarcsecond-scale morphology permitting direct estimates of the jet proper motion. Based on two-epoch observations, we constrained its radio jet proper motion using the very long baseline interferometry (VLBI) technique. The observations were conducted with the European VLBI Network (EVN) at 5 GHz on 1999 November 26 and 2015 October 6. We imaged the central 10-pc scale radio jet emission and modelled its brightness distribution. By identifying a jet component at both epochs separated by 15.86 yr, a proper motion of μ = 0.035 ± 0.023 mas yr^-1 is found. It corresponds to an apparent superluminal speed of β_a = 4.1 ± 2.7 c. Relativistic beaming at both epochs suggests that the jet viewing angle with respect to the line of sight is smaller than 20°, with a minimum bulk Lorentz factor Γ = 4.3. The small value of the proper motion is in good agreement with the expectations from the cosmological interpretation of the redshift and the current cosmological model. Additionally we analysed archival Very Large Array observations of J2143-0419 and found indication of a bent jet extending to ~30 kpc.