This paper aims to define the broad concept of fairness and investigate how it can be measured, especially considering fairness in automated negotiations. The report relies on the work on fairness issues that have been derived from the research of C. Albin [1]. Firstly, the paper elaborates on different fairness metrics from the literature review. Then these metrics are tested to assess if they capture the effect of considering fairness by the agents in the negotiations. That is done by simulating multiple bilateral negotiations under the open-source GeniusWeb framework, where eccentric agents are compared with fairness-oriented parties by using the fairness metrics. Based on the conducted experiment, the most consistent fairness metric in automated negotiations is the distance to the game-theoretic solutions to the bargaining problem, which considers much of the outcome fairness concept. However, other investigated metrics also capture the different scope of fairness and can be used as metrics, especially when combined, and some interdependence between metrics is modelled.