The purpose of an entanglement witness experiment is to certify the creation of an entangled state from a finite number of trials. The statistical confidence of such an experiment is typically expressed as the number of observed standard deviations of witness violations. This method implicitly assumes that the noise is well-behaved so that the central limit theorem applies. In this work, we propose two methods to analyze witness experiments where the states can be subject to arbitrarily correlated noise. Our first method is a rejection experiment, in which we certify the creation of entanglement by rejecting the hypothesis that the experiment can only produce separable states. We quantify the statistical confidence by a p-value, which can be interpreted as the likelihood that the observed data is consistent with the hypothesis that only separable states can be produced. Hence a small p-value implies large confidence in the witnessed entanglement. The method applies to general witness experiments and can also be used to witness genuine multipartite entanglement. Our second method is an estimation experiment, in which we estimate and construct confidence intervals for the average witness value. This confidence interval is statistically rigorous in the presence of correlated noise. The method applies to general estimation problems, including fidelity estimation. To account for systematic measurement and random setting generation errors, our model takes into account device imperfections and we show how this affects both methods of statistical analysis. Finally, we illustrate the use of our methods with detailed examples based on a simulation of NV centers.