Nitsche's method is a popular approach to implement Dirichlet-type boundary conditions in situations where a strong imposition is either inconvenient or simply not feasible. The method is widely applied in the context of unfitted finite element methods. Of the classical (symmetric) Nitsche's method it is well-known that the stabilization parameter in the method has to be chosen sufficiently large to obtain unique solvability of discrete systems. In this short note we discuss an often used strategy to set the stabilization parameter and describe a possible problem that can arise from this. We show that in specific situations error bounds can deteriorate and give examples of computations where Nitsche's method yields large and even diverging discretization errors.