Learning new concepts is a difficult task for autonomous robots. These robots can adapt to changes in the situations. To adapt to a situation, they should be able to determine the usefulness of objects around them. The usefulness of objects is highly dependent on situational context, making pre-programming of adaptation behaviour to all possible situations difficult. Automatic learning of this information by the robot from its own observations during deployment is a more feasible approach. We encode the usefulness of objects in logical statements that symbolise regularities in observations of objects in a Markov Logic Network. This network forms the basis from which the usefulness of newly observed objects or situations can be inferred by a robot. Each logical statement in the network has an associated weight that indicates how strong the robot believes the statement is true. These weights can be adjusted over time based on new observations made by the robot, strengthening its beliefs or creating a new belief for an unknown object in a process called cumulative learning. We extend the Markov Logic Network framework with a cumulative learning algorithm called MLN-CLA. This algorithm contains several different knowledge updating strategies that handle conflicts. We demonstrate the ability of MLN-CLA to learn the usefulness of unknown objects over time. In addition, we demonstrate the abilities of MLN-CLA to incorporate new logical statements to better capture information about objects and situational contexts. Together, these two abilities enable robots to autonomously adapt to changing situations during deployment.