The issue of explainability for autonomous systems is becoming increasingly prominent. Several researchers and organisations have advocated the provision of a “Why did you do that?” button which allows a user to interrogate a robot about its choices and actions. We take previous work on debugging cognitive agent programs and apply it to the question of supplying explanations to end users in the form of answers to why-questions. These previous approaches are based on the generation of a trace of events in the execution of the program and then answering why-questions using the trace. We implemented this framework in the agent infrastructure layer and, in particular, the Gwendolen programming language it supports – extending it in the process to handle the generation of applicable plans and multiple intentions. In order to make the answers to why-questions comprehensible to end users we advocate a two step process in which first a representation of an explanation is created and this is subsequently converted into natural language in a way which abstracts away from some events in the trace and employs application specific predicate dictionaries in order to translate the first-order logic presentation of concepts within the cognitive agent program in natural language. A prototype implementation of these ideas is provided.

@en

To improve a negotiator's ability to recognise bidding strategies, we pro-actively provide explanations that are based on the opponent's bids and the negotiator's guesses about the opponent's strategy. We introduce an aberration detection mechanism for recognising strategies and the notion of an explanation matrix. The aberration detection mechanism identifies when a bid falls outside the range of expected behaviour for a specific strategy. The explanation matrix is used to decide when to provide what explanations. We evaluated our work experimentally in a task in which participants are asked to identify their opponent's strategy in the environment of a negotiation support system, namely the Pocket Negotiator (PN). We implemented our explanation mechanism in the PN and experimented with different explanation matrices. As the number of correct guesses increases with explanations, indirectly, these experiments show the effectiveness of our aberration detection mechanism. Our experiments with over 100 participants show that suggesting consistent strategies is more effective than explaining why observed behaviour is inconsistent.

@en

The evaluation of cognitive agent systems, which have been advocated as the next generation model for engineering complex, distributed systems, requires more benchmark environments that offer more features and involve controlling more units. One issue that needs to be addressed time and again is how to create a connector for interfacing cognitive agents with such richer environments. Cognitive agents use knowledge technologies for representing state, their actions and percepts, and for deciding what to do next. Issues such as choosing the right level of abstraction for percepts and action synchronization make it a challenge to design a cognitive agent connector for more complex environments. The leading principle for our design approach to connectors for cognitive agents is that each unit that can be controlled in an environment is mapped onto a single agent. We design a connector for the real-time strategy (RTS) game StarCraft and use it as a case study for establishing a design method for developing connectors for environments. StarCraft is particularly suitable to this end, as AI for an RTS game such as StarCraft requires the design of complicated strategies for coordinating hundreds of units that need to solve a range of challenges including handling both short-term as well as long-term goals. We draw several lessons from how our design evolved and from the use of our connector by over 500 students in two years. Our connector is the first implementation that provides full access for cognitive agents to StarCraft: Brood War.

@en

We study the impact of using different organisational paradigms on the design and implementation of a Multi-Agent System (MAS) for Real-Time Strategy (RTS) games We examine systems designed and implemented according to a specific paradigm on their performance in a practical scenario, as well as examining software-engineering concepts like size and complexity entailed by the according implementations In contrast to related theoretical work, we deal with the practical constraints and implications of the paradigms by targeting the prototypical RTS game StarCraft: Brood War Through careful analysis of this environment, agent systems for four separate paradigms that operate at different levels of autonomy and communication are designed, implemented, and evaluated by thousands of instrumented runs One of the main findings is that using a central processing agent, e.g in a market-based approach, increases task performance, but at the cost of increased code complexity.

@en

Agent-oriented programming (AOP) is a programming paradigm introduced roughly thirty years ago as an approach to problems in Artificial Intelligence (AI). An agent is a piece of software that can perceive its environment (e.g., through sensors) and act upon that environment (e.g., through actuators). A cognitive agent is a specific type of agent that executes a decision cycle in which it processes events and selects actions based on cognitive notions such as beliefs and goals. Often, multiple agents are used, which is referred to as a multi-agent system (MAS). MAS is generally advertised an approach to handling problems that require multiple problem solving methods, multiple perspectives, and multiple problem solving entities. Tools and techniques for the programming of cognitive agents need to be based on the underlying agent-oriented paradigm, which is a significant challenge, as unlike more traditional paradigms, they should for example take into account that agents execute a specific decision cycle and operate in non-deterministic environments. Therefore, in this thesis, we take existing AOP theories a step further by designing tools for the development of cognitive agent programs with an explicit focus on usability. Each development tool we propose is extensively evaluated on hundreds of (novice) agent programmers. In the context of AOP, the process of detecting, locating and correcting mistakes in a computer program, known as debugging, is particularly challenging. As large part of the effort of a programmer consists of debugging a program, efficient debugging is an essential factor for both productivity and program quality. In this thesis, we contribute both to the process of locating mistakes in agent programs as well as the process of identifying misbehaviour of an agent in the first place...@en

The main goal of our demonstration is to show how omniscient debugging can be applied in practice to cognitive agents. A concrete implementation of the mechanisms proposed in [Koeman et al., 2017] has been created for the GOAL agent programming language in the Eclipse environment, integrated with the source-level debugger of [Koeman et al., 2016], thus fully implementing the proposal within a state-of-the-art setting. The implementation will be used together with typical agent programs to demonstrate its practical use.@en

For real-time programs reproducing a bug by rerunning the system is likely to fail, making fault localization a time-consuming process. Omniscient debugging is a technique that stores each run in such a way that it supports going backwards in time. However, the overhead of existing omniscient debugging implementations for languages like Java is so large that it cannot be effectively used in practice. In this paper, we show that for agent-oriented programming practical omniscient debugging is possible. We design a tracing mechanism for efficiently storing and exploring agent program runs. We are the first to demonstrate that this mechanism does not affect program runs by empirically establishing that the same tests succeed or fail. Usability is supported by a trace visualization method aimed at more effectively locating faults in agent programs.

@en

When an agent program exhibits unexpected behaviour, a developer needs to locate the fault by debugging the agent’s source code. The process of fault localisation requires an understanding of how code relates to the observed agent behaviour. The main aim of this paper is to design a source-level debugger that supports single-step execution of a cognitive agent program. Cognitive agents execute a decision cycle in which they process events and derive a choice of action from their beliefs and goals. Current state-of-the-art debuggers for agent programs provide insight in how agent behaviour originates from this cycle but less so in how it relates to the program code. As relating source code to generated behaviour is an important part of the debugging task, arguably, a developer also needs to be able to suspend an agent program on code locations. We propose a design approach for single-step execution of agent programs that supports both code-based as well as cycle-based suspension of an agent program. This approach results in a concrete stepping diagram ready for implementation and is illustrated by a diagram for both the Goal and Jason agent programming languages, and a corresponding full implementation of a source-level debugger for Goal in the Eclipse development environment. The evaluation that was performed based on this implementation shows that agent programmers prefer a source-level debugger over a purely cycle-based debugger.@en

Debugging is notoriously dicult and extremely time consuming but also essential for ensuring the reliability and quality of a software system. In order to reduce debugging effort and enable automated failure detection, we propose an automated testing framework for detecting failures in cognitive agent programs. Our approach is based on the assumption that modules within such programs are a natural unit for testing. We identify a minimal set of temporal operators that enable the specication of test conditions and show that the test language is suciently expressive for detecting all failures in an existing failure taxonomy. We also introduce an approach for specifying test templates that supports a programmer in writing tests. Furthermore, empirical analysis of agent programs allows us to evaluate whether our approach using test templates detects all failures.@en