Reinforcement learning (RL) is promising for complicated stochastic nonlinear control problems. Without using a mathematical model, an optimal controller can be learned from data evaluated by certain performance criteria through trial-and-error. However, the data-based learning approach is notorious for not guaranteeing stability, which is the most fundamental property for any control system. In this paper, the classic Lyapunov's method is explored to analyze the uniformly ultimate boundedness stability (UUB) solely based on data without using a mathematical model. It is further shown how RL with UUB guarantee can be applied to control dynamic systems with safety constraints. Based on the theoretical results, both off-policy and on-policy learning algorithms are proposed respectively. As a result, optimal controllers can be learned to guarantee UUB of the closed-loop system both at convergence and during learning. The proposed algorithms are evaluated on a series of robotic continuous control tasks with safety constraints. In comparison with the existing RL algorithms, the proposed method can achieve superior performance in terms of maintaining safety. As a qualitative evaluation of stability, our method shows impressive resilience even in the presence of external disturbances.

@en

This project aims to improve employees’ social well-being in offices through informal interactions. It consists of 3 phases in total, context research, idea generation and concept development. Experiencing prototyping was mainly used in the design process.

In the first phase, observation and interviews were conducted in 2 offices in Beijing to know about the context and discover problems and design opportunities. Another round of research was carried out in StudioLab for verification of the insights from Beijing offices. The results, combining with the results of the literature review, led to the design goal defined.

Then the project started to focus on stimulating meaningful chats among employees to increase their sense of belonging. Ideas were brainstormed and selected, prototypes and storyboards were made to let participants evaluate the concepts through interviews. After 3 cycles of idea generation, the final concept direction was defined.

After that, 2 cycles of concept development were conducted to iterate the concept. Again, prototypes were made and interviews were done to evaluate the concept and get feedback from participants. In the end, the final design was defined, final prototypes were made and final evaluation was conducted.

Overall, it is a design project exploring solutions for solving one of employees’ social problems in offices, lacking the sense of belonging.