This article showcases our effort to explore the music club of the future. We present the development and results of an end-to-end system which enhances the club-going experience through the use of wearable technology. Each party guest wearing one of the wristbands actively contributes to the overall experience with their movement and location patterns. The system collects acceleration data from each of the attendees in real-time and feeds it into a pluggable network infrastructure, which processes the data, affecting the environment via data visualization or controlling of the light and sound system of a curated space within the club. Finally we describe the results of a two night, 450 person per night deployment.@en

This article presents an end-to-end system for capturing physiological sensor data and visualising it on a real-time graphic dashboard and as part of an art installation. More specifically, it describes an event where the level of engagement of the audience was measured by means of Galvanic Skin Response (GSR) sensors and of the presenter through a sweater fitted with GSR, ECG and acceleration sensors. The gathered data was presented in real-time through a visualisation projected onto a screen and a physical electro-mechanical installation, which would change the height of helium-filled balloons depending on the atmosphere in the auditorium. Thereby trying to create a tangible way of making the invisible visible.

@en

The CWI-ADE2016 Dataset is a collection of more than 40 million Bluetooth Low Energy (BLE) packets and of 14 million accelerometer and temperature samples generated by wristbands that people wore in a nightclub. The data was gathered during Amsterdam Dance Event 2016 in an exclusive club experience curated around human senses, which leveraged technology as a bridge between the club and the guests. Each guest was handed a custom-made wristband with a BLE-enabled device that broadcast movement, temperature and other sensor readings. A network of Raspberry Pi receivers deployed for the occasion captured broadcast packets from wristbands and any other BLE device in the environment. This data provides a full picture of the performance of the real life deployment of a sensing infrastructure and gives insights to designing sensing platforms, understanding networks and crowds behaviour or studying opportunistic sensing. This paper describes an analysis of this dataset and some examples of usage.

@en

Dynamic Adaptive Streaming over HTTP (DASH) is a technology for delivering video content over the Internet. It provides an effective mechanism, which has been adopted by major content providers. Nevertheless, available DASH player implementations have a number of drawbacks such as performance problems on shared network connections, which lead to video freezes and frequent video quality changes. In this paper, we propose a method to reduce the performance problems that exist in networks with a large number of DASH players. These networks can be found in hotels, apartment complexes, and airports. In experiments with up to 600 simultaneously active players, we are able to reduce the number of DASH players with freezes by 95% (from 345 to 15) compared to throughput-based adaptation and by 75% (from 62 to 15) compared to BOLA using our DASH Assisting Network Element DANE). In addition, we reduced the number of quality switches by 94% compared to throughput-based adaptation, and by 85% compared to BOLA.@en

In September 2014, a local theatre company performed "the Tempest" simultaneously at two different locations to two separate audiences. Both audiences were linked together using an advanced video system, where several cameras captured the play. This is just one example of the radical shift in performing arts, where small theatre companies can use the Internet and a range of digital tools for reaching a wider remote audience. The question remains: what is the influence of this shift on the experience of the audience members? In order to better understand the problem space, we conducted two experiments focusing on two common current scenarios: remote asynchronous and synchronous watching of a theatre play. First, a theatre play was recorded and shown at a later date in a cinema to an audience. Second, a play in one theatre was broadcast to another theatre in real time. This paper reports the results of the experiments and discusses the implications towards the audience when bridging technology and performing arts. According to the results, a shift in time has a deep impact, with the audience rating their watching experience less intensive by 25% to the audience at the live venue. In the second experiment, on the other hand, both audiences reported fairly similar experiences, but different parts of the play had significant different impacts depending on the location where the audience was (in front of the stage or at another theatre). In particular it seems that lacking a way to show appreciation to the play e.g., applause has a big impact on the watching experience. The main conclusion though is that better mechanisms for including remote audiences in the experience are needed.

@en

Research has shown that physiological sensors provide a valuable mechanism for quantifying the experience of audiences attending cultural events. In the strength of the applause or questionnaires, bio-sensors provide fine-grained timed data that can be used to infer the quality of the experience of the audience members. Unfortunately, available commercial sensors are designed for lab or home usage and studies, focusing on the individual instead of the reactions of a crowd of people. In this study, we present our own designed physiological measurement system that overcomes the challenges of using and deploying sensors in a theatrical environment (anonymity and privacy, real-time gathering of data, support for large crowds of 30-100 people), and the wearable system has two unique features that distinguished between adult audience members and children ones. We report our experimental results which particularly answered the research questions proposed by the producer, director, and the artists.@en

Multi-screen applications have been a research topic for the last 10 years. Recent technical advances make authoring and broadcasting of interactive multi-platform experiences possible. However, most of the efforts have been dedicated to the delivery and transmission technology (e.g., HbbTV2.0), but not to the production process. The hypothesis of this paper is that studio and outside broadcast production requires radical changes in the production workflow, in order to allow for an efficient management of interactive multi-platform experiences. This paper explores such changes, investigating workflows and roles, and identifying key requirements for supporting these. The final objective is to create a new set of tools, which are extending current processes, that allow broadcasters to curate new types of experiences. We conducted a set of interviews with broadcast producers and directors that allowed us to identify two major (sub-)workflows, one for pre-recorded and one for live experiences. We could then assign roles to the different stages of the workflows and derive a number of requirements for the next generation of production tools.@en

The curved screen has attracted considerable attentions in recent years, since it enables to enlarge the view angle and to enhance the immersive perception for users. However, existing curved surface projections are frequently prone to geometric distortion or loss of content. This paper presents a content-aware and depth-aware image adaptation solution for curved displays. An efficient optimization approach of image deformation is proposed to preserve local scene content and to minimize scene geometry distortion. To follow the original 3D perception of objects in different depth layers, the depth information is re-mapped for individual content scaling. Objective evaluation results reveal that our approach can effectively preserve foreground objects. We also perform a subjective evaluation of the proposed solution, and compare it to two alternative mapping methods, which are tested on different curvatures on both a traditional screen and an ad-hoc curvature-controllable curved display. Experimental results demonstrate that our approach outperforms other existing mapping methods for immersive display of rectangle images on curved screens.@en

We present a generic and real-time time-varying point cloud codec for 3D immersive video. This codec is suitable for mixed reality applications where 3D point clouds are acquired at a fast rate. In this codec, intra frames are coded progressively in an octree subdivision. To further exploit inter-frame dependencies, we present an inter-prediction algorithm that partitions the octree voxel space in N times N times N macroblocks (N=8,16,32). The algorithm codes points in these blocks in the predictive frame as a rigid transform applied to the points in the intra coded frame. The rigid transform is computed using the iterative closest point algorithm and compactly represented in a quaternion quantization scheme. To encode the color attributes, we defined a mapping of color per vertex attributes in the traversed octree to an image grid and use legacy image coding method based on JPEG. As a result, a generic compression framework suitable for real-time 3D tele-immersion is developed. This framework has been optimized to run in real-time on commodity hardware for both encoder and decoder. Objective evaluation shows that a higher rate-distortion (R-D) performance is achieved compared to available point cloud codecs. A subjective study in a state of art mixed reality system shows that introduced prediction distortions are negligible compared to the original reconstructed point clouds. In addition, it shows the benefit of reconstructed point cloud video as a representation in the 3D Virtual world. The codec is available as open source for integration in immersive and augmented communication applications and serves as a base reference software platform in JCT1/SC29/WG11 (MPEG) for the further development of standardized point cloud compression solutions.@en

Videos are an important part of social platforms. With growing data speeds and high resolution cameras on mobile devices and smartphones, mobile instant and live video clip sharing become increasingly popular. However, video uploads are resource consuming which leads to long upload times, especially in environments with poor data connections. In current mobile applications, the user has little to no influence on optimizing the upload of her/his video according to the current (network) context. In this work, we propose a mobile application that shows an accurate upload time estimation and a current network speed indication. The user can select a video quality for uploading and by that possibly reach faster uploads in low bandwidth connection areas. In a user study with 21 users, we show that users perceive the upload speed as higher with given upload estimation and network speed indication when they have less bandwidth available. With this information, participants perceive the application as more reliable and have an increased feeling of control over the upload process. All users liked the proposed video quality customization feature. Compared to a graphical representation of the network speed, the upload time was the more helpful information to customize the upload.

@en

Accurately measuring the audience response during a performance is a difficult task. This is particularly the case for connected performances. In this paper, we staged a connected performance in which a remote audience enjoyed the performance in real-time. Both objective (galvanic skin response and behaviours) and subjective (interviews) responses from the live and remote audience members were recorded. To capture galvanic skin response, a group of self-built sensors was used to record the electrical conductance of the skin. The results of the measurements showed that both the live and the remote audience members had a similar response to the connected performance even though more vivid artistic artefacts had a stronger effect on the live audience. Some technical issues also influenced the experience of the remote audience. In conclusion we found that the remoteness had little influence on the connected performance.@en

This article presents the results of modelling audience response to new types of networked theatre plays. As the main contribution of the work we introduce two types of metrics: intensity, relating to how intensively co-present and remote aspects of a performance are rated, and cohesion, relating to how a performance as a whole, the combination of co-present and remote aspects, affects an audience. In particular, we model audience response based on two in the wild evaluations, staged by a low budget theatre company, a streamed and a distributed performance. The streamed performance is similar to NT Live, where a theatre play is delivered to other theatres with an audience. The distributed performance, on the other hand, connects actors in two different theatres (with audiences) creating one single play. The streamed performance was experienced as less intense as well as less cohesive by the remote audience, whilst the distributed performance integrated co-present and remote aspects tightly. Remote aspects of the distributed performance were still experienced as less intense, but the performance as a whole was highly cohesive. Apart from the identification of these two new metrics (intensity and cohesion), based on our experiences we argue that an innovative way of bundling relevant emerging technologies is needed to give a voice to the, as yet silent, remote audience.@en

Dynamic adaptive streaming over HTTP (DASH) is a simple, but effective, technology for video streaming over the Internet. It provides adaptive streaming while being highly scalable at the side of the content providers. However, the mismatch between TCP and the adaptive bursty nature of DASH traffic results in underperformance of DASH streams in busy networks. This paper describes a networking architecture based on the Software Defined Networking (SDN) paradigm. Controllers in the network with a broad overview on the network activity provide two mechanisms for adaptation assistance: explicitly signaling target bitrates to DASH players and dynamic traffic control in the network. We evaluate how each of these mechanisms can contribute to the delivery of a stable and high quality stream. It shows that our architecture improves the quality of experience by doubling the video bitrate and reducing disturbing quality switches. As such, this paper contributes insights on how to implement DASH-aware networking that also enables internet service providers, network administrators, and end-users to configure their networks to their requirements.@en

In modern video conferencing services, just as in commonvideo delivery, most of the resource optimization is taken careof in the codec layer. Modern codecs like H.264 use detailedperceptual models to optimize the data reduction in way that itis least noticed by us. Already early evaluations oftelecommunication systems could establish that there aredifferent thresholds for a good quality depending on thesituation. It is further known that subjective quality perceptionsvary from user to user. But the space of user and context factorsis still largely unexplored. To gain insight in which parametersare key in differentiating quality perception, we need to explorethe interaction in different situations while keeping a tightcontrol over the system parameters. In this paper we explorehow clustering participants by their interaction or ratingbehavior can reveal subgroups that show significantly differentperception of the QoE delivered by the same videoconferencingsystem. While for a cluster of users we find video quality toinfluence other QoE dimensions such as audio, for anothercluster this is not the case. We explore whether this effect is dueto conversational dynamics (contextual factor) or individualpreferences (user factor) and discuss what this would mean forthe design of future video-conferencing systems, that want todynamically adapt to situation and participants.@en

In this paper, we present a model for evaluating bandwidth sharing policies, that can be applied to networks that handle both video streaming traffic, as well as other traffic. Video streaming is a demanding network application. In crowded networks, resources need to be properly divided in order not to diminish the streaming experience. However, in network deployments with a large number of users, the streaming performance cannot be obtained straightforwardly from a sharing policy. Therefore, we propose a Markov model that is compatible with Dynamic Adaptive Streaming over HTTP (DASH), the major technology for video streaming over the Internet. If DASH is combined with in-network resource management, its performance can be significantly improved. Nevertheless, resource sharing policies need to be configured. This requires evaluation of many different configurations. Real deployments or network simulations demand many system resources and time. In contrast, our model can quickly evaluate many configurations, and for each configuration output the expected video bitrate and number of changes in video bitrate. These two parameters play an important role in the Quality of Experience of the viewer. In this paper, we demonstrate how our model can be used to analyze and optimize resource sharing policies. As such, our model is a useful tool for network administrators and allows them to better provision and configure their networks.@en

We present MP3DG-PCC, an open source framework for design, implementation and evaluation of point cloud compression algorithms. The framework includes objective quality metrics, lossy and lossless anchor codecs, and a test bench for consistent comparative evaluation. The framework and proposed methodology is in use for the development of an international point cloud compression standard in MPEG. In addition, the library is integrated with the popular point cloud library, making a large number of point cloud processing available and aligning the work with the broader open source community. Copyright is held by the owner/author(s).@en

HTTP adaptive streaming (HAS) has become the dominant technology for streaming video over the Internet. It gained popularity because of its ability to adapt the video quality to the current network conditions and other appealing properties such as usage of off-the-shelf HTTP servers and easy firewall traversal. However, when multiple HAS players share a bottleneck link for streaming, the individual adaptation techniques in the players have difficulties to maintain a stable bitrate and fairly share the network resources. HAS-assisting network elements can solve these performance problems and allow execution of advanced policies for sharing the available bandwidth. Nonetheless, testing and evaluating new sharing policies is costly and time consuming. This motivated us to formulate a model that allows to differentiate between groups of users depending on the type of user or device, and that can describe the mean bitrate of the video streams and how often this bitrate is expected to change during playout. To show how our model can be used, we demonstrate two applications of our model. Furthermore, we validate the model based results against results obtained using our streaming testbed and proxy server based HAS-assistant. The results show that our model is highly accurate for both the mean bitrate and the number of changes in video bitrate. As such, our model is a useful tool for network administrators and internet service providers for evaluating the performance of sharing policies and for managing and provisioning video delivery networks.@en

Most video platforms deliver HD video in high bitrate encoding. Modern video-conferencing systems are capable of handling HD streams, but using multiparty conferencing, average internet connections in the home are on their bandwidth limit. For properly managing the encoding bitrate in videoconferencing, we must know what is the minimum bitrate requirement to provide users an acceptable experience, and what is the bitrate level after which QoE saturates?. Most available subjective studies in this area used rather dated technologies. We report on a multiparty study on video quality with HD resolution. We tested different encoding bitrates (256kbs, 1024kbs and 4096kbs) and packet loss rates (0, 0.5%) in groups of 4 participants with a scenario based on the ITU building blocks task. We discuss the influence of group interaction and individual idiosyncrasies based on different mixed models, and look at covariates engagement and enjoyment as further explanatory factors. We found that 256kbs is still sufficient to provide a fair overall experience, but video quality is noticed to be poor. On the higher bitrate end, most people will not perceive the difference between 1024kbs and 4096kbs, considering in both cases the quality to be close to excellent. Independent on bitrate, packet loss has a small but significant impact, quantifiable in, on average, less than half a point difference on a 5-point ITU scale.@en

Measuring the experience of audience of arts events is essential in the experience economy of this day and age, but it is a difficult task. The value of such information goes beyond evaluating the impact of the arts, as it can provide insights and feedback to enhance the work of artists and the experiences of other audience members. Through in-depth understanding of the needs of the providers and consumers of the arts, we progressively developed a biosensor infrastructure that was deployed in theaters. Over the years, we identified the challenges and issues related to developing and deploying a biosensor infrastructure in theaters. These collective experiences and identified issues were categorized into three main areas: processes, data, and system. A total of seven heuristics are developed across the three main areas. Processes place the stakeholders and audiences at the core of the research; data provides guidelines for data validity, collecting a variety of data, and supporting real-time data gathering; and systems covers the concurrency, scalability, deployment and feedback of the infrastructure. We believe that this set of heuristics forms the foundation for an adequate infrastructure to measure audience experience in the wild and it is a valuable source of guideline for future work.

@en

Several years ago, first conceptualizations of the usages of the secondary screen in the television environment were proposed. At the time, the real challenge was to convince stakeholders that interactivity was not a threat, but an opportunity. Ten years later, the mass adoption of smaller devices has reshaped the media landscape, truly enabling interactivity while consuming media content at home. What was perceived as hindering the user experience - the second screen - has resulted into an essential companion to the television. Paradoxically, even though key players are investing on secondary screen applications, there are very few successful examples. In this paper we provide an overview of the present state of the art through representative examples and discuss future possibilities and challenges. In particular, we will focus on the importance of immersion, taking into account the surrounding of the users, and of sociability, involving her social network.

@en