Analysing transcripts of design activity typically involve either close reading or manual coding of data, which limits the amount of data that can be analysed. In contrast, we explore a machine-learning based linguistic analysis tool called Empath to identify patterns of reasoning in design talk. The data we use derives from the Design Thinking Research Symposium (DTRS) shared-data workshops which we analyse to look at two contrasting aspects of design talk: the expression of tentativeness, characterising designers' generative thinking; and the articulation of explanations, characterising their deductive or analytical thinking. We show, at the level of speech turns, how tentativeness and explanation relate to, and overlap, each other. Finally, we discuss the limitations of this ‘linguistic analysis at scale’ approach.

With the most advanced natural language processing and artificial intelligence approaches, effective summarization of long and multi-topic documents - such as academic papers - for readers from different domains still remains a challenge. To address this, we introduce ConceptEVA, a mixed-initiative approach to generate, evaluate, and customize summaries for long and multi-topic documents. ConceptEVA incorporates a custom multi-task longformer encoder decoder to summarize longer documents. Interactive visualizations of document concepts as a network reflecting both semantic relatedness and co-occurrence help users focus on concepts of interest. The user can select these concepts and automatically update the summary to emphasize them. We present two iterations of ConceptEVA evaluated through an expert review and a within-subjects study. We find that participants' satisfaction with customized summaries through ConceptEVA is higher than their own manually-generated summary, while incorporating critique into the summaries proved challenging. Based on our findings, we make recommendations for designing summarization systems incorporating mixed-initiative interactions.

Creative conversation among designers and stakeholders in a design project enables new ideas to naturally originate and evolve. Language allows for the exchange of values, priorities, and past experience whilst keeping solution forms usefully ambiguous. Yet there is a danger that only the language of people directly involved in the design process gets to be heard, limiting how inclusively the problems are interpreted, which in turn can impede how complex design problems are addressed. Recent advances in artificial intelligence (AI) have shown the exclusionary spaces that are often inhabited by designers, engineers, and developers of new artefacts and technologies. On the other hand, text data used to train language models for machine learning applications have the potential to highlight societal biases in ways that designers can utilise. In this paper, we report the results of an exploratory study using AI text generation to synthesize and narrate opinions and experiences that may be unfamiliar to designers. Three pairs of designers were given a complex socio-technical problem to solve. Of these, two pairs interacted with an AI text generator during the task, while one pair acted as a baseline condition. Analysing the conversational exchanges between the designers and the designers & AI, we observe how the use of AI leads to prompting nuanced interpretations of problems and ideas, opening up the objective problem and design lenses and interpretations. Finally, we discuss how the designers (re)assign different roles to the AI to suit their creative purposes.

In today’s world, the advantages of data-enabled design are undeniable, increasing the performance of organisations drastically by informing and inspiring the design process. While organisations seem to be more experienced with quantitative data for evaluative purposes, they do struggle to use data as creative material to inspire the design process. Choosing the right type of data representation is critical for using data for creative purposes. Data visualization has proven to be highly effective in increasing understanding of data, as it is fast, accurate and flexible. Data physicalization, on the other hand, remains unexplored in comparison, especially its effect on creativity. This paper presents the results of two studies (one preliminary and one follow-up study), which explored the use of data physicalization in creative settings. The preliminary study enabled to collect initial requirements for the development of a physicalization toolkit, while the follow-up study investigated its impact on the design process, in comparison to data visualization. From the studies, we developed Concreate, a collaborative data physicalization toolkit designed to lead to creative insights from quantitative data. Our results show that Concreate can potentially stimulate creative thinking, by encouraging intense, tangible interaction with data leading to increased reflection-in-action and a deeper understanding of data. The two studies and toolkit development were carried out at a multinational automotive company, interested in innovating by incorporating data as creative material. Besides the immediate practical implications, we conclude this paper with a discussion on future recommendations for using data physicalization in the design process.

Multidimensional data is often visualized using coordinated multiple views in an interactive dashboard. However, unlike in infographics where text is often a central part of the presentation, there is currently little knowledge of how to best integrate text and annotations in a visualization dashboard. In this paper, we explore a technique called FacetNotes for presenting these textual annotations on top of any visualization within a dashboard irrespective of the scale of data shown or the design of visual representation itself. FacetNotes does so by grouping and ordering the textual annotations based on properties of (1) the individual data points associated with the annotations, and (2) the target visual representation on which they should be shown. We present this technique along with a set of user interface features and guidelines to apply it to visualization interfaces. We also demonstrate FacetNotes in a custom visual dashboard interface. Finally, results from a user study of FacetNotes show that the technique improves the scope and complexity of insights developed during visual exploration.

Visual data storytelling is gaining importance as a means of presenting data-driven information or analysis results, especially to the general public. This has resulted in design principles being proposed for data-driven storytelling, and new authoring tools being created to aid such storytelling. However, data analysts typically lack sufficient background in design and storytelling to make effective use of these principles and authoring tools. To assist this process, we present ChartStory for crafting data stories from a collection of user-created charts, using a style akin to comic panels to imply the underlying sequence and logic of data-driven narratives. Our approach is to operationalize established design principles into an advanced pipeline that characterizes charts by their properties and similarities to each other, and recommends ways to partition, layout, and caption story pieces to serve a narrative. ChartStory also augments this pipeline with intuitive user interactions for visual refinement of generated data comics. We extensively and holistically evaluate ChartStory via a trio of studies. We first assess how the tool supports data comic creation in comparison to a manual baseline tool. Data comics from this study are subsequently compared and evaluated to ChartStory's automated recommendations by a team of narrative visualization practitioners. This is followed by a pair of interview studies with data scientists using their own datasets and charts who provide an additional assessment of the system. We find that ChartStory provides cogent recommendations for narrative generation, resulting in data comics that compare favorably to manually-created ones.

Dynamic networks-networks that change over time-can be categorized into two types: offline dynamic networks, where all states of the network are known, and online dynamic networks, where only the past states of the network are known. Research on staging animated transitions in dynamic networks has focused more on offline data, where rendering strategies can take into account past and future states of the network. Rendering online dynamic networks is a more challenging problem since it requires a balance between timeliness for monitoring tasks-so that the animations do not lag too far behind the events-and clarity for comprehension tasks-to minimize simultaneous changes that may be difficult to follow. To illustrate the challenges placed by these requirements, we explore three strategies to stage animations for online dynamic networks: time-based, event-based, and a new hybrid approach that we introduce by combining the advantages of the first two. We illustrate the advantages and disadvantages of each strategy in representing low- and high-throughput data and conduct a user study involving monitoring and comprehension of dynamic networks. We also conduct a follow-up, think-aloud study combining monitoring and comprehension with experts in dynamic network visualization. Our findings show that animation staging strategies that emphasize comprehension do better for participant response times and accuracy. However, the notion of 'comprehension' is not always clear when it comes to complex changes in highly dynamic networks, requiring some iteration in staging that the hybrid approach affords. Based on our results, we make recommendations for balancing event-based and time-based parameters for our hybrid approach.

A necessary condition of understanding how designers work is understanding how designers talk. In this paper we show how new methods of linguistic data analysis are beginning to reveal insights into the general nature of design conversations. For the first time we combine design activity data collected over 30 years by the Design Thinking Research Symposium (DTRS) ‘shared data’ series into a single corpus. We apply emerging techniques of analysis on this corpus and explore word forms, expressions, topics, and themes related to the particularities of how designers talk. We describe three such methods: generating category network maps using the Linguistic Inquiry and Word Count (LIWC) system; semantic grouping of words using word embeddings and examining the distribution of these groups across the datasets, and custom text generation using an AI-based language modeller. In applying these methods, we show that exploring design activity data at the corpus level can reveal more general patterns of design talk and raise key questions and hypotheses for further study. We see these methods as a first step in developing an understanding of how people not considered to be designers (e.g., scientists, business people, politicians) talk in ways that might be considered ‘designerly’.

Analysis of large, high-dimensional, and heterogeneous datasets is challenging as no one technique is suitable for visualizing and clustering such data in order to make sense of the underlying information. For instance, heterogeneous logs detailing machine repair and maintenance in an organization often need to be analyzed to diagnose errors and identify abnormal patterns, formalize root-cause analyses, and plan preventive maintenance. Such real-world datasets are also beset by issues such as inconsistent and/or missing entries. To conduct an effective diagnosis, it is important to extract and understand patterns from the data with support from analytic algorithms (e.g., finding that certain kinds of machine complaints occur more in the summer) while involving the human-in-the-loop. To address these challenges, we adopt existing techniques for dimensionality reduction (DR) and clustering of numerical, categorical, and text data dimensions, and introduce a visual analytics approach that uses multiple coordinated views to connect DR + clustering results across each kind of the data dimension stated. To help analysts label the clusters, each clustering view is supplemented with techniques and visualizations that contrast a cluster of interest with the rest of the dataset. Our approach assists analysts to make sense of machine maintenance logs and their errors. Then the gained insights help them carry out preventive maintenance. We illustrate and evaluate our approach through use cases and expert studies respectively, and discuss generalization of the approach to other heterogeneous data.

Creating Spatial Computing (SComp) artifacts (including Virtual Reality, Augmented Reality, Mixed Reality, and Ambient Intelligent artifacts) is a rapidly-emerging domain in need of new design methodologies. In this paper, we examine whether and how ethics are procedurally integrated into the creations of SComp artifacts. After an introduction to terminology—including a re-framed definition of Spatial Computing—findings of interviews with Spatial Computing practitioners are shared. The interviews indicated an awareness among professionals about the inordinate vulnerability of SComp artifacts, and about the need for—and the lack thereof—processes and tests to mitigate negative effects of SComp artifacts. Results from the domain expert interviews are integrated into a proposed framework: The Framework for Ethical Spatial Computing Design Engineering. Our framework serves to support researchers and practitioners in devising new methodologies unique to Spatial Computing by highlighting considerations central to the creation of ethical artifacts. The framework integrates the findings from the in-depth interview study and builds on existing models in Design Process, Methods, and Human-Computer Interaction (HCI) Research that highlight important barriers and opportunities between research and practice. It maps the three-phases journey consisted of (1) Enablers, (2) Synthesizers, and (3) SComp Artifacts. We trust that our work sheds light on considerations necessary to the creation of ethical Spatial Computing artifacts.

Current text visualization techniques typically provide overviews of document content and structure using intrinsic properties such as term frequencies, co-occurrences, and sentence structures. Such visualizations lack conceptual overviews incorporating domain-relevant knowledge, needed when examining documents such as research articles or technical reports. To address this shortcom-ing, we present ConceptScope, a technique that utilizes a domain ontology to represent the conceptual relationships in a document in the form of a Bubble Treemap visualization. Multiple coordi-nated views of document structure and concept hierarchy with text overviews further aid document analysis. ConceptScope facilitates exploration and comparison of single and multiple documents re-spectively. We demonstrate ConceptScope by visualizing research articles and transcripts of technical presentations in computer sci-ence. In a comparative study with DocuBurst, a popular document visualization tool, ConceptScope was found to be more informative in exploring and comparing domain-specifc documents, but less so when it came to documents that spanned multiple disciplines.

Wearable technologies draw on a range of disciplines, including fashion, textiles, HCI, and engineering. Due to differences in methodology, wearables researchers can experience gaps or breakdowns in values, goals, and vocabulary when collaborating. This situation makes wearables development challenging, even more so when technologies are in the early stages of development and their technological and cultural potential is not fully understood. We propose a common ground to enhance the accessibility of wearables-related resources. The objective is to raise awareness and create a convergent space for researchers and developers to both access and share information across domains. We present CHIMERA, an online search interface that allows users to explore wearable technologies beyond their discipline. CHIMERA is powered by a Wearables Taxonomy and a database of research, tutorials, aesthetic approaches, concepts, and patents. To validate CHIMERA, we used a design task with multidisciplinary designers, an open-ended usability study with experts, and a usability survey with students of a wearables design class. Our findings suggest that CHIMERA assists users with different mindsets and skillsets to engage with information, expand and share knowledge when developing wearables. It forges common ground across divergent disciplines, encourages creativity, and affords the formation of inclusive, multidisciplinary perspectives in wearables development.

Engineering design typically occurs as a collaborative process situated in specific context such as computer-supported environments, however there is limited research examining the dynamics of design collaboration in specific contexts. In this study, drawing from situative learning theory, we developed two analytic lenses to broaden theoretical insights into collaborative design practices in computer-supported environments: (a) the role of spatial and material context, and (b) the role of social interactions. We randomly assigned participants to four conditions varying the material context (paper vs. tablet sketching tools) and spatial environment (private room vs commons area) as they worked collaboratively to generate ideas for a toy design task. We used wearable sociometric badges to automatically and unobtrusively collect social interaction data. Using partial least squares regression, we generated two predictive models for collaboration quality and creative fluency. We found that context matters materially to perceptions of collaboration, where those using collaboration-support tools perceived higher quality collaboration. But context matters spatially to creativity, and those situated in private spaces are more fluent in generating ideas than those in commons areas. We also found that interaction dynamics differ: synchronous interaction is important to quality collaboration, but reciprocal interaction is important to creative fluency. These findings provide important insights into the processual factors in collaborative design in computer-supported environments, and the predictive role of context and conversation dynamics. We discuss the theoretical contributions to computer-supported collaborative design, the methodological contributions of wearable sensor tools, and the practical contributions to structuring computer-supported environments for engineering design practice.