Abstract: This work aims to validate the “minimal AR” authoring approach in a real industrial assembly scenario. It focuses on optimizing visual assets in Augmented Reality (AR) work instructions. The design of AR assembly documentation is influenced by three main variables: work instructions, affordance (dependent on equipment components and operator capabilities), and AR signifiers (combination of visual assets with their properties). In this study, we fixed the instruction complexity while exploring the relationship between affordance and AR signifiers. First, we set up a focus group of 10 experts in AR technical documentation to extract guidelines for the design of minimal AR signifiers for assembly instructions with a variable affordance. Then, we validated these guidelines through an industrial case study involving 34 participants in four assembly tasks. We verified if the candidate minimal AR signifier, obtained using the proposed guidelines, corresponded to the minimal AR signifier established by users. The results showed that in 33% of the cases, users exploited the candidate minimal AR signifier to accomplish the task successfully. Beyond the minimal AR signifier, an additional one conveying the notification about the task success must always be provided to ensure failure by those operators with reduced capabilities. We also found that, in 29% of the cases, users needed less information than the candidate minimal AR signifier due to their higher capabilities. However, as expected, this condition leads users to make more errors than with the candidate minimal AR signifier. Moreover, the study confirms that AR signifiers with redundant information or attractive appearance, such as animated product models, are unnecessary to improve task comprehension. Still, animations could be beneficial in reinforcing understanding when object properties are difficult to detect.

Keywords: Assembly | Authoring | Industrial augmented reality | Minimal information | Visual asset | Work instruction

Abstract: This study proposes a ground-breaking idea at the intersection of Artificial Intelligence and virtual education: the creation of an AI-powered Digital Twin instructor in a Metaverse-based classroom using Large Language Models. We aim to build a teacher avatar capable of dynamic interactions with students, tailored teaching approaches, and contextual response inside a virtual world. The research aims to address two major issues for both students and teachers: the Digital Twin can provide feedbacks to resolve doubts about course content and material; also, it can improve student management and allow teachers to answer the trickiest questions raised by students.

Keywords: Artificial intelligence | Computer education programs | Computer-assisted instruction | dialogue and pragmatics Applied computing | Discourse | Education | Natural language processing | Professional topics | Social and professional topics | Software engineering education; Computing methodologies

Abstract: The demand for ethically sourced and safe products has surged, prompting industries to adopt intricate traceability systems. Blockchain technology revolutionizes traceability by ensuring data integrity and transparency in supply chains. However, complexities within supply chains often obfuscate meaningful insights for consumers. This paper explores leveraging Augmented Reality to enhance Blockchain-based traceability systems. By integrating AR, consumers can seamlessly access traceability information through QR codes, presented via optimized 3D models. This immersive approach fosters trust by visually demonstrating product quality. The architecture combines QR codes, Vuforia markers, and Blockchain, ensuring data security and immutability.

Keywords: Computer systems organization | Distributed architectures | Security and privacy | Security services | Software and its engineering | Software design engineering

Abstract: Over the decades, technical documentation has moved from paper to digital, until today in which operators, through Augmented Reality (AR), have the possibility to be helped in component location and recognition for different procedures. To support these activities, AR interfaces need to be developed with instructions provided through visual assets tailored to the information to convey. For localization tasks, auxiliary models, i.e., standard geometric shapes are the most suited. In this work, we propose ADAM (Automatic Development of Auxiliary Models): an authoring tool to support designers in generating various shapes of auxiliary models in the right dimensions and without modeling them in a CAD software. Thanks to this tool, the designer has a ready-to-use database of auxiliary models to choose from reusable in different AR technical documentations. In this way, ADAM allows to significantly reduce the time spent creating each auxiliary model from scratch. We validated our tool in a user study. A focus group of expert designers were asked to implement an AR interface, using auxiliary models for a localization task, with both ADAM Tool and the current authoring practice. The results confirmed that our tool outperforms the current authoring practice in terms of time performance and user experience.

Keywords: Augmented Reality | Authoring | Auxiliary Model | CAD | Technical Documentation

Abstract: This work aims to propose a virtual mirror as a tool for an Augmented Reality (AR) interface in support of operators who perform maintenance tasks, especially in blind areas with component occlusion. In the literature, it is still not clear what is the best AR solution to help workers perform maintenance tasks with occluded components. Thus, we designed an AR interface with a virtual mirror that can assist workers in identifying occluded components thanks to an additional workspace viewpoint. While physical mirrors are commonly employed in maintenance tasks to address blind spots, virtual mirrors are not yet widely adopted. Therefore, our study intends to design and evaluate a virtual mirror that simulates workers’ behavior using a real one in maintenance tasks. We planned a user study to compare the real and virtual mirrors in a real maintenance context. We conducted a performance evaluation with 20 users. Findings indicated no statistically significant differences between the virtual and physical mirrors regarding completion time, accuracy, and cognitive load. Our solution makes it possible to replace a real mirror providing the same user performance but with the advantage of being able to place it without space limitations and observe additional information exploiting AR.

Keywords: Augmented Reality | Blind Area | Maintenance | Occlusion | Virtual Mirror

Abstract: In recent years, Augmented Reality (AR) has been effectively proposed as a tool to support workers in manual procedural tasks in industry, such as assembly and maintenance. It is very common for workers to deal with complete equipment in maintenance. Then, instructions may refer to components located in blind areas, i.e., visually occluded by part of the equipment. Displaying in-situ AR instructions in such blind areas requires handling occlusions; otherwise, side-by-side instructions can be exploited. In the literature, it is still unclear which solution better leverage workers’ performance in maintenance tasks accomplished in blind areas. Thus, we designed three AR presentation modes to convey maintenance instructions in a real machine: 3D in-situ rendered with X-ray technique; 3D side-by-side on a CAD model replicating the blind area; 2D side-by-side on a virtual mirror. We conducted a user study comparing these three presentation modes with a 2D drawing extracted from the original maintenance documentation of the machine. The performance of 42 participants was evaluated in terms of completion time, accuracy, and cognitive load. The results revealed that both in-situ and 3D side-by-side presentation modes perform better than 2D drawing. Specifically, the in-situ presentation mode outperforms the 3D side-by-side mode in terms of completion time. The side-by-side virtual mirror does not improve performance with respect to 2D drawing, then it needs to be redesigned for effective use in AR maintenance interfaces.

Keywords: Augmented reality | Blind area | Industrial operator support | Maintenance | Occlusion | Work instructions

Abstract: In this work, we present a technique to simplify the authoring of Augmented Reality Technical Documentation, allowing technical writers with limited knowledge in Augmented Reality (AR) to produce this new type of documentation in their companies. Contrarily to Traditional Technical Documentation, AR offers the opportunity to provide the exact amount of information needed through a careful design of the AR interface. However, in the literature, there are no established techniques to break work instructions down into elemental pieces of information and define how to convey this information through visual assets in AR. In this work, we proposed identifying the information contained in work instructions of technical documentation, extending the method of “Therbligs,” already used in the literature to describe assembly tasks. First, we defined six classes of information types: identity, location, order, way-to, notification, and orientation. Then, we showed how to use these information types to break down the work instructions of an assembly manual used as a case study. We found that the six information types were enough to analyze the complete manual. The second contribution of this work is the proposal of the most suitable visual asset and its properties for each information type. This goal was accomplished through a technical discussion in a focus group with ten experts in the design of AR technical documentation.

Keywords: Augmented reality | Authoring | Information presentation | Technical documentation | Work instructions

Abstract: Augmented Reality (AR) has been proved to be effective in maintenance operations in the industrial field. In a concurrent engineering approach, the authoring of AR manuals, to convey instructions to operators exploiting AR, must be done during the design phase of the product lifecycle. A reliable solution is needed to speed up the AR manual development process when the product is not physically available. We compared three solutions for displaying a demo version of an AR manual when the real product is not available, opting to replace it with its CAD model. Based on the user study results, the main features for each demo version were collected. The Augmented Reality (AR) solution allows to show the product that is not physically available in a real scale with the drawback of needing a physical printed marker. The Desktop Virtual Reality (DVR) prototype overcomes the problem of managing different devices and real-life locations, but without a real scale and a natural interaction. The Augmented Desktop Virtual Reality (ADVR) prototype allows to distinguish more easily the virtual elements of the true AR from those simulating the real product, but with a less natural interaction due to the use of a secondary screen. As a case study, we chose a compressor that a local company is going to produce with its AR manual. Although users overall preferred the AR demo version, the company chose the ADVR solution due to a better perception of what would be the result of the true AR application.

Keywords: Augmented Reality | Prototyping | Task simulation | Work instructions

Abstract: For the sake of being competitive in an ever-changing market, industrial companies need a redefinition of traditional design and integration of parts, equipment, and services such a redefinition allows effectively addressing the interaction between machines and operators, particularly in the area of complex production lines. In this context, enhancing ergonomics is crucial to reduce fatigue and stress of workers and increase work-place efficiency and comfort. Moreover, identifying ergonomic flaws in three-dimensional human-machine design problems (e.g., body posture, reach, visibility) at an early stage of the engineering process allows to prevent these issues at a low cost. Virtual reality (VR) is emerging as a powerful tool to improve the ergonomic assessment in the design of complex production lines. However, VR is not yet a well-consolidated practice for industrial companies, and the state-of-the-art applications are limited to simplified, isolated, and customized experiments. This work proposes the use of a virtual golden zone (VGZ) as a standard and efficient VR method for the ergonomic analysis and optimization of operator activities in manual manufacturing stations. The resulting effectiveness and benefits are highlighted through the application of the approach to a real industrial case study. Finally, the outcomes of a usability questionnaire, compiled by the professionals involved in the VR reviews, are presented to evaluate the usability of the VGZ methodology in the design process of complex production lines.

Keywords: Complex production lines | Ergonomics | Human computer interaction | Virtual reality

Abstract: The creation and management of content are among the main open issues for the spread of Augmented Reality. In Augmented Reality interfaces for procedural tasks, a key authoring strategy is chunking instructions and using optimized visual cues, i.e., tailored to the specific information to convey. Nevertheless, research works rarely present rationales behind their choice. This work aims to provide design guidelines for the localization of in-view and not occluded components, which is recurrent information in technical documentation. Previous studies revealed that the most suited visual cues to convey this information are auxiliary models, i.e., abstract shapes that highlight the space region where the component is located. Among them, 3D arrows are widely used, but they may produce ambiguity of information. Furthermore, from the literature, it is unclear how to design auxiliary model shapes and if they are affected by the component shapes. To fill this gap, we conducted two user studies. In the first study, we collected the preference of 45 users regarding the shape, color, and animation of auxiliary models for the localization of various component shapes. According to the results of this study, we defined guidelines for designing optimized auxiliary models based on the component shapes. In the second user study, we validated these guidelines by evaluating the performance (localization time and recognition accuracy) and user experience of 24 users. The results of this study allowed us to confirm that designing auxiliary models following our guidelines leads to a higher recognition accuracy and user experience than using 3D arrows.

Keywords: Augmented reality | Authoring | Auxiliary model | Localization

Abstract: Mixed Reality (MR) technology has gained popularity in the manufacturing industry over the last few decades. In the maintenance field, it helps workers in document analysis and decision-making during operations. While numerous studies have demonstrated the empirical benefits of MR, few have examined its applicability in a real industrial context. Among them, many focus on user performance. The novelty of our work is to evaluate mainly the acceptance of MR technology by experienced technicians using smart glasses for maintenance tasks in a real industrial context. We conducted a user study involving 7 workers who were asked to follow the instructions provided by an MR Technical Documentation (MRTD) developed considering authoring guidelines already validated in the literature but only in a laboratory context. The industrial operators performed both navigation tasks within the menu of the MRTD and a disassembly task, following the instructions displayed on the smart glass. The evaluation in the real working scenario showed that MRTD is highly appreciated by workers even if they have no previous experience with smart glasses. These results allow us to confirm the goodness of the design guidelines followed for the authoring of the MRTD.

Keywords: Industrial Operator Support | Mixed Reality | Smart Glasses | Technical Documentation | User Acceptance

Abstract: The biophilic design of virtual workplaces consists of introducing additional natural elements in the virtual environment (VE) with respect to those needed for the main task. It has received increasing attention in recent years, and an increasing number of studies are showing that exposure to biophilic elements in an immersive VE produces positive effects on human well-being. However, in the literature, there are no guidelines about the setting of the different variables of biophilic design in a VE. In this work, we investigated one of these variables, i.e., the effect of the animations of biophilic elements, formulating the following research question: how do animations of biophilic virtual elements affect user attention while performing a working task? We carried out an experiment in which users performed two levels of difficulty of the n-Back cognitive test in three VEs: two identical biophilic VE, one with all static elements and another one in which some of these elements were animated, and an additional empty VE used as a baseline. Performance measurements, eye-tracking measurements, and subjective measurements were collected from 24 users. We found that introducing animations of biophilic elements does not affect user performance in the cognitive task. Furthermore, we found that users’ gaze remains fixed on the main task for more than 94% of the time. However, the use of animations does not bring added value to users in terms of user experience, whereas it causes an increase in perceived distraction and mental effort.

Keywords: Animations | Biophilic Design | Positive Computing | Virtual Reality

Abstract: In this work, we propose a Mixed Reality (MR) application to support laboratory lectures in STEM distance education. It was designed following a methodology extendable to diverse STEM laboratory lectures. We formulated this methodology considering the main issues found in the literature that limit MR’s use in education. Thus, the main design features of the resulting MR application are students’ and teachers’ involvement, use of not distracting graphics, integration of traditional didactic material, and easy scalability to new learning activities. In this work, we present how we applied the design methodology and used the framework for the case study of an engineering course to support students in understanding drawings of complex machines without being physically in the laboratory. We finally evaluated the usability and cognitive load of the implemented MR application through two user studies, involving, respectively, 48 and 36 students. The results reveal that the usability of our application is “excellent” (mean SUS score 84.7), and it is not influenced by familiarity with Mixed Reality and distance education tools. Furthermore, the cognitive load is medium (mean NASA TLX score below 29) for all four learning tasks that students can accomplish through the MR application.

Keywords: augmented and virtual reality | distance education and online learning | improving classroom teaching | mixed reality | mobile learning

Abstract: This work investigates the possibility of using a novel “minimal AR” authoring approach to optimize the visual assets used in augmented reality (AR) interfaces to convey work instructions in manufacturing. In the literature, there are no widely supported guidelines for the optimal choice of visual assets (e.g., CAD models, drawings, and videos). Therefore, to avoid the risk of having AR technical documentation based only on the author’s preference, our work proposes a novel authoring approach that enforces the minimal amount of information to accomplish a task. Minimal AR was tested through a simulated AR LEGO-based assembly task. The performance (completion time, mental workload, errors) of 40 users was evaluated with 4 combinations of visual assets in 4 tasks with an increasing amount of information needed. The main result is that visual assets with an excess of information do not significantly increase performance. Therefore, the location of a specified object should be “minimally” authored by an auxiliary model (e.g., a circle and an arrow). For identifying an object within a couple, color coding is preferred to using additional visual assets. If more than two objects must be identified, a drawing visual asset is also needed. Only when the orientation of a selected object must be conveyed, animated product models are required. These insights could be helpful for an optimal design of AR work instructions in a wide range of industrial fields.

Keywords: Authoring | Industrial augmented reality | Industrial metaverse | Minimal information | Visual asset | Work instruction

Abstract: Industrial Augmented Reality (iAR) has demonstrated its advantages to communicate technical information in the fields of maintenance, assembly, and training. However, literature is scattered among different visual assets (i.e., AR visual user interface elements associated with a real scene). In this work, we present a systematic literature review of visual assets used in these industrial fields. We searched five databases, initially finding 1757 papers. Then, we selected 122 iAR papers from 1997 to 2019 and extracted 348 visual assets. We propose a classification for visual assets according to (i) what is displayed, (ii) how it conveys information (frame of reference, color coding, animation), and, (iii) why it is used. Our review shows that product models, text and auxiliary models are, in order, the most common, with each most often used to support operating, checking and locating tasks respectively. Other visual assets are scarcely used. Product and auxiliary models are commonly rendered world-fixed, color coding is not used as often as expected, while animations are limited to product and auxiliary model. This survey provides a snapshot of over 20 years of literature in iAR, useful to understand established practices to orientate in iAR interface design and to present future research directions.

Keywords: Augmented reality | industry | reviews | user interfaces | visualization

Abstract: Although Virtual Reality Social Skills Training has proven its effectiveness in treating psychiatric disorders, this VR application field is still under-researched for two main reasons. The first one is the unavailability of low-cost VR technologies with sufficient computational capacity needed to render realistic Virtual Environments. The second one consists of the need for specialized VR application developers, usually far from the mental health research field. The recent diffusion of low-cost stereoscopic viewers and the introduction of easy and fast VR content authoring systems, such as Cinematic Virtual Reality (CVR), allow overcoming these limitations. CVR makes it possible to capture real scenes through 360 cameras, augment them with additional virtual objects, and finally immerse the user in these synthetic but highly immersive environments. We present the design and the features of the Entellect360 prototype -an innovative tool supporting the rehabilitation process of subjects affected by schizophrenia. It exploits CVR technology to create Virtual Environments aimed at the rehabilitation of psychiatric patients. The Entellect360 features allow for rehabilitation sessions and patient-performance data-collection even under conditions of social distancing. We also explain the experimental protocol and the validation procedure the prototype will undergo to assess its effectiveness.

Keywords: 360-degree virtual reality | Cognitive rehabilitation | Human-computer interaction | Mental health | Social skills training

Abstract: The introduction of the new generation of Head Mounted Displays (HMD) makes users’ experiences in Mixed Reality (MR) environments more engaging. However, these devices still have a limited field of view, which negatively affects the spatial localization process of virtual objects in the 3D environment. The literature presents several visualization techniques to address this issue, but they currently have several drawbacks, such as visual clutter, occlusion of the real scene, high user workload, and there is still no visualization technique that solves such issues definitively. Therefore, inspired by the gaming industry, we present CompassbAR a visualization technique for out-of-view objects. CompassbAR encodes the position of all out-of-view objects surrounding the user, in a 2D bar positioned at the top of the field of view. In addition, we propose a validation procedure and metrics that aim to evaluate the ability of the CompassbAR visualization technique to guide users towards the out-of-view objects.

Keywords: Head-mounted display | Mixed Reality | Visualization techniques

Abstract: Mixed Reality (MR) could help students in the understanding of complex concepts as well as increase their motivation in the learning process. In this work, our aim is to propose a MR application for the support of engineering students in the understanding of assembly drawings of complex machines. We presented the application of our design methodology for this case study. Then, based on the results of a user study with a sample of students, we tried to improve the usability and the user experience of the MR application, proposing an updated version. The usability of the revised application was in the range “good-excellent” (mean SUS score 77.0). We also presented the lessons learned in this case study, that can be a starting point for a renewal of consolidated didactic processes aiming at future application of MR in other STEM courses.

Keywords: Augmented and virtual reality | Distance education and online learning | Improving classroom teaching | Mixed Reality | Mobile learning

Abstract: In this work, a Coarse-Grained Lattice Spring Model to characterize the mechanical behavior of human mesenchymal stem cells subjected to nanoindentation measurements is presented. The model simulated the action of adhesive structures acting on cells, necessary for attaching them to a substrate, and a nanoindentation process, performed by means of an atomic force microscope with a spherical tip. Cells were hypothesized to behave as elastic materials and the model included several subcellular components such as cell cortex and cytoskeleton. The lattice spring model was integrated within an optimization algorithm that iteratively compared the force-indentation curve numerically predicted to the data experimentally obtained, until a best fit condition was reached. The computed mechanical properties of the cell were compared to those obtained via the Hertz contact theory and finite element modelling, showing a good agreement. The proposed lattice spring model appears as a promising tool that can be used, with a very low computational cost, to characterize cell materials and other biological materials.

Keywords: Cell mechanical properties | Lattice spring models | Nanoindentation | Stem cells

Abstract: Numerous computer-based therapies have been designed for cognitive-behavioral interventions to support children with Autism Spectrum Disorder (ASD) in recent years. Among these technologies, Augmented Reality (AR) offers unique educational benefits because it provides children with direct guidance on their learning tasks. In this work, we propose “AR-brickhouse,” an AR application to support ASD children during LEGO therapy. It combines the benefits derived from AR technology and the LEGO tangible user interface with caregivers’ involvement. The novelty of our system concerns the improvement of ASD children’s basic skills such as positioning in space; focusing on tasks; acquisition of concepts of shape, color, and size. A preliminary user study involved eight ASD children and twelve therapists from a real medical center. Our results suggest that the proposed system is easy for therapists and allows children to improve the aforementioned basic skills. In fact, they were able to accomplish LEGO assembly tasks with better accuracy and in less time than traditional LEGO therapy.

Keywords: Augmented Reality | Autism Spectrum Disorder | LEGO assembly | Play therapy

Abstract: In the future, many activities will be carried out in the Metaverse: hybrid offices and video-based education are just some examples. The way research is carried out could change, too. In this context, this work investigates the possibility of simulating Augmented Reality (AR) user studies on information presentation in a virtual environment. Organizing an industrial setup is complex; thus, most studies are executed in laboratories. However, lab experiments present limitations, e.g., the number and variety of participants and the availability of facilities. User studies may also be carried out by exploiting simulated AR, as an initial step for the Metaverse, where people are connected regardless of their location. This alternative could be used to carry out experiments on AR information presentation to solve common issues, such as the lack of physical equipment to perform component location tasks and the long time required to collect a large sample of users. Indeed, researchers could propose CAD models with information that simulates the same visual realism achieved with true AR. Moreover, multiple tests could be conducted in parallel by not relying on a limited amount of physical equipment per user. In this work, we developed and evaluated a desktop-simulated testing environment (DSTE) to conduct AR information presentation experiments remotely. We applied it in a pilot user study, revealing that the proposed DSTE was effective for the related research goals. Furthermore, 40 participants reported a positive user experience. The evaluation confirms that using a DSTE is promising for collecting and analyzing data from a wide range of people.

Keywords: AR simulation | industrial metaverse | information presentation | user interaction | user study

Abstract: Immersive Vimial Reality (IVR) training offers the capability to industrial workers to acquire skills and address complex tasks by immersing them in a safe and controlled virtual environment (VE). However, in the literature, IVR training is mainly based on principles of standardization and efficiency without considering the operators' well-being. A novel design approach consists of the introduction in the VE of Positive Computing to improve workers' well-being by applying the Biophilia hypothesis. In this work, we explored the possibility of introducing biophilic elements in a VE training scenario that would support psychological well-being and human potential. However, the introduction of virtual elements not related to the training task may distract operators, impairing their performance. We selected as a training scenario the assembly of a real truck engine. It is accomplished in a workstation, and operators do not interact with the surrounding VE. Therefore, we placed the training area into four different types of VEs: 3D Minimal (MIN), 3D Minimal Biophilic enriched (MIN+BIO), 3D Realistic (REAL), and 3D Realistic Biophilic enriched (REAL+BIO). We compared the MIN and REAL scenarios with the respective biophilic enriched scenarios. The performance of 40 participants was evaluated in terms of completion time, object fixation time, training task accuracy, knowledge accuracy, cognitive load, and user experience. The results revealed that introducing biophilic elements in a VR training environment attracts users' attention in the idle phase of the training. In contrast, they keep concentrating on the task without worsening their performance during the task accomplishment

Keywords: Biophilia Hypothesis | Industrial Training | Positive Computing | Virtual Reality

Abstract: [No abstract available]

Abstract: The definition of an innovative category of implantable devices, characterized not only by a fully customization but also by improved osteoconductive and functionalized bioactive surfaces, needs to be supported by a systematic approach. In the present work, a new human-machine interface based on the Mixed Reality (MR) is proposed and focused on the implantation of a fully customized prosthesis. By means of an informal and exploratory focus group (i.e., without using a structured questionnaire), the limitations belonging to the current procedure were highlighted and a first list of user needs was subsequently defined. The MR interface was then considered to be the most suitable solution to match the gathered requirements. However, the doctors proposed also to develop a desktop interface for a finer and easier manipulation of 3D models. The proposed MR application offers several advantages from the possibility to display 3D anatomical structures and 3D models of custom prostheses in an immersive environment to the optimized communication and data exchange among the players (medical staff, doctors and engineers). A mock-up of the MR applications is presented in this work to show the results of the design stage, before the deployment of the application.

Keywords: Collaborative Network | Fully Custom Implants | Immersive Environment | Mixed Reality

Abstract: This study investigates the use of augmented reality technology (AR) in the field of maritime navigation and how researchers and designers have addressed AR data visualisation. The paper presents a systematic review analysing the publication type, the AR device, which information elements are visualised and how, the validation method and technological readiness. Eleven AR maritime solutions identified from scientific papers are studied and discussed in relation to previous navigation tools. It is found that primitive information such as course, compass degrees, boat speed and geographic coordinates continue to be fundamental information to be represented even with AR maritime solutions.

Keywords: augmented reality | data visualisation | human-computer interaction | maritime

Abstract: The procedure commonly adopted to characterize cell materials using atomic force microscopy neglects the stress state induced in the cell by the adhesion structures that anchor it to the substrate. In several studies, the cell is considered as made from a single material and no specific information is provided regarding the mechanical properties of subcellular components. Here we present an optimization algorithm to determine separately the material properties of subcellular components of mesenchymal stem cells subjected to nanoindentation measurements. We assess how these properties change if the adhesion structures at the cell-substrate interface are considered or not in the algorithm. In particular, among the adhesion structures, the focal adhesions and the stress fibers were simulated. We found that neglecting the adhesion structures leads to underestimate the cell mechanical properties thus making errors up to 15%. This result leads us to conclude that the action of adhesion structures should be taken into account in nanoindentation measurements especially for cells that include a large number of adhesions to the substrate.

Keywords: Cell cortex | Cell mechanics | Cytoskeleton | Finite element method | Focal adhesion | Stress fibers

Abstract: Today’s sailing visualization instruments struggle to cope with the increasing number of onboard sensors, automation, artificial intelligence, and the high dy-namism of the crew. Current solutions scatter multiple displays all over the boat, both inside and outside, potentially reducing usability and increasing costs. This work presents a systematic review of augmented reality (AR) as an integral solution for sailing data visualization, which revealed four scientific papers and eight commercial products. We analyzed the publication type, the AR hardware, what and how information is presented using AR, the validation method (if present), and the technological readiness. We defined the technical requirements needed for the AR device for sailing and distinguished a first generation of commercial solutions based on head-up displays from a second one based on proper augmentation with stereo head-mounted displays. The displayed information elements are limited in number and are commonly 2-D graphics (e.g., text and symbols) with a screen-relative frame of reference (as opposed to body-or world-relative). The most visu-alized elements are heading (10) followed by wind direction (8), boat speed (7) compass (7), and wind speed (7). We also found that most of the solutions lack critical evaluation. We conclude that AR has the potential to integrate sailing data from different systems and to improve accessibility, situation awareness, and safety for a large group of users. However, the main limitations are the lack of AR head-mounted displays suitable or adaptable for sailing conditions, an extensive exploration of 3-D interface elements, and an adequate number of usability studies in the scientific literature.

Keywords: Augmented reality | Data visualization | Human-computer interaction | Nautical instruments | Sailing

Abstract: The knowledge of the mechanical properties is the starting point to study the mechanobiology of mesenchymal stem cells and to understand the relationships linking biophysical stimuli to the cellular differentiation process. In experimental biology, Atomic Force Microscopy (AFM) is a common technique for measuring these mechanical properties. In this paper we present an alternative approach for extracting common mechanical parameters, such as the Young's modulus of cell components, starting from AFM nanoindentation measurements conducted on human mesenchymal stem cells. In a virtual environment, a geometrical model of a stem cell was converted in a highly deformable Coarse-Grained Elastic Network Model (CG-ENM) to reproduce the real AFM experiment and retrieve the related force-indentation curve. An ad-hoc optimization algorithm perturbed the local stiffness values of the springs, subdivided in several functional regions, until the computed force-indentation curve replicated the experimental one. After this curve matching, the extraction of global Young's moduli was performed for different stem cell samples. The algorithm was capable to distinguish the material properties of different subcellular components such as the cell cortex and the cytoskeleton. The numerical results predicted with the elastic network model were then compared to those obtained from hertzian contact theory and Finite Element Method (FEM) for the same case studies, showing an optimal agreement and a highly reduced computational cost. The proposed simulation flow seems to be an accurate, fast and stable method for understanding the mechanical behavior of soft biological materials, even for subcellular levels of detail. Moreover, the elastic network modelling allows shortening the computational times to approximately 33% of the time required by a traditional FEM simulation performed using elements with size comparable to that of springs.

Keywords: Atomic force microscopy | Cell material characterization | Elastic network model | Meshless methods

Abstract: This research investigates the application of positive computing principles to Virtual Reality (VR) training scenarios where the Virtual Environment (VE) has not a direct influence on operator learning. We propose to place the 3D models of the only objects needed for the task in a VE consisting of 360° panoramas of natural environments. We made a preliminary evaluation of the user experience which showed that the hedonic quality is significantly higher with this VE than a 3D modeled empty room. However, we also observed a reduction of the pragmatic quality, due to potential distractions. Thus, further research is needed to demonstrate the efficacy of our positive computing approach in training against a traditional one based on the faithful 3D reproduction of the real environment.

Keywords: Industrial Training | Positive Computing | Virtual Reality

Abstract: The research presented in this contribution aims to investigate user preferences about how to convey information in Industrial Augmented Reality (IAR) interfaces to the user. Our interest is focused on the opinion of potential technical writers of IAR documentation for assembly or maintenance operations. Authoring of IAR interfaces imply a choice among various visual assets, that is influenced by the information type and the AR display used. There are no specific standards in the literature to follow and it is challenging to extract guidelines from the literature. This study gathers preferences of 105 selected users that have knowledge about IAR issues, graphical user interfaces (GUI) designing, and assembly/maintenance procedures. The results of this survey show a great preference for 3D CAD models of components (product model) for almost all the information types. However, some alternative visual assets have also been proposed, such as video and auxiliary models. Contrary to common practices in industry, text was the least preferred visual asset. The insights from this research can help other IAR technical writers in the authoring of their interfaces.

Keywords: Graphic Interface | Industrial Augmented Reality | Manual Assembly | Visual Asset

Abstract: Sailing is a multidisciplinary activity that requires years to master. Recently this sustainable sport is becoming even harder due to the increasing number of onboard sensors, automation, artificial intelligence, and the high performances obtainable with modern vessels and sail designs. Augmented Reality technology (AR) has the potential to assist sailors of all ages and experience level and improve confidence, accessibility, situation awareness, and safety. This work presents our ongoing research and methodology for developing AR assisted sailing. We started with the problem definition followed by a state of the art using a systematic review. Secondly, we elicited the main task and variables using an online questionnaire with experts. Third, we extracted the main variables and conceptualized some visual interfaces using 3 different approaches. As final phase, we designed and implemented a user test platform using a VR headset to simulate AR in different marine scenarios. For a real deployment, we witness the lack of available AR devices, so we are developing one specific headset dedicated to this task. We also envision the possible redesign of the entire boat as a consequence of the introduction of AR technology.

Keywords: Augmented Reality | Human Computer Interaction | Nautical | Sailing | Yacht

Abstract: Since its beginning at the end of 2019, the pandemic spread of the severe acute respiratory syndrome coronavirus 2 (Sars-CoV-2) caused more than one million deaths in only nine months. The threat of emerging and re-emerging infectious diseases exists as an imminent threat to human health. It is essential to implement adequate hygiene best practices to break the contagion chain and enhance society preparedness for such critical scenarios and understand the relevance of each disease transmission route. As the unconscious hand–face contact gesture constitutes a potential pathway of contagion, in this paper, the authors present a prototype system based on low-cost depth sensors able to monitor in real-time the attitude towards such a habit. The system records people’s behavior to enhance their awareness by providing real-time warnings, providing for statistical reports for designing proper hygiene solutions, and better understanding the role of such route of contagion. A preliminary validation study measured an overall accuracy of 91%. A Cohen’s Kappa equal to 0.876 supports rejecting the hypothesis that such accuracy is accidental. Low-cost body tracking technologies can effectively support monitoring compliance with hygiene best practices and training people in real-time. By collecting data and analyzing them with respect to people categories and contagion statistics, it could be possible to understand the importance of this contagion pathway and identify for which people category such a behavioral attitude constitutes a significant risk.

Keywords: Azure kinect | Body tracking | Hygiene best practices | Occupational safety | Pandemics containment | Safety training

Abstract: Despite the wide use of scaffolds with spherical pores in the clinical context, no studies are reported in the literature that optimize the micro-architecture dimensions of such scaffolds to maximize the amounts of neo-formed bone. In this study, a mechanobiology-based optimization algorithm was implemented to determine the optimal geometry of scaffolds with spherical pores subjected to both compression and shear loading. We found that these scaffolds are particularly suited to bear shear loads; the amounts of bone predicted to form for this load type are, in fact, larger than those predicted in other scaffold geometries. Knowing the anthropometric characteristics of the patient, one can hypothesize the possible value of load acting on the scaffold that will be implanted and, through the proposed algorithm, determine the optimal dimensions of the scaffold that favor the formation of the largest amounts of bone. The proposed algorithm can guide and support the surgeon in the choice of a "personalized" scaffold that better suits the anthropometric characteristics of the patient, thus allowing to achieve a successful follow-up in the shortest possible time.

Keywords: Bone tissue engineering | Computational mechanobiology | Geometry optimization | Parametric CAD (Computer aided design) model | Python code

Abstract: Technical documentation is evolving from static contents presented on paper or via digital publishing to real-time on-demand contents displayed via virtual and augmented reality (AR) devices. However, how best to provide personalized and context-relevant presentation of technical information is still an open field of research. In particular, the systems described in the literature can manage a limited number of modalities to convey technical information, and do not consider the 'people' factor. Then, in this work, we present a Context-Aware Technical Information Management (CATIM) system, that dynamically manages (1) what information as well as (2) how information is presented in an augmented reality interface. The system was successfully implemented, and we made a first evaluation in the real industrial scenario of the maintenance of a hydraulic valve. We also measured the time performance of the system, and results revealed that CATIM performs fast enough to support interactive AR.

Keywords: Augmented reality | Context-aware | Human-centered design | Human-computer interaction | Industrial | Information manager | Maintenance | Technical documentation

Abstract: In spite of the rather large use of the fused deposition modeling (FDM) technique for the fabrication of scaffolds, no studies are reported in the literature that optimize the geometry of such scaffold types based on mechanobiological criteria. We implemented a mechanobiology-based optimization algorithm to determine the optimal distance between the strands in cylindrical scaffolds subjected to compression. The optimized scaffolds were then 3D printed with the FDM technique and successively measured. We found that the difference between the optimized distances and the average measured ones never exceeded 8.27% of the optimized distance. However, we found that large fabrication errors are made on the filament diameter when the filament diameter to be realized differs significantly with respect to the diameter of the nozzle utilized for the extrusion. This feasibility study demonstrated that the FDM technique is suitable to build accurate scaffold samples only in the cases where the strand diameter is close to the nozzle diameter. Conversely, when a large difference exists, large fabrication errors can be committed on the diameter of the filaments. In general, the scaffolds realized with the FDM technique were predicted to stimulate the formation of amounts of bone smaller than those that can be obtained with other regular beam-based scaffolds.

Keywords: Biomaterials | Geometry optimization | Mechanobiology | Scaffold design | Tissue engineering

Abstract: The enabling technologies of the Industry 4.0 program can support the smart factory of the future to face the challenges related to their sustainable growth. In particular, given the progressive ageing of the population, it is mandatory to develop systems able to preserve operators' wellbeing and to prevent the incidence of work-related musculoskeletal disorders. By exploiting a recently introduced low-cost sensor we developed and validated a reliable prototype for automatic assessment of ergonomic postural risk in the factory shopfloor. Encouraged by the results of the validation process, we enhanced the prototype functionalities. The tool will serve both as a monitoring system for the evaluation of postural risk and a training system for increasing operators' awareness. In this paper, we describe the design of the prototype and the enhanced functionalities of the final version, - the ErgoSentinel.

Keywords: Ergonomics | I4.0 | Kinect® V2 | Postural risk assessment | RULA | Sustainable work

Abstract: Augmented Reality is considered one of the most promising technologies for Industry 4.0. Augmented Reality allows to display the needed information at the right time and to locate it in the desired space, superimposing it to the real world. In this way, it could simplify the work of industrial operators in a variety of tasks, from planning and commissioning to assembly and maintenance. Despite the growing research interest and efforts for the development of Augmented Reality applications and technologies for the industrial domain, Augmented Reality is still rarely used in real industrial procedures, often remaining at a conceptual level. In fact, developers fail to completely answer the actual needs of industries and workers because of a lack of knowledge of the context and the absence of guidelines to drive the design and development of Augmented Reality applications. One of the open issues is the choice of the proper visualization methods to display technical information in Augmented Reality, which is not trivial. Both the paper-based documentation, based on text and illustrations, and digital documentation, including CAD models and image or video-based tutorials, present different criticalities when adapted to the Augmented Reality technology. Other visual material has been developed specifically to be used in Augmented Reality, but it is difficult to use because of a lack of standards which are instead present for the former paper-based and digital documentation. In this work, we categorize and compare different Augmented Reality visual assets, illustrating their advantages and disadvantages and providing directions for future research.

Keywords: Augmented Reality | Graphical User Interface | Industry 4.0 | Technical documentation

Abstract: Schizophrenia poses a significant economic burden on the healthcare system as well as it has a significant impact on society at large. Reasons for such a high economic burden of schizophrenia include the frequent relapses and hospitalizations occurring in this disorder. We analyze the effectiveness of long-acting injectable antipsychotics (LAIs) compared to oral medications, in terms of “clinical process management” in a sample of patients with a diagnosis of schizophrenia spectrum disorder treated in community mental health centers. An observational, retrospective, mirror-image study was carried out to evaluate the effectiveness of LAIs compared to oral medications in terms of number of hospitalizations, emergency visits and planned visits on a 10-year period (from July 2007 to June 2017). Differences between first and second generation LAIs were also explored. Our findings show that hospitalization and emergency visits are significantly decreased with the use of LAIs, while planned visits are increased in patients treated with LAIs. Our results suggest that LAIs, in particular, second generation ones, reduce hospitalization rates and emergency visits, improving the economic burden of schizophrenia. Therefore, LAIs should be considered a cost-effective treatment in the management of schizophrenia under routine conditions.

Abstract: A number of studies have recently demonstrated that the geometry of scaffolds for bone tissue engineering significantly affects the tissue differentiation process and the rate of bone tissue regeneration. These findings and the possibility of fabricating any kind of sophisticated geometries by additive manufacturing techniques led many researchers throughout the world to investigate strategies for the design of scaffolds and for the optimization of their geometry. In this chapter, after revising the numerical optimization algorithms recently implemented to determine the best scaffold geometry we will investigate, in particular, those that are mechanobiology-driven. These algorithms perturb the scaffold microarchitecture until the optimal scaffold geometry, i.e., the geometry that allows maximizing the amounts of bone forming within the scaffold pores, is computed. Different applications of these algorithms to different regular and irregular scaffold geometries will be shown.

Keywords: Beam-based scaffold | Irregular scaffold | Mechanobiology | Mechanoregulation algorithm | Regular scaffold

Abstract: By combining load adaptive algorithms with mechanobiological algorithms, a computational framework was developed to design and optimize the microarchitecture of irregular load adapted scaffolds for bone tissue engineering. Skeletonized cancellous bone-inspired lattice structures were built including linear fibers oriented along the internal flux of forces induced by the hypothesized boundary conditions. These structures were then converted into solid finite element models, which were optimized with mechanobiology-based optimization algorithms. The design variable was the diameter of the beams included in the scaffold, while the design objective was the maximization of the fraction of the scaffold volume predicted to be occupied by neo-formed bony tissue. The performance of the designed irregular scaffolds, intended as the capability to favor the formation of bone, was compared with that of the regular ones based on different unit cell geometries. Three different boundary and loading conditions were hypothesized, and for all of them, it was found that the irregular load adapted scaffolds perform better than the regular ones. Interestingly, the numerical predictions of the proposed framework are consistent with the results of experimental studies reported in the literature. The proposed framework appears to be a powerful tool that can be utilized to design high-performance irregular load adapted scaffolds capable of bearing complex load distributions.

Keywords: finite element method | irregular and regular scaffolds | load adaptive algorithms | mechanobiological algorithms | robustness of optimized structures | structural optimization algorithms

Abstract: Due to the recent advances in technologies for gesture recognition, midair gestures can be considered the interface of the future in a large number of applications. However, designing effective interfaces with midair gestures is not an easy task because the design is application dependent and it must fulfill many requirements at the same time. Despite the availability of general guidelines in the literature, clear and well-established procedures for the optimal design of midair gesture-based interfaces are, to date, not available and remain an open issue. The main contribution of this paper is a user-centered modular framework, which integrates existing and novel methods. It supports the designer considering multiple aspects including ergonomics, memorability, and specific user requirements tailored to the application scenario. The framework involves three design steps and a final validation step, also supported by dedicated software. We tested with success the proposed framework in an industrial case study, where technicians must easily access technical information by browsing digital manuals during maintenance operations.

Keywords: Consumed endurance (CE) | ergonomics | gesture vocabulary | midair gesture interface | user-centered elicitation approach

Abstract: Augmented Reality (AR), is one of the most promising technology for technical manuals in the context of Industry 4.0. However, the implementation of AR documentation in industry is still challenging because specific standards and guidelines are missing. In this work, we propose a novel methodology for the conversion of existing “traditional” documentation, and for the authoring of new manuals in AR in compliance to Industry 4.0 principles. The methodology is based on the optimization of text usage with the ASD Simplified Technical English, the conversion of text instructions into 2D graphic symbols, and the structuring of the content through the combination of Darwin Information Typing Architecture (DITA) and Information Mapping (IM). We tested the proposed approach with a case study of a maintenance manual of hydraulic breakers. We validated it with a user test collecting subjective feedbacks of 22 users. The results of this experiment confirm that the manual obtained using our methodology is clearer than other templates.

Keywords: Augmented reality | Industry 4.0 | Maintenance support | Technical documentation

Abstract: Technical information presentation is evolving from static contents presented on paper or via digital publishing to real-time context-aware contents displayed via virtual and augmented reality devices. We present a Context-Aware Technical Information Management system (CATIM), that dynamically manages (1) what information as well as (2) how information is presented in an augmented reality interface. CATIM acquires context data about activity, operator, and environment, and then based on these data, proposes a dynamic augmented reality output tailored to the current context. The system was successfully implemented and preliminarily evaluated in a case study regarding the maintenance of a hydraulic valve.

Keywords: Aware information | Context | Industrial augmented reality | Technical information manager

Abstract: In this work, we present an Augmented Reality (AR) application for handheld devices that support operators in information retrieval tasks in maintenance procedures in the context of Industry 4.0. Indeed, using AR allows the integration of knowledge-based information, traditionally used by operators and mainly provided in the form of technical drawings, and data available from sensors on the equipment. This approach is suggested by companies, especially Small and Medium-sized Enterprises, that want a gradual introduction of Industry 4.0 technologies within their established practices. We implemented a prototype of the application for the case study of a milling plant. The application augments on a Piping and Instrumentation Diagram (P&ID) of the plant some virtual interactive graphics (hotspots) referenced to specific components drawn. Component data are retrieved, through a user interface, directly from the factory database and displayed on the screen. We evaluated the application through a user study aimed at comparing the AR application with the current practice, based on paper documentation, for an information retrieval task within a maintenance procedure. Results of the study revealed that AR is effective for this task in terms of task time reduction and usability. The AR application was tested both with a tablet and a smartphone, but results revealed that using tablet does not improve user performance in terms of task time, error rate, and usability.

Keywords: Augmented Reality | Industry 4.0 | Information retrieval | Maintenance | User evaluation

Abstract: In this work, we present an Augmented Reality framework for handheld devices that enhance users in the comprehension of plant information traditionally conveyed through printed Piping and Instrumentation Diagrams (P&ID). The proposed framework augments on the P&ID of a plant some virtual interactive graphics (hotspots) referenced to specific components drawn on the P&ID. In this way, it is possible to easily find all the components belonging to the same category (e.g., all the pumps). By tapping, on the tablet screen, on a single hotspot further multimedia information can be displayed: Technical data, 3D CAD model of the component, and 360° images of the plant section. The application is connected to the factory database where all the information associated with the plant components is stored. We used, as a case study, the cleaning section of a milling plant. With the tool presented in this work, technicians will be able to find information updated and in less time, so reducing the intervention time and increasing the accuracy of the operations. Furthermore, the cognitive load associated with the task of understanding the plant is highly reduced through the use of virtual information displayed using Augmented Reality.

Keywords: Augmented Reality | Industrial plant | Industry 4.0 | P&ID | Technical information

Abstract: Scaffolds are porous biomaterials that serve to replace missing portions of bone. Scaffolds must possess a proper geometry and hence have to be adequately designed to correctly undergo to the load and to favor the differentiation of the mesenchymal stem cells invading it, into osteoblasts. It is commonly known that scaffold geometry affects the quality of the regenerated bone creating within the scaffold pores. Scaffold properly designed trigger favorable values of biophysical stimuli that are responsible for the reactions cascade leading to the bone formation. In this paper an optimization algorithm is proposed that, based on mechano-regulation criteria, identifies the optimal geometry of scaffolds, i.e. the geometry that favors the formation of the largest amounts of bone in the shortest time. In detail, the algorithm, written in the Matlab environment, incorporates parametric finite element models of different scaffold types, a computational mechanobiological model and structural optimization routines. The scaffold geometry is iteratively perturbed by the algorithm until the optimal geometry is computed, i.e. the geometry that triggers the most favorable values of the biophysical stimulus which lead to the formation of mature bone. Mesenchymal stem cells were hypothesized to spread within the fracture domain and uniformly occupy the scaffold pores.

Keywords: Hexahedron unit cell | Mechanobiology | Rhombicuboctahedron unit cell | Unit cell geometry

Abstract: Enhancing the performance of scaffolds for bone regeneration requires a multidisciplinary approach involving competences in the fields of Biology, Medicine and Engineering. A number of studies have been conducted to investigate the influence of scaffolds design parameters on their mechanical and biological response. The possibilities offered by the additive manufacturing techniques to fabricate sophisticated and very complex microgeometries that until few years ago were just a geometrical abstraction, led many researchers to design scaffolds made from different unit cell geometries. The aim of this work is to find, based on mechanobiological criteria and for different load regimes, the optimal geometrical parameters of scaffolds made from beam-based repeating unit cells, namely, truncated cuboctahedron, truncated cube, rhombic dodecahedron and diamond. The performance, -expressed in terms of percentage of the scaffold volume occupied by bone-, of the scaffolds based on these unit cells was compared with that of scaffolds based on other unit cell geometries such as: hexahedron and rhombicuboctahedron. A very intriguing behavior was predicted for the truncated cube unit cell that allows the formation of large amounts of bone for low load values and of very small amounts for the medium-high ones. For high values of load, scaffolds made from hexahedron unit cells were predicted to favor the formation of the largest amounts of bone. In a clinical context where medical solutions become more and more customized, this study offers a support to the surgeon in the choice of the best scaffold to be implanted in a patient-specific anatomic region.

Keywords: Beam-based scaffolds | Bone tissue engineering | Diamond | Mechanobiology | Rhombic dodecahedron | Truncated cube | Truncated cuboctahedron | Unit cell

Abstract: One of the most effective strategies that can be adopted to make successful cultural heritage expositions consists in attracting the visitors’ attention and improving their enjoyment/engagement. A mid-air gesture-based Natural User Interface was designed, through the user-centric approach, for the navigation of virtual tours in cultural heritage exhibitions. In detail, the proposed interface was developed to “visit” Murgia, a karst zone lying within Puglia, very famous for its fortified farms, dolines, sinkholes, and caves. Including an “immersive” gesture-based interface was demonstrated to improve the user's experience thus giving her/him the sensation of “exploring” in a seamless manner the wonderful and rather adventurous sites of Murgia. User tests aimed at comparing the implemented interface with a conventional mouse-controlled one confirmed the capability of the proposed interface to enhance the user engagement/enjoyment and to make “more” natural/real, the virtual environment.

Keywords: Gesture vocabulary design | Natural user interface | User-centric approach | Virtual tour

Abstract: The evolution of technical documentation in the age of Industry 4.0 is going towards the use of visual manuals, in particular exploiting Augmented Reality (AR) technology. Traditional manuals are rich of text instructions that in AR applications are not advisable. In fact text occludes the real scene behind and it is an issue for the translation. For this reason, we propose to create and adopt a controlled and exhaustive vocabulary of graphical symbols, to be used in AR to represent maintenance instructions. In particular, in this work we identified the most frequent maintenance actions used in manuals, and converted them into graphical symbols. Then, we made an elicitation of the symbols designed and created different candidate vocabularies of symbols basing on the criteria found in literature of guessability and homogeneity. Moreover, the vocabularies had to respect two constraints: conflict set and reversibility. Finally, we identified the best of symbols and integrated this one in a real AR application for remote maintenance.

Keywords: Augmented Reality | Industry 4.0 | Maintenance

Abstract: Augmented reality (AR) is a key technology for the development of smart manufacturing. One of the main advantages of AR is that it can help workers to accomplish several tasks, making it possible the shift from mass production to mass customization. However, it is still not clear how these promises can be fulfilled in an industrial scenario. In particular, the question about which display solutions fit better the industrial constraints remains open. Based on the literature overview, laboratory experiments, and feedbacks from industrial companies, we supported the use of spatial augmented reality (SAR), designing a prototype intended to be used for manual working stations of the future smart factories. This work presents the evaluation of the effectiveness of conveying technical instructions with this SAR prototype as compared to paper manual. We run a within-subjects experiment with 16 participants to measure user task performance (completion times and error rates) and to collect subjective evaluation. We projected technical information on a motorbike engine during a seven-task maintenance procedure. Our results proved that SAR technology improves the operators’ performance with respect to a paper manual and that users well accept it. We found that SAR is more effective for difficult tasks than for simple ones and that the main advantage of SAR is related more to the reduction of error rates than to completion times. These results confirm the goodness of our design choices; then our prototype can be a valid candidate solution for a smart manufacturing application.

Keywords: Assembly | Industry 4.0 | Maintenance | Projection | Spatial augmented reality | Technical instructions

Abstract: This article explores what it takes to make interactive computer graphics and VR attractive as a promotional vehicle, from the points of view of tourism agencies and the tourists themselves. The authors exploited current VR and human-machine interface (HMI) technologies to develop an interactive, innovative, and attractive user experience called the Multisensory Apulia Touristic Experience (MATE). The MATE system implements a natural gesture-based interface and multisensory stimuli, including visuals, audio, smells, and climate effects.

Keywords: computer graphics | gesture controls | human-machine interface | multisensory virtual environment | natural user interfaces

Abstract: Text legibility in augmented reality with optical see-through displays can be challenging due to the interaction with the texture on the background. Literature presents several approaches to predict legibility of text superimposed over a specific image, but their validation with an AR display and with images taken from the industrial domain is scarce. In this work, we propose novel indices extracted from the background images, displayed on an LCD screen, and we compare them with those proposed in literature designing a specific user test. We collected the legibility user ratings by displaying white text over 13 industrial background images to 19 subjects using an optical see-through head-worn display. We found that most of the proposed indices have a significant correlation with user ratings. The main result of this work is that some of the novel indices proposed had a better correlation than those used before in the literature to predict legibility. Our results prove that industrial AR developers can effectively predict text legibility by simply running image analysis on the background image.

Abstract: Due to the Industry 4.0 initiative, Augmented Reality (AR) has started to be considered one of the most interesting technologies companies should invest in, especially to improve their maintenance services. Several technological limitations have prevented AR to become an effective industrial tool in the past. Now some of them have been overcome, some others not yet by off-the-shelf technologies. In this paper, we present a solution for remote maintenance based on off-the-shelf mobile and AR technologies. The architecture of the application allows us to remotely connect a skilled operator in a control room with an unskilled one located where the maintenance task has to be performed. This application, which has been initially described in a previous work, has been improved on the basis of feedback received by industrial partners. We describe the important features we have added and the rationale behind them to make the remote communication more effective.

Keywords: Augmented Reality | Industry 4.0 | Remote Maintenance

Abstract: In this work, we proposed a method to reduce text in technical documentation, aiming at Augmented Reality manuals, where text must be reduced as much as possible. In fact, most of technical information is conveyed through other means such as CAD models, graphic signs, images, etc. The method classifies technical instructions into two categories: instructions that can be presented with graphic symbols and instructions that should be presented with text. It is based on the analysis of the action verbs used in the instruction, and makes use of ASD Simplified Technical English (STE) for remaining text instructions and let them easier to translate into other languages.

Keywords: Augmented Reality | Graphic symbols | Industry 4.0 | Simplified Technical English | Technical Documentation | Text reduction | Visual

Abstract: We present the design and a prototype of a projective AR workbench for an effective use of the AR in industrial applications, in particular for Manual Working Stations. The proposed solution consists of an aluminum structure that holds a projector and a camera that is intended to be mounted on manual working stations. The camera, using a tracking algorithm, computes in real time the position and orientation of the object while the projector displays the information always in the desired position. We also designed and implemented the data structure of a database for the managing of AR instructions, and we were able to access this information interactively from our application.

Abstract: Augmented Reality is a promising technology for the product lifecycle development, but it is still not established in industrial facilities. The most relevant issues to be addressed relate to the ergonomics: avoid the discomfort of Head-Worn Displays, allow the operators to have free hands and improve data visualization. In this work we study the possibility to use projection-based Augmented Reality (projected AR), as optimal solution for technical visualization on industrial workbenches. In particular, text legibility in projected AR is difficult to optimize since it is affected by many parameters: environment conditions, text style, material and shape of the target surface. This problem is poorly addressed in literature and in the specific industrial field. We analyze the legibility of a set of colors prescribed by international standards for the industrial environments, on six widely used industrial workbenches surfaces. We compared the performance of 14 subjects using projected AR, with that using a traditional LCD monitor. We collected about 2500 measurements (times and errors) through the use of a test application, followed by qualitative interviews. The results showed that, as regards legibility, projected AR can be used in place of traditional monitors in most of the cases. Another not trivial finding is that the influence on legibility of surface irregularities (e.g., grooves, prominences) is more important than that of surface texturization. A possible limitation for the use of projected AR is given by the blue color, whose performance turned out to be lower than that of other colors with every workbench surface.

Keywords: Industrial applications | Spatial Augmented Reality | Text legibility | Visualization

Abstract: Text readability with augmented reality head-worn displays is critical and at present time, there are no standard guidelines to follow. The readability depends mainly on background lighting, display technology (i.e., OST: optical see-through or VST: video see-through), and text style (e.g., plain text, outline or billboard). In this work, we addressed the readability limits for industrial activities. We experimented the effects of two background illuminances levels (1,000 lx for very fine basic industrial tasks and 4,000 lx for fine machining), two commercially available head-worn display technologies, variable outline widths and contrast polarity of text. We analyzed the performance of 12 subjects by collecting about 3,400 measurements using a specific test application and followed by qualitative interviews. With high illuminances, VST performed better than OST, regardless of contrast polarity and outline width. We found that negative contrast polarity is preferable with VST, and that just a minimum outline (1 px) around black text is optimal. On the contrary, positive contrast polarity should be used with OST and outline is not effective. Therefore, we evaluated the usage limits of the OST by sampling its contrast sensitivity function.

Keywords: Augmented reality | Contrast sensitivity function | Industrial lighting | Optical see-through | Video see-through

Abstract: In the Industrie 4.0 vision, the creation of leading-edge options for interaction between people and technology occupies a key role. In this context, augmented reality (AR) is one of the most suitable solutions. However, it is still not ready to be effectively used in industry. A crucial problem is the legibility of text seen through AR head-worn displays (HWDs). AR interface designers have no standard guidelines to follow, especially for these devices. Literature and anecdotal evidence suggest that legibility depends mainly on background, display technology (that is, see-through optical or video HWDs), and text style (for example, plain text, outline, or billboard). Furthermore, there are constraints to consider in industrial environments, such as standard color-coding practices and workplace lighting. The authors examine aspects affecting text legibility with an emphasis on deriving guidelines to support AR interface designers. Their results suggest that enhancing text contrast via software, along with using the outline or billboard style, is an effective practice to improve legibility in many situations. If one text style is needed for both types of HWD, their results suggest that colored billboards (with neutral white text) are effective. When color coding is not mandatory, white text and blue billboard are more effective than other styles tested.

Keywords: augmented reality | computer graphics | head-worn displays | human-computer interface | Industrie 4.0 | mixed reality | style guides | usability engineering | vision I/O

Abstract: We present an empirical study that evaluates the effectiveness of technical maintenance assisted with interactive augmented reality instructions. Our approach consists in an augmented visualization on a large screen and a combination of multiple fixed and mobile cameras. We used commercially available solutions. In our test, 14 participants completed a set of 4 maintenance tasks based on manual inspections of a motorbike engine. Tool selection, removal of bolts, and part dis\assembly, are supported by visual labels, 3D virtual models and 3D animations. All participants executed similar operations in two modalities: paper manuals and augmented instructions. Statistical analyses proved that augmented instructions reduced significantly participants' overall execution time and error rate. © 2013 Elsevier B.V. All rights reserved.

Keywords: Augmented reality | Computer aided task guidance | Large screen instruction | Maintenance

Abstract: Efficient text visualization in head-worn augmented reality (AR) displays is critical because it is sensitive to display technology, text style and color, ambient illumination and so on. The main problem for the developer is to know the optimal text style for the specific display and for applications where color coding must be strictly followed because it is regulated by laws or internal practices. In this work, we experimented the effects on readability of two head-worn devices (optical and video see-through), two backgrounds (light and dark), five colors (white, black, red, green, and blue), and two text styles (plain text and billboarded text). Font type and size were kept constant. We measured the performance of 15 subjects by collecting about 5,000 measurements using a specific test application and followed by qualitative interviews. Readability turned out to be quicker on the optical see-through device. For the video see-through device, background affects readability only in case of text without billboard. Finally, our tests suggest that a good combination for indoor augmented reality applications, regardless of device and background, could be white text and blue billboard, while a mandatory color should be displayed as billboard with a white text message. © 1995-2012 IEEE.

Keywords: Augmented reality | optical see-through | style guides | video see-through | vision I/O