Digital semeiotics: from CNR Wize Mirror, the “mirror” that reads the state of health on the face
To exploit the face as a major indicator of individual’s health and well-being is the central idea of the Wize Mirror device designed within SEMEOTICONS, a “digital semeiotic” project coordinated by the National Research Council and funded by the European Union under the Seventh Framework Programme. Promoted in collaboration with research institutions and companies, the initiative aims to develop an integrated system to assess, by mapping the human face, the risk for development of cardiovascular and metabolic diseases.
An advanced multi-sensor device is being implemented. It is similar to a mirror and records, analyses and interprets the signs of the human face in order to assess the risk for development of cardiovascular or metabolic diseases. This is Wize Mirror, an innovative tool for personalized monitoring of the state of health, developed under SEMEOTICONS, a European project under Italian leadership, coordinated by the National Research Council (CNR) through the Institute of Information Science and Technology (ISTI) and the Institute of Clinical Physiology (IFC). A number of partners collaborate to this project, including research centres and companies from France, Sweden, Norway, Greece, Spain, and the United Kingdom. Besides CNR, Italy is also represented by an industrial actor, Intecs, and a small/medium-sized entreprise, Cosmed.
Funded by the European Union under the Seventh Framework Programme, SEMEOTICONS, which includes Wize Mirror, aims to contribute to the digitalization of semeiotics, a medical discipline aimed to study signs of the body that lead to diagnosis. The main objectives of this R&D line include the improvement and personalization of prevention strategies, an important field where intervention is needed in order to limit the endemic spreading of diseases, such as cardiovascular and metabolic disorders. In fact, the individual life style may contribute to the development of these diseases through “risk” behaviours and conditions, including obesity or overweight, wrong dietary regimen, sedentary life style, smoking and alcohol abuse.
As the full name of the project suggests – Semeiotic Oriented Technology for Individual’s Cardiometabolic risk self-assessment and Self-monitoring – SEMEOTICONS intends to help people to assess and reduce their cardio-metabolic risk associated with an inappropriate life style. The heart of the project is Wize Mirror, an innovative multi-sensor system integrated into a hardware platform looking like a mirror that maps a number of relevant characteristics of the face that are associated with individual physical and emotional state. Data are collected mainly in the form of videos and images.In particular, the system acquires multi-spectrum images to assess skin perfusion and monitor skin accumulation of cholesterol and glycated end products. Deep sensors provide a 3D reconstruction of the face to study morphological changes and, through the acquisition of video sequences, the system assesses heart rate variability and measures any signs associated with negative psychological states, such as stress, anxiety and fatigue. Finally, the Mirror Wize Sniffer device analyses the composition of breath and exhaled gas to monitor the effect of harmful habits such as smoking and alcohol abuse.
Following a semeiotic approach, the face signs are mapped in computational measures and descriptors that are assessed automatically. The final objective is to integrate descriptors into a wellness index that describes the evolution of the individual’s state over time. Besides assessing the level of exposure to the risk of developing diseases, the system stimulates users to improve their life styles, by sending them personalized messages and suggestions, and to share their data, stored in a diary, with their physicians or other healthcare operators.
Started in 2013, the project has implemented the first Wize Mirror prototype – consisting of hardware and software platforms and full suite of sensors – to be used in R&D activities. The next step will be the integration of all the methods and procedures needed to calculate the wellness index. The final goal is the implementation of a device that will easily fit into users’ home or other sites of their daily lives (e.g. fitness and nutritional centres, pharmacies, schools and so on).