From Facial Expression to Diseases

Facial landmark detection can identify different facial features such as nose tips, mouth corners and face contours, which plays a critical role in face recognition. Prof Tim Cheng is taking this technology one step further by applying it to monitor facial expression and emotion. He and his team have developed a facial landmark detector that can perform 3D facial landmark detection consisting of 128 landmark points.  

Detected facial landmarks could be used to perform early diagnosis of certain diseases. Prof Cheng is partnering with Haven of Hope Christian Service to collect video data for target analysis from their local care centres for senior citizens with dementia. They have installed a number of cameras in the room where most activities were being carried out and have recorded footage for up to two months. In addition to performing facial landmark detection, they will also use computer vision technology to accurately detect the actions of the subjects. Besides clinically relevant actions including sitting, standing and walking, his team is also interested in the elderly’s interaction with other people. 

Another track of research analyses the intensity of facial expression. By analysing visual data, Prof Cheng and his team have learnt that dementia patients are far less expressive compared to the younger population. The majority of patients only show very subtle change in expression upon visual stimulation. As a result, it could be challenging to capture every small fraction of time when the elderly shows any change in expression. Using landmark detection technology, Prof Cheng’s team will analyse the change in facial expression of the target elderly. Instead of annotating the exact expression, they try to determine the expressiveness of the elderly and see if there is any reason behind it. This particular line of study is distinctive because, whilst most expression detection research focuses on building a model to predict emotions within large populations, Prof Cheng’s work seeks to monitor a particular subject over a long period of time and analyse the correlation between facial expressions and certain diseases. In future studies, this same set of valuable visual data collected from elderly centres will solicit other new lines of research such as whether certain kind of behaviours indicate an individual’s tendency to leave the current location. This could potentially help caregivers of dementia patients. Going forward, Prof Cheng’s team will observe the characteristics of certain patients to discover new patterns and behavioural insights.