May I take your order?
They say the eyes are the window to the soul, but what if you just want your takeaway food order understood from behind a face mask?
Lockdowns may come, and lockdowns may go. But face masks will be with us for a long time yet.
I don’t mind wearing a face mask. While inconvenient, it seems a small price to pay for reducing, if not eliminating, the impact of a viral pandemic. Those who refuse to wear a face mask because it’s not so much a piece of cloth, but a ball and chain preventing their flight to freedom, are morons.
Whatever your opinion, you need to learn to love your facemask, because it could well be the closest thing to your skin for some time yet. Given this, let’s put aside all those fruitless ‘new normal’ discussions around the future of work, and address a more fundamental consequence of the pandemic, and its demand that we now wear a face mask.
Namely, how the hell are we going to understand each other when we must wear a mask everywhere we go?
Cod? Haddock? Turnip?
Ordering fish and chips from a takeaway the other night was a painful experience, involving much confusion on the part of myself and the nice woman who served me. Both wearing face masks, her ensconced behind transparent plastic curtains, neither of us could understand what the other was saying. I have no idea what I actually ate later on, but it was battered.
When we engage with someone, we are encouraged to look in their eyes. And while inspecting someone’s eyes may be important in gauging true intention, seeing them in isolation, with a mask covering the rest of the face, makes discernment tricky, to say the least.
Worse, as both myself and the fish shop attendant who was trying to serve me grew ever more confused, our eyes lost any ability to support what we were trying to say, and simply expressed growing frustration, which provided a further impediment to communication.
<Confused face emoji>
Seeing someone’s entire face when communicating is important. Mouths are extremely expressive. A smile is a highly nuanced construction and can speak volumes, as a certain Mona Lisa would confirm. You may feel you can tell what’s going on in someone’s mind from their eyes alone, but it’s hard to order dinner through your eyes unless it’s one blink for cod, two for haddock, and seventeen for whatever the hell I ended up with the other night.
To help make my case let me offer up the emoji, an icon designed specifically to convey emotion simply and easily, without having to resort to all those messy words and their potential for misinterpretation. Look at the top half of these two emojis, selected because they represent polar opposites: the ‘glass half full’, and the ‘glass half empty’ of the emoji world. The eyes are literally the same in each. It’s the mouth that says everything.
I know what you’re saying, and it concerns the highly unscientific use of emojis to argue my case. Even my regular reminder to you that these posts are pretty much a science-free zone won’t help. I know you’re thinking there has to be more to communication than simplistic facial expressions.
“I second that valence and arousal”
But is there? I recently stumbled across a blog that shows how Facial Expression Recognition (FER) is possible by referencing a dataset containing 35,888 images of faces. It guides the reader in using a convolutional neural network to call on these images and use them as the basis of a machine learning exercise. Ultimately, it allows a computer to identify one of seven emotions in pictures of faces found online.
While there is suddenly much science at work here, even my blissfully-calculator-free education fannying about in the humanities doesn’t prevent me from identifying the problem with this model. It identifies seven — YES, SEVEN — emotions.
Or, in scientific parlance, it classifies each image according to whether they represent one of only seven states of ‘valence and arousal’, (the word ‘emotion’ apparently being too human a concept for the creators of our brave new world to deal with.)
The system, therefore, identifies ‘anger’, ‘disgust’, ‘fear’, ‘happiness’, ‘sadness’ and ‘surprise’. Oh, and ‘neutral’. Only computer scientists could reduce the wealth of human emotional experience to seven states of being. Sorry, ‘valance and arousal’.
That’s the problem with the development of technology: those that create it do so from their limited experience of life and design it largely for themselves. Maybe seven does indeed cover the entire gamut of emotions for computer scientists? Wait, that can’t right. What about mistaken superiority, social indifference, reductive obsessiveness, poor diet, lack of hygiene, the hoarding of toys designed for children, and an unhealthy obsession with the Star Wars chronology? That’s 14 by my reckoning.
When I trapped my thumb in a car door this morning a passer-by would have seen anger, fear, disgust, sadness and surprise at the same time in my unmasked face. How a computer would have decided which one was dominant, I have no idea. Happiness certainly wasn’t there, so that would have made the computation slightly easier. Although there was also pain, frustration and resignation. And not one of these is what I would term ‘neutral.’
And, coming to the real point, the same passer-by would have detected these states of emotions not from my eyes alone, but my face as a whole. Also from my gait, my demeanour, and the way I danced in the middle of a busy road like a James Brown angel dust-and-automotive-offences-years tribute act. In flip flops. They would have also heard some of the choicest swear words the moment could offer up.
I’d wager pretty much all of this extraneous detail would have escaped our computational neural network. But how much would have also escaped the same passerby if I had been wearing a face mask? I could have simply looked drunk at 8:15 on a Saturday morning.
The ayes have it
No, if we are going to be wearing them for some time yet, we are going to have to get used to communicating better behind our face masks. Our eyes only tell so much of the story, and our faces, which do so much in delivering the message we intend to convey, are unhelpful when hidden behind a mask.
The tech solution would, of course, involve ordering everything from an app on our phones, which I have to admit has its attractions. In the UK, where customer service still languishes in some feudal state of resentment, the idea that a transactional process could be automated — and so avoid any human interaction — is quite appealing.
But outsourcing human interaction to technology is a slippery slope. If ordering fish and chips, perhaps the most mundane of human interactions, is to be automated, then what use will we have for communications skills, at all? Who will need the learned ability to persuade, cajole, inspire and convince, when life is reduced to commanding droids to clean your home, navigating apps to order food, or feigning interest on a zoom call with similarly communications-challenged, home-office-tanned human beings?
According to the blogger who outlined his investigations in emotional analysis, the computer returned a ‘neutral’ verdict too often for his liking, seemingly unable to locate the happiness behind an otherwise empty facial expression. Good. Life at the moment is a challenge; so is effective communication. Like wearing a face mask, we will just have to get used to it and hope we get better at it.
For once, there is absolutely not an ‘app for that’. Tech cannot provide all the answers, thank God. Although I can’t see emojis going away any time soon.