Wits scientist discusses the art of innovation

Adversity leads to innovation, says iconic biomedical engineer Dr Adam Pantanowitz, a lecturer at the University of the Witwatersrand (Wits).

“Frugality can be seen as a downfall, a pitfall, but strife creates great art. When you have infinite budgets and infinite timing, you don’t actually end up producing anything. Frugality creates a precious scarcity of resources. Adversity creates pressure, and pressure creates novel solutions.”

Pantanowitz, who lectures in engineering and medicine, was speaking on 27 October at Wits’s Science, Society, and Innovation lecture series.

Innovation is about looking at a problem from a new perspective to take a small step forward, Pantanowitz said. “It’s about breaking down a problem to its simplest form, and then allowing a solution to bubble up.”

When Pantanowitz was studying at the Wits School of Electrical Information Engineering in 2008, they pondered the problem surgeons face in adjusting the overhead light. They knew the Nintendo Wii remote had a little infrared camera, so they decided to try to automate the surgery environment using infrared light.

“We went to the Magic Company, which runs big toy arcades in South Africa,” Pantanowitz said. “It has a huge warehouse with old broken versions of arcade machines. We were able to have donated one of the sets of motors that moves the toy crane around. We repurposed it to our Surgewii system [a surgery automation system which ultimately became a patent]. We were able to control a computer remotely by bouncing infrared light and using Wii remotes to control the environment. Then we had this overhead crane directing light where we needed it. Now, doctors could control the surgery environment without having to touch anything. It became a wireless type of setup.”

Technology can considerably improve the lives of people with disadvantages or disabilities, Pantanowitz said.

He and others used off-the-shelf, cheap consumer electronics to allow a wheelchair user to control their wheelchair by moving their eyes.

“Another example is that we used an off-the-shelf cricket helmet and PlayStation camera to try to detect the position of someone’s pupil and create a more natural human-computer interaction in which they are able to drag a computer mouse around just by using their eyes. This gives them a much more natural way of interacting and interfacing with a computer.”

Pantanowitz also helped create a device for a friend who had a brain implant due to Parkinson’s disease. “The device looks a bit like a mobile phone,” the friend said. “It’s the difference between me being able to talk and move, and not being able to. I have a brain pacemaker, if you like. This is my lifeline. Literally, you can turn me off completely.” He can bungee jump and skydive thanks to his brain implants.

“We used off-the-shelf technology for a CPR [cardiopulmonary resuscitation] project, in which we used an armband to see whether a person was practising CPR effectively,” Pantanowitz said. “This is about using technology to save lives. Hopefully, we’ll have a new generation of really well-equipped students who have come through the programmes using the armband.”

Pantanowitz and others turned into silhouettes the individual photos of 161 participants who took part in a study. “Taking a silhouette massively reduces the complexity of the image. Then, we were able to try to predict the person’s BMI [body-mass index] relative to their actual BMI for unseen test data. The problem with this study is that we used a lot of our data to train the network, and then we had few samples left to prove that what we had done was actually useful. We started to think about the principle of data protection. We manipulated a single image to allow us to amplify the data and get all sorts of examples of what different people may look like. With a single image, we were able to produce something like 150 examples.

“The underlying rationale of this is that it’s costly to go out and get that data. It’s time-consuming. It requires ethical considerations. We need to consider how and where the data will ultimately end up, whether we’re treating our data responsibly, and so on.”

People wore face masks differently during the pandemic, Pantanowitz said. “Some wore a face mask covering their mouth and the nose; others wore masks under their nose. Some people didn’t wear masks at all. We thought we could use computer vision and image processing to help a person get quick feedback about whether they were wearing their mask correctly just to raise their bar of awareness. But we were in lockdown. We couldn’t go out and get data about people wearing their masks poorly, not poorly, or not wearing masks at all.”

They generated a fake photorealistic face through thispersondoesnotexist.com, and used processing techniques to mask those faces, allowing them to produce their own dataset without ever having gone out and gathering data on real people.

Pantanowitz believes innovation is permissionless. “On reflection, we need to make sure we’re doing ethical things. When we wield all this magic and power, we have a responsibility to society. When we bring about a new technology and release it into the wild, so to speak, there are countless examples of second-order and third-order effects which technology brings about. With all that in mind, you’re in this amazing environment in the university in which you’re able to, without necessarily asking for permission, go and change how something works. That’s a powerful thought and a huge privilege to us all in working in these spaces.”