Blog

We developed this real time face recognition app as a research project, trying to combine CRM and AI. The use case involves sales and business developers in networking events (conference, trade show …). They usually have access to prospects and customers information through their company CRM but how can they look for a name if they don’t remember it in the first place? Typical «the tip of the tongue» situation: I know the face, but I can’t remember the name. This app uses top of the art face detection and recognition to find in real time the names of the persons in front of you.

As for the technical details, we have encapsulated a C++ library (dlib) into a Java android app. We used dlib’s HOG (Histogram of Oriented Gradients) face detection which is a very good “frontal” face detector and thus suitable for real time video. Then, we integrated the app with Salesforce from which we extracted pictures, names and ids. Pictures were translated into facial landmarks though a first neural network and then into 128 measurements for each face through a second neural network. The magic lies in the 128 measurements which brings us into a Euclidean space where distances correspond to a measure of face similarity. Such distances happen to be really easy to compute in real time.

Eventually, we decided not to turn this project into a commercial product. For one, some of the pre-trained models that we used are based on datasets whose licences exclude commercial use. Consequently, significant investment would be required, in addition to fine tuning and features development, just to recreate these models. There are also some legal challenges in storing face pictures and facial landmarks as these are often considered, and rightly so, as biometric data. Finally, apart from using Google glasses which is not really available anymore, it is actually quite creepy to hold the phone to capture people faces. In fact, you can’t even turn off the shutter sound on mobile phones in some countries like Japan, because of an anti-upskirt photos law!

These last two points are characteristics of the legal and ethical issues that face many AI initiatives. However, I believe that with time and practice, we’ll find answers to these challenges. What’s sure for now, is that AI has the potential to greatly enhance our capabilities.