Reinforcement Learning (RL) model for self-improving chatbots, specifically targeting FAQ-type chatbots.

Elena Ricciardelli, Debmalya Biswas

Abstract. We present a Reinforcement Learning (RL) model for self-improving chatbots, specifically targeting FAQ-type chatbots. The model is not aimed at building a dialog system from scratch, but to leverage data from user conversations to improve chatbot performance. At the core of our approach is a score model, which is trained to score chatbot utterance-response tuples based on user feedback. The scores predicted by this model are used as rewards for the RL agent. Policy learning takes place offline, thanks to an user simulator which is fed with utterances from the FAQ-database. Policy learning is implemented…

Spoiler Alert: Trained models can leak (properties of) training data

Data Privacy Photo by ThisIsEngineering from Pexels

Supervised Machine Learning (ML)

Most of today’s ML models are supervised and applied on a prediction / classification task. Given a dataset, the Data Scientist has to go through a laborious process called feature extraction and the model’s accuracy depends entirely upon the Data Scientist’s ability to pick the right feature set. For simplicity, each feature can be considered a column of a dataset provided as a CSV file.

Automated Classification of Facial Recognition Tiles

Isidora Stankovic, Nevena Milenkovic, Goran Benak, Miljan Vuletic, Nikola Milojevic, Debmalya Biswas — Darwin Edge, Switzerland

Abstract. Facial recognition platforms based on Computer Vision/Deep Learning techniques have considerably matured over the last few years. This has led to their sometimes controversial usage in surveillance use-cases. However, it has also enabled many healthcare use-cases, e.g., sentiment analysis, assessing (intensity of) pain based on facial expressions. In this work, we take a detailed look at the typical facial recognition inference pipeline. Tiles generated from face detectors are not always suitable for feature detection as they sometimes do not contain faces. …

Notes from Industry

Fusion of AI/ML Services, integrating DataOps, MLOps, APIOps

Abstract. Enterprise adoption of AI/ML services has significantly accelerated in the last few years. However, the majority of ML models are still developed with the goal of solving a single task, e.g., prediction, classification. In this work, we will present the emerging paradigm of Compositional AI, also known as Compositional Learning. Compositional AI envisions seamless composition of existing AI/ML services, to provide a new (composite) AI/ML service, capable of addressing complex multi-domain use-cases. In an enterprise context, this enables reuse, agility, and efficiency in development and maintenance efforts.

This is an extended article of a recent keynote that I gave…

Deploying AI/ML models on Edge Devices

(Darwin Edge, Switzerland) Miljan Vuletić, Vladimir Mujagić, Nikola Milojević, Debmalya Biswas

Edge AI (Image by Authors)

Abstract. Edge AI enables intelligent solutions to be deployed on edge devices, reducing latency, allowing offline execution, and providing strong privacy guarantees. Unfortunately, achieving efficient and accurate execution of AI algorithms on edge devices, with limited power and computational resources, raises several deployment challenges. Existing solutions are very specific to a hardware platform/vendor. In this work, we present the MATE framework that provides tools to (1) foster model-to-platform adaptations, (2) enable validation of the deployed models proving their alignment with the originals, and (3) empower engineers and architects to…

Challenges in deploying Edge AI applications

Fig: Edge AI deployment pipeline (Image by Author)


AI/ML use-cases are pervasive. The enterprise use-cases can be broadly categorized based on the three core technical capabilities enabling them: Predictive Analytics, Computer Vision (CV) and Natural Language Processing (NLP). The Enterprise AI story has so far been focused on the Cloud. The general perception is that it takes a large amount of data and powerful machines, e.g., Graphical Processing Units (GPUs), to run AI applications.

Edge AI, also known as TinyML, aims to bring all the goodness of AI to the device. The idea is to bring the processing as close as possible to the devices generating the data.

Real-world Enterprise use-cases of Reinforcement Learning: Recommendation systems, NLP/Chatbots, Energy optimization


With the current saturation setting into Deep Learning (DL) methods, there is quite a bit of expectation that Reinforcement (RL) will be the next big thing in AI.

Given that RL based approaches can basically be applied to any optimization problem, its enterprise adoption is picking up fast.

Figure 1. Reinforcement Learning: based on the pic by Andrea Piacquadio from Pexels

RL refers to a branch of Artificial Intelligence (AI), which is able to achieve complex goals by maximizing a reward function in real-time. The reward function works similar to incentivizing a child with candy and spankings, such that the algorithm is penalized when it takes a wrong decision and rewarded when it…

Statistical metrics to quantify mobile app privacy risk based on Location, Contacts & Content

Debmalya Biswas, Imad Aad, Gian Paolo Perrucci

Abstract. The ever increasing popularity of apps stems from their ability to provide highly customized services to the user. The flip side is that in order to provide such services, apps need access to very sensitive private information about the user. This leads to malicious apps that collect personal user information in the background and exploit it in various ways. Studies have shown that current app vetting processes which are mainly restricted to install time verification mechanisms are incapable of detecting and preventing such attacks. We argue that the missing fundamental aspect here…

Secure Multiparty Computation to the Rescue

Federated Learning: based on the pic by Andrea Piacquadio from Pexels

Federated learning [1], also known as Collaborative Learning, or Privacy preserving Machine Learning, enables multiple entities who do not trust each other (fully), to collaborate in training a Machine Learning (ML) model on their combined dataset; without actually sharing data — addressing critical issues such as privacy, access rights and access to heterogeneous confidential data.

This is in contrast to traditional (centralized) ML techniques where local datasets (belonging to different entities) need to be first brought to a common location before model training. …

Manage licensing risks of Open Source Data Science projects

This is an extended article accompanying the presentation on “Open Source Enterprise AI/ML Governance” at Linux Foundation’s Open Compliance Summit, Dec 2020 (link) (pptx)

Open Source Software Licensing: based on the pic by Andrea Piacquadio from Pexels

Open Source Software Enterprise Adoption Trends

The best AI/ML software today from model development (scikit-learn, TensorFlow, PyTorch) to deployment (Kubeflow, Spark) is Open Source. According to Gartner, “more than 90% of companies rely on Open Source Software”. The below snapshot should give you an idea of the pervasiveness of Open Source Software (OSS) in the Enterprise.

Debmalya Biswas

CTO, Darwin Edge | AI, Privacy and Open Source | Ex-Nokia, SAP, Oracle | 50+ Patents |

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store