Bruno Trentini

Production ML · GSK · 2021–2022

Reinforcement learning for campaign allocation

A reinforcement-learning and Bayesian system for multi-objective allocation, implemented from data pipeline to production interface and evaluated in a live A/B test.

Problem

Campaign allocation involves sequential decisions under uncertain response. A static allocation can waste budget when channel performance changes, but an adaptive system must also present its recommendations clearly enough for operational use.

Method and system

I developed a reinforcement-learning and Bayesian approach to multi-objective allocation. The production implementation covered Spark and Databricks data pipelines, PyTorch models, service APIs, Azure hosting and an interactive web interface for the operating team.

observationsSpark / DBXpolicy +posteriorallocationAPI / UI
System loop: observations update the model; the service returns allocations; outcomes feed the next decision.

My role

I was responsible for the modelling and implementation, from data preparation to the user interface. I also explained the statistical methods to colleagues and helped the operating teams use the system.

Evaluation

The system was tested against two major industry platform providers. In the A/B test it reduced cost per completed view by 55%. The project was shortlisted in the top three at the DataIQ industry awards.

55%lower cost per completed view
A/Blive controlled evaluation
Top 3DataIQ industry award shortlist

Technical scope

  • Reinforcement learning and Bayesian statistics for sequential allocation.
  • Spark/Databricks ETL and PyTorch modelling.
  • Production APIs and Azure deployment.
  • Interactive HTML and JavaScript interface for operational users.