Individual recourse for Black Box Models

Explained intuitively by Patrick Altmeyer (Finance ’18, Data Science ’21) through a tale of cats and dogs

Is artificial intelligence (AI) trustworthy? If, like me, you have recently been gobsmacked by the Netflix documentary Coded Bias, then you were probably quick to answer that question with a definite “no”. The show documents the efforts of a group of researchers headed by Joy Buolamwini, that aims to inform the public about the dangers of AI.

One particular place where AI has already wreaked havoc is automated decision making. While automation is intended to liberate decision making processes of human biases and judgment error, it all too often simply encodes these flaws, which at times leads to systematic discrimination of individuals. In the eyes of Cathy O’Neil, another researcher appearing on Coded Bias, this is even more problematic than discrimation through human decision makers because “You cannot appeal to [algorithms]. They do not listen. Nor do they bend.” What Cathy is referring to here is the fact that individuals who are at the mercy of automated decision making systems usually lack the necessary means to challenge the outcome that the system has determined for them. 

In my recent post on Towards Data Science,  I look at a novel algorithmic solution to this problem. The post is based primarily on a paper by Joshi et al. (2019) in which the authors develop a simple, but ingenious idea: instead of concerning ourselves with interpretability of black-box decision making systems (DMS), how about just providing individuals with actionable recourse to revise undesirable outcomes? Suppose for example that you have been rejected from your dream job, because an automated DMS has decided that you do not meet the shortlisting criteria for the position. Instead of receiving a standard rejection email, would it not be more helpful to be provided with a tailored set of actions you can take in order to be more successful on your next attempt? 

The methodology proposed by Joshi et al. (2019) and termed REVISE is an attempt to put this idea into practice. For my post I chose a more light-hearted topic than job rejections to illustrate the approach. In particular, I demonstrate how REVISE can be used to provide individual recourse to Kitty 🐱, a young cat that identifies as a dog. Based on information about her long tail and short overall height, a linear classifier has decided to label Kitty as a cat along with all the other cats that share similar attributes (Figure below). REVISE sends Kitty on the shortest possible route to being classified as a dog 🐶 . She just needs to grow a few inches and fold up her tail (Figure below).

The following summary should give you some flavour of how the algorithm works:

  1. Initialise x, that is the attributes that will be revised recursively. Kitty’s original attributes seem like a reasonable place to start.
  2. Through gradient descent recursively revise x until g(x*)=🐶. At this point the descent terminates since for these revised attributes the classifier labels Kitty as a dog.
  3. Return x*-x, that is the individual recourse for Kitty.
Animation illustrates how Kitty crosses the decision boundary
The simplified REVISE algorithm in action: how Kitty crosses the decision boundary by changing her attributes. Regularisation with respect to the distance penalty increases from top left to bottom right. Image by author.

This illustrative example is of course a bit silly and should not detract from the fact that the potential real-world use cases of the algorithm are serious and reach many domains. The work by Joshi et al. adds to a growing body of literature that aims to make AI more trustworthy and transparent. This will be decisive in applications of AI to domains like Economics, Finance and Public Policy, where decision makers and individuals rightfully insist on model interpretability and explainability. 

Further reading

The article was featured on TDS’ Editor’s Picks and has been added to their Model Interpretability column. This link takes you straight to the publication. Readers with an appetite for technical details around the implementation of stochastic gradient descent and the REVISE algorithm in R may also want to have a look at the original publication on my personal blog.

Connect with the author

portrait


Following his first Master’s at Barcelona GSE (Finance Program), Patrick Altmeyer worked as an economist for the Bank of England for two years. He is currently finishing up the Master’s in Data Science at Barcelona GSE.

Upon graduation Patrick will remain in academia to pursue a PhD in Trustworthy Artificial Intelligence at Delft University of Technology.

How can we rethink our economy for a more sustainable future?

Nils Handler ’18 presents the D\carb Future Economy Forum

I recently founded the D\carb Future Economy Forum with the goal of better informing the public debate on climate change on topics such as green growth, green macroeconomics and green innovation.

D\carb is strongly inspired by my Master’s studies at Barcelona GSE such as Antonio Ciccone’s class on Economic Growth and Albert Bravo-Biosca’s course on innovation policy.

Last week we held our virtual kick-off event, “Green Growth: Technological Innovation, Market Incentives and Investments for a Green Economy” to discuss the opportunities and risks of transitioning our economy into a sustainable future.

Our speakers were Prof. Ottmar Edenhofer, Director and Chief Economist of the Potsdam Institute for Climate Impact Research, and Prof. Cameron Hepburn, Director of the Economics of Sustainability Programme and Professor of Environmental Economics at the University of Oxford. Johanna Schiele, McCloy-fellow at the Harvard Kennedy School, moderated the event.

The event was organized jointly with the Mercator Research Institute on Global Commons and Climate Change and the Sustainability Centre of the Hertie School of Governance.

Upcoming events and more information about these topics is available on our website:

Visit the D\carb Future Economy Forum website

Connect with the author

Nils Handler ’18 is a PhD Student at DIW Berlin. He is an alum of the Barcelona GSE Master’s in International Trade, Finance, and Development.

How we used Bayesian models to balance customer experience and courier earnings at Glovo

Javier Mas Adell ’17 (Data Science)

Neon sign depicts Bayes' Theorem

Glovo is a three-sided marketplace composed of couriers, customers, and partners. Balancing the interests of all sides of our platform is at the core of most strategic decisions taken at Glovo. To balance those interests optimally, we need to understand quantitatively the relationship between the main KPIs that represent the interests of each side.

I recently published an article on Glovo’s Engineering blog where I explain how we used Bayesian modeling to help us tackle the modeling problems we were facing due to the inherent heterogeneity and volatility of Glovo’s operations. The example in the article talks about balancing interests on two of the three sides of our marketplace: the customer experience and courier earnings.

The skillset I developed during the Barcelona GSE Master’s in Data Science is what’s enabled me to do work like this that requires knowledge of machine learning and other fields like Bayesian statistics and optimization.

Read the full article on Glovo’s Engineering blog

Connect with the author

portrait

Javier Mas Adell ’17 is Lead Data Scientist at Kannact. He is an alum of the Barcelona GSE Master’s in Data Science.

International cooperation on carbon pricing

OECD working paper by Sneha Thube ’16 (Economics)

Photo by Markus Spiske on Pexels

As we are approaching the COP26 meeting to be held in Glasgow later this year, a highly anticipated milestone that is to be expected is the finalization of the rulebook for Article 6 of the Paris Agreement. Article 6 calls for ‘voluntary cooperation’ between public and private actors in carbon markets and other forms of international cooperation to meet the climate goals.

Ex-ante policy modelling assessments have shown that international cooperation on carbon pricing can result in economic and environmental gains that potentially could be used to boost the ambition of the climate targets. In our OECD working paper (jointly with Sonja Peterson, Daniel Nachtigall and Jane Ellis) we present a review of the literature on ex-ante policy modelling studies that examine the economic and environmental gains that could be realised if nations cooperate on climate action. Ex-ante modelling studies usually use Computable General Equilibrium (CGE) models or Integrated Assessment Models (IAM) to understand the socio-economic and environmental impacts of climate policies. We group the research articles into the following five types of cooperative actions that could be realised between countries – carbon price harmonization, extending the coverage of carbon pricing systems, implementing a multilateral fossil fuel subsidy reform, establishing international sectoral agreements and, mitigating carbon-leakage through strategic climate coalitions and border carbon adjustment.

The literature shows that all forms of international cooperation could potentially deliver economic and environmental benefits. Extending carbon markets to include new regions would reduce the aggregate mitigation costs but would not lead to unanimous gains for each of the participating countries and thus compensation mechanisms would be needed to incentivize participation from countries that would face costs. Sectoral agreements have a limited impact but could help in the reduction of GHG emissions though not cost-effectively. All of the studies unambiguously show that removal of fossil fuel subsidies would lead to an improvement in aggregate global welfare.

Further details about the results and individual papers can be found here:

OECD iLibrary

Connect with the author

portrait

Sneha Thube ’16 is a researcher at the Kiel Institute for World Economy. She is an alum of the Barcelona GSE Master’s in Economics.

The Poor and the Rich: Preferences over Inflation and Unemployment

Paper in the Journal of Money, Credit and Banking by José Nicolás Rosas ’20 (Economics) and Marc Hofstetter

Photo by olia danilevich on Pexels

Our paper “The Poor and the Rich: Preferences over Inflation and Unemployment”, written jointly with Marc Hofstetter, has now been published in the Journal of Money, Credit and Banking (JMCB). Here is a summary of our work:

Strong Central Bank’s anti-inflationary postures are often viewed as a way to implement policies consistent with the preferences of the poor. Five examples:

  1. Mankiw (2006): inflation “is not a tax on all assets but only on non-interest-bearing assets, such as cash. The rich are able to keep most of their wealth in forms that can avoid the inflation tax”.
  2. FED Kansas City President, George (2017): “not as enthusiastic or encouraged as some when I see inflation moving higher,” because “inflation is a tax and those least able to afford it generally suffer the most.”
  3. Cœuré, ECB (2012): “inflation is also particularly harmful to the poorest parts of the population”; “poorer households tend to hold a larger fraction of their financial wealth in cash, implying that both expected and unexpected increases in inflation make them even poorer.”
  4. Central Bank of Colombia: low & stable inflation is important because “increasing inflation means a redistribution of income against the poor.” 
  5. Central Bank of Chile: inflation tends to hurt those who have a greater proportion of their wealth in money, that is, the poorest.

But do the poor prefer stronger anti-inflationary policies than the rich? 

This is not obvious: anti-inflationary policies often come at the cost of less economic activity and higher unemployment rates, and these side effects of contractionary monetary policies are not necessarily evenly spread across the income distribution. 

Accordingly, preferences vis-à-vis inflation versus unemployment might also not be evenly distributed across income groups. We study these relative preferences across the income distribution.

We find that:

  1. Both the poor and the rich dislike inflation and unemployment and they both dislike extra points of unemployment more than extra points of inflation. 
  2. The aversion to unemployment relative to inflation is higher in Latin America than in Europe.
  3. Our main point: the poor have a higher aversion to unemployment relative to inflation than the rich. This finding is at odds with the commonly held view by Central Banks that hawkish monetary policies line up with the poor’s preferences.

The idea that a compassionate Central Bank should fight inflation strongly notwithstanding the consequences on unemployment is at odds with the preferences along the income distribution estimated in our paper.

Read the paper in JMCB

References

José Nicolás Rosas G. ’20 is an MRes/PhD student at UPF and Barcelona GSE. He is an alum of the Barcelona GSE Master’s in Economics.

New evidence of granular business cycles from German cities

Federica Daniele ’13 shares a paper accepted to Review of Economics and Statistics.

journal cover

My paper “The Micro-Origins of Business Cycles: Evidence from German Metropolitan Areas” joint with Heiko Stueber has been accepted to the Review of Economics and Statistics. Here is a summary of our work:

Cities compete to attract large firms. When Amazon announced in 2017 the opening of its second headquarters, 238 US cities signed up for it. Large firms bring jobs and can boost local productivity through spillovers. However, the downside is that they generate excessive local volatility.

We leverage quarterly data on size of all German establishments from 1990 to 2014 to show that a buildup of concentration of economic activity in the hands of few sizable firms is systematically associated with higher volatility in local labor markets in subsequent months.

The reason is granularity. When concentration is high, shocks to large firms do not average out with shocks to smaller ones and the evolution of local employment ends up mimicking the evolution of employment in the large firm. The economy experiences “granular” business cycles.

Our paper is the first to provide solid time-series support to granular business cycles as in Carvalho and Grassi. However, we show that large firms do not seem capable to trigger both booms and busts alike. Our evidence points in favor of granularity-driven recessions only.

Finally, we calibrate the parameters governing local firm dynamics to match the local employment law of motion, because we want to see what are the causes of the disproportionate presence of large firms in big cities. We find that it’s because of higher growth opportunities in big cities.

Bottom line: the volatility externality imposed by large firms encourages short-time work schemes as opposed to layoffs and may justify using size-dependent forms of public support for crisis management, but the benefit might have to be weighed against potential moral hazard.

Read abstract and download from MIT Press

Connect with the author

portrait

Federica Daniele ’13 is an economist at the Bank of Italy. She holds a PhD from UPF and Barcelona GSE and is an alum of the Barcelona GSE Master’s in Economics.

Tackling domestic violence using large-scale empirical analysis

New paper in Journal of Empirical Legal Studies co-authored by Ria Ivandić ’13 (Economics)

A woman holds a sign in front of her face that reads, "Love shouldn't hurt."
Photo by Anete Lusina from Pexels

In England, domestic violence accounts for one-third of all assaults involving injury. A crucial part of tackling this abuse is risk assessment – determining what level of danger someone may be in so that they can receive the appropriate help as quickly as possible. It also helps to set priorities for police resources in responding to domestic abuse calls in times when their resources are severely constrained. In this research, we asked how we can improve on existing risk assessment, a research question that arose from discussions with policy makers who questioned the lack of systematic evidence on this.

Currently, the risk assessment is done through a standardised list of questions – the so-called DASH form (Domestic Abuse, Stalking and Harassment and Honour- Based Violence) – which consists of 27 questions that are used to categorise a case as standard, medium or high risk. The resulting DASH risk scores have limited power in predicting which cases will result in violence in the future.  Following this research, we suggest that a two-part procedure would do better both in prioritising calls for service and in providing protective resources to victims with the greatest need. 

In our predictive models, we use individual-level records on domestic abuse calls, crimes, victim and perpetrator data from the Greater Manchester Police to construct the criminal and domestic abuse history variables of the victim and perpetrator. We combine this with DASH questionnaire data in order to forecast reported violent recidivism for victim-perpetrator pairs.  Our predictive models are random forests, which are a machine-learning method consisting of a large number of classification trees that individually classify each observation as a predicted failure or non-failure. Importantly, we take the different costs of misclassification into account.  Predicting no recidivism when it actually happens (a false negative) is far worse in terms of social costs than predicting recidivism when it does not happen (a false positive). While we set the cost of incurring a false negative versus a false positive at 10:1, this is a parameter that can be adjusted by stakeholders. 

We show that machine-learning methods are far more effective at assessing which victims of domestic violence are most at risk of further abuse than conventional risk assessments. The random forest model based on the criminal history variables together with the DASH responses significantly outperforms the models based on DASH alone. The negative prediction error – that is, the share of cases that would be predicted not to have violence yet violence occurs in the future – is low at 6.3% as compared with an officer’s DASH risk score alone where the negative prediction error is 11.5%.  We also examine how much each feature contributes to the model performance. There is no single feature that clearly outranks all others in importance, but it is the combination of a wide variety of predictors, each contributing their own ‘insight’, which makes the model so powerful.

Following this research, we have been in discussion with police forces across the United Kingdom and policy makers working on the Domestic Abuse Bill to think how our findings could be incorporated in the response to domestic abuse. We hope this research acts as a building block to increasing the use of administrative datasets and empirical analysis to improve domestic violence prevention.

This post is based on the following article:

Grogger, J., Gupta, S., Ivandic, R. and Kirchmaier, T. (2021), Comparing Conventional and Machine-Learning Approaches to Risk Assessment in Domestic Abuse Cases. Journal of Empirical Legal Studies, 18: 90-130. https://doi.org/10.1111/jels.12276 

View abstract in Journal of Empirical Legal Studies

Media coverage

Connect with the author

Ria Ivandić ’13 is a Researcher at LSE’s Centre for Economic Performance (CEP). She is an alum of the Barcelona GSE Master’s in Economics.

Stop dropping outliers, or you might miss the next Messi!

Jakob Poerschmann ’21 explains how to teach your regression the distinction between relevant outliers and irrelevant noise

Jakob Poerschmann ’21 (Data Science) has written an article called “Stop Dropping Outliers! 3 Upgrades That Prepare Your Linear Regression For The Real World” that was recently posted on Towards Data Science.

The real world example he uses to set up the piece will resonate with every fan of FC Barcelona (and probably scare them, too):

You are working as a Data Scientist for the FC Barcelona and took on the task of building a model that predicts the value increase of young talent over the next 2, 5, and 10 years. You might want to regress the value over some meaningful metrics such as the assists or goals scored. Some might now apply this standard procedure and drop the most severe outliers from the dataset. While your model might predict decently on average, it will unfortunately never understand what makes a Messi (because you dropped Messi with all the other “outliers”).

The idea of dropping or replacing outliers in regression problems comes from the fact that simple linear regression is comparably prone to extremes in the data. However, this approach would not have helped you much in your role as Barcelona’s Data Scientist. The simple message: Outliers are not always bad!

Dig into the full article to find out how to prepare your linear regression for the real world and avoid a tragedy like this one!

Read “Stop dropping outliers” on Towards Data Science

Connect with the author

portrait

Jakob Poerschmann ’21 is student in the Barcelona GSE Master’s in Data Science.

Two alumni launch Mont^2, a collaborative research lab

New research venture created by Francesco Amodio (Econ ’10), Giorgio Chiovelli (Econ ’11) and Serafín Frache

A pair of Barcelona GSE Alumni and their frequent co-author and friend have recently launched a new research centre to provide a platform for their vision of ideal research collaboration and to bring those learnings to a wider audience. This venture is Mont^2, the Montréal x Montevideo Econ Lab.

This initiative was not without its challenges despite appearing to be an easy path for a group of friends and co-authors from the outside. Its genesis happened just before the COVID-19 pandemic struck both Canada and Uruguay, where the founders are based (not to mention the rest of the world) forcing them to adapt their plans for the launch of a physical working group to an online one.

Francesco Amodio ’10 and Giorgio Chiovelli ’11 are Economics Program alumni and became firm friends after meeting as TA and student in an econometrics class. They began collaborating during their respective PhD’s, Francesco at UPF and Giorgio at Bologna in Italy. After graduating and starting their careers in Montréal and Montevideo respectively, they included a third member into their collaborative efforts, Serafín Frache, and started laying the groundwork for what would ultimately become Mont^2. Serafín had local knowledge of Uruguayan administrative data and its potential to answer exciting economic questions. From these roots, the three researchers began thinking about their long-term career plans and how they can make an impact on their communities and give back to their respective local communities and the wider academic and policy-world.

It is with this foundation that Mont^2 was created. The professors applied to Social Sciences and Humanities Research Council (SSHRC) for seed funding to utilise the unique Uruguayan data and begin building the infrastructure of Mont^2.

They also aim for the lab to structure the mentorship of the professors’ current and future pre-doctoral research assistants. This would give them the tools to work with big data and be prepared for their future careers where this skill is in demand, whether in academia or the private sector. The trio also want to bring attention to the role academic research has to play with policy-making institutions regardless of where they might be located.

Mont^2 has just been launched, but already they are hard at work on a handful of projects with RAs already enlisted. It is a working environment meant to provide a formalised structure to the growing network of researchers and collaborators that began with Francesco, Giorgio and Serafín but now stretching far beyond. The hope is for Mont^2 to strengthen their ties with policy institutions and begin to promote best practices when dealing with confidential government big data.

Connect with the founders

Francesco Amodio ’10 is an Associate Professor, McGill University. He is a co-founder of Mont^2 and an alum of the Barcelona GSE Master’s in Economics.

Giorgio Chiovelli ’11 is an Assistant Professor, Universidad de Montevideo. He is a co-founder of Mont^2 and an alum of the Barcelona GSE Master’s in Economics.

Serafín Frache is an Assistant Professor, Universidad de Montevideo. He is a co-founder of Mont^2.

This post was written by Maximilian Magnacca Sancho ’21 (ITFD) and edited by Ashok Manandhar ’21 (Economics).

Data Science team “Non-Juvenile Asymptotics” wins 3rd prize in annual Novartis Datathon

Patrick Altmeyer, Eduard Gimenez, Simon Neumeyer and Jakob Poerschmann ’21 competed against 57 teams from 14 countries.

Screenshot of team members on videoconference
Members of the “Non-Juvenile Asymptotics” Eduard Gimenez, Patrick Altmeyer, Simon Neumeyer and Jakob Poerschmann, all Barcelona GSE Data Science Class of 2021

The Novartis Datathon is a Data Science competition taking place annually, usually in Barcelona. In 2020, the Barcelona GSE team “Non-Juvenile Asymptotics” consisting of Eduard Gimenez, Patrick Altmeyer, Simon Neumeyer and Jakob Poerschmann won third place after a fierce competition against 57 teams from 14 countries all over the globe. While the competition is usually hosted in Barcelona, the Covid-friendly version was fully remote. Nevertheless, the increased diversity of teams clearly made up for the missed out atmosphere.

This year’s challenge: predict the impact of generic drug market entry

The challenge of interest concerned predicting the impact of generic drug market entry. The risk of losing ground against cheaper drug replicates once the patent protection runs out is evident for pharmaceutical companies. The solutions developed helped solving exactly this problem, making drug development much easier to plan and calculate.

While the problem could have been tackled in various different ways, the Barcelona GSE team focused on initially developing a solid modeling framework. This represented a risky extra effort in the beginning. In fact more than half of the competition period passed without any forecast submission by the Barcelona GSE team. However, the initial effort clearly paid off: as soon as the obstacle was overcome, the “Non-Juvenile Asymptotics” were able to benchmark multiple models at rocket speed.

Fierce competition until the very last minute

The competition was a head-to-head race until the last minute. Still in first place until minutes before the final deadline, the predictions of two teams from Hungary and Spain ended up taking the lead by razor sharp margins.

Congratulations to the winners!!!

Group photo of the team outside the entrance of Universitat Pompeu Fabra
The team at Ciutadella Campus (UPF)

Connect with the team