Machine Learning Bias: What Is It, Why Is It Important, and What Can You Do About It?

In 2016, Microsoft tried to engage users by launching a machine learning-powered chatbot on Twitter. This initiative turned into a disaster when the chatbot started tweeting racist and insensitive statements—going as far as supporting Hitler’s inhumane ideology.

As in Microsoft's case, a machine learning model can send unintended, dangerous conversational responses. Train your models to avoid these outcomes by learning what machine learning bias is and how to minimize it.

What Is Machine Learning Bias?

Machine learning bias, also known as AI bias, refers to a phenomenon that occurs when an algorithm produces systematically biased outcomes. This systematic error is caused by faulty assumptions during the machine learning process.

In machine learning, models are fed training data that is similar to real-world scenarios, so the model can learn from it and act accordingly after it starts running without manual interference. In more technical terms, bias is the error between your model’s average predicted values and the actual values.

Bias can creep in during this training process due to a number of reasons. Here are the types of machine learning bias.

• Exclusion bias occurs when important data is kept out after it is deemed irrelevant. Say you have a dataset of customer sales in city A and city B. Your model might neglect city B if 97% of your sales come from city A. This choice would be an exclusion bias if the model ignored other relevant details, like customers in city B spending twice as much as those in city A.
• Sample bias occurs when a wrong sample is used for model training. This sample can be small in size, contain wrong data points, or fail to represent the whole data pool. Say you’re training facial recognition software to identify your users and only use the data samples of young people. Your facial recognition software is likely to make an error against aging adults and children.
• Algorithmic bias occurs when an algorithm’s design is faulty, skewing the model’s final outcome. For example, a study showed that Google’s algorithm doesn’t show high-paying job ads to women.
• Measurement bias occurs due to inaccurate data collection caused by faulty measurements. For instance, if you collect training data and production data with different cameras, it can affect your model’s accuracy.
• Prejudice bias occurs when training data is influenced by real-world stereotypes within the population—for example, an HR recruiting tool reviewing only female applications for nursing jobs due to the stereotype of women working as nurses.

Why Is Minimizing Machine Learning Bias Important?

People trust machine learning algorithms when using apps and use them to inform decisions. If machine learning bias affects AI-powered systems, it can cause a lot of issues—particularly in the case of facial recognition or automated decision-making systems.

If these systems are affected by bias, they might discriminate against employees based on their race or identity. In 2015, a Black software developer tweeted his photo with a friend. Google’s Photo Service, which uses AI to label photos, categorized both of them as gorillas.

Bias doesn’t only target humans but can also inflict damage on business processes. Suppose an eCommerce website uses machine learning to categorize its clothing products that are sold all over the world. If the model is trained to identify clothes by only using the data of a specific country, like Western countries, then it will fail to detect Eastern styles of clothing.

Bias vs. Variance: What’s the Difference?

Variance refers to how scattered the predicted values (by the model) are from the actual values. Bias, on the other hand, refers to the difference between the average prediction of the model and the actual values.

Variance and bias have an inversely proportional relationship. Models with high variance have low bias, while models with low variance have high bias.

If you try to modify a machine learning model to better fit your dataset, this action will increase variance and lower bias. A high variance leads to overfitting, where you have trained your model with a lot of irrelevant data, also known as noise. This increases the likelihood of unreliable or incorrect predictions for new data points.

On the other hand, if you increase bias and lower variance, then your model might miss a key relation or pattern between the input and output. When the bias gets high, it can cause the model to go through underfitting—failing to identify main relationships.

Ideally, you want a delicate balance between bias and variance to minimize overfitting and underfitting. This balance is referred to as a bias-variance trade-off. You can improve this balance by using a number of techniques, such as feature engineering, hyperparameter tuning, and model optimization, that can help maintain this balance.

How to Minimize Machine Learning Bias

Fortunately, bias is something that can be mitigated. You can use the following tactics to minimize machine learning bias in your models.

Start with Accurate, Representative Data

If you feed your model with biased training data, then your system is always going to produce faulty results. And when all of your datasets have similar characteristics, then the characteristics you are leaving out can cause bias. Include representative, accurate samples in your training data to minimize these biases.

Say you want to screen your customers via facial recognition. You should supply your model with training data of all ethnicities (e.g., Caucasian, Asian, Black) to make sure the sample is representative.

Anonymize the Data

Data that contains age, gender, race, and other identifying information can increase the likelihood of profiling subjects. Anonymize this data to make sure your model doesn’t use this information for profiling.

Hire Representative Annotators

In real-world scenarios, most training datasets contain ambiguous information. That’s why you need to hire a diverse group of annotators who can label your data to express a subjective point of view.

Revise Your Model Regularly

Some might think that once a machine learning model has been trained with the right techniques, it will not err in the future. However, contrary to your testing environment, your model will operate in a more dynamic real-world environment and needs to be periodically retrained using newer datasets to avoid bias in the future.