Adding Bias Nodes to a Multilayer Perceptron in Python

This article explains how to incorporate bias nodes into a multilayer perceptron (MLP) using Python, and evaluates their effect on classification performance.

Welcome to the All About Circuits neural‑network series, authored by Director of Engineering Robert Keim. If you haven’t yet explored the other lessons, you’ll find them listed below. They provide a solid foundation for understanding the concepts discussed in this post.

1. How to Perform Classification Using a Neural Network: What Is the Perceptron?
2. How to Use a Simple Perceptron Neural Network Example to Classify Data
3. How to Train a Basic Perceptron Neural Network
4. Understanding Simple Neural Network Training
5. An Introduction to Training Theory for Neural Networks
6. Understanding Learning Rate in Neural Networks
7. Advanced Machine Learning with the Multilayer Perceptron
8. The Sigmoid Activation Function: Activation in Multilayer Perceptron Neural Networks
9. How to Train a Multilayer Perceptron Neural Network
10. Understanding Training Formulas and Backpropagation for Multilayer Perceptrons
11. Neural Network Architecture for a Python Implementation
12. How to Create a Multilayer Perceptron Neural Network in Python
13. Signal Processing Using Neural Networks: Validation in Neural Network Design
14. Training Datasets for Neural Networks: How to Train and Validate a Python Neural Network
15. How Many Hidden Layers and Hidden Nodes Does a Neural Network Need?
16. How to Increase the Accuracy of a Hidden Layer Neural Network
17. Incorporating Bias Nodes into Your Neural Network

Bias nodes are additional inputs that provide a constant value—typically +1—directly to a perceptron’s input or hidden layers. They allow the network to shift decision boundaries without altering the input data itself. This article presents two practical methods for adding bias nodes and shows how to assess their impact on model accuracy.

Method 1: Adding Bias via the Spreadsheet

When your training or validation data reside in a spreadsheet, you can insert a bias column without touching the Python code.

Steps:

1. Insert a new column in the spreadsheet and fill it with the desired bias value (usually +1).
2. Adjust the input‑layer dimensionality in your code to account for the new column.

This approach is quick and requires no code changes, making it ideal for prototyping.

Method 2: Integrating Bias Directly in Python

For more flexibility—such as adding bias to hidden layers—you’ll modify the network architecture in code. Below is a concise example that adds bias nodes to both the input and hidden layers.

Updating Input‑Layer Dimensionality

# Load training data
training_data = np.array(...)  # shape: (samples, features)
training_count = training_data.shape[0]

# Append a column of +1s for the input bias
bias_column = np.ones((training_count, 1))
training_data = np.hstack((training_data, bias_column))

# Repeat for validation data
validation_data = np.hstack((validation_data, np.ones((validation_count, 1))))

Here, np.ones creates a column of ones that represents the constant bias input. The np.hstack function then appends this column to the existing feature matrix.

Adding a Hidden‑Layer Bias Node

# During feedforward computation
for node in range(hidden_dim):
    # Compute weighted sum for each hidden node
    z = np.dot(input_vector, weights_input_hidden[:, node])
    post_activation[node] = activation_function(z)

# Insert bias node value (+1) into the hidden layer
post_activation[hidden_dim] = 1.0  # Bias node at the end

When hidden_dim includes the bias node, the loop stops before the last node, and the bias value is assigned afterward. Adjust the corresponding validation loop in the same way.

Choosing Different Bias Values

Although +1 is conventional, you can experiment with other constants. For the input bias, multiply bias_column by the desired value. For the hidden bias, modify the assignment in the feedforward loop accordingly.

Evaluating Bias Impact

In Part 16 of the series, the MLP struggled to classify the high‑complexity Experiment 3 dataset. To determine whether bias nodes could help, we performed ten independent training runs with a hidden layer of seven units, comparing four configurations: no bias, input bias only, hidden bias only, and both biases.

The results show minimal differences—bias nodes did not significantly improve accuracy for this particular dataset. This outcome aligns with findings from other studies that suggest bias nodes can be beneficial in certain contexts but are not a universal performance booster.

Conclusion

While bias nodes rarely transform classification accuracy in all scenarios, they remain a valuable architectural tool. Incorporating bias support into your codebase ensures you can quickly test its effectiveness on future datasets or model variants. By following the methods outlined above, you can add bias nodes with minimal effort and evaluate their impact systematically.