ResNet34 vs. PlainNet: Deep Residual Learning on CIFAR-10

This project implements and compares a ResNet-34 model with a 34-layer Plain CNN (without skip connections) to analyze the impact of residual learning on model training and performance for image classification.

Inspired by the seminal paper: "Deep Residual Learning for Image Recognition" – He et al., CVPR 2016

📄 Original Paper

---

🔍 Project Overview

The goal of this project is to empirically validate the core hypothesis of the ResNet paper: that deep residual networks are easier to optimize and can achieve higher accuracy than their "plain" counterparts, which suffer from a degradation problem as depth increases.

• Objective: Compare the performance of a PlainNet-34 against a ResNet-34.
• Dataset: CIFAR-10 (50,000 training + 10,000 test images across 10 classes).

---

🔬 Results & Analysis

The models were trained for 30 epochs on the CIFAR-10 dataset. The ResNet-34 model demonstrated significantly better optimization and achieved a much higher test accuracy, confirming the effectiveness of residual connections.

Model	Test Accuracy	Test Loss
PlainNet34	37.59%	1.6171
ResNet34	68.47%	0.8777

✅ Conclusion: The ResNet-34 model achieved a 30.88 percentage point increase in test accuracy over the PlainNet-34, successfully demonstrating that residual learning helps mitigate the degradation problem in very deep networks.

Training History

The training plots clearly show the optimization advantage of ResNet. The PlainNet's validation loss is erratic and fails to converge effectively, while the ResNet model shows stable convergence.

ResNet34 Results

PlainNet34 Results

---

🧱 Architecture

Both models share a similar 34-layer structure, with the key difference being the presence of skip connections in ResNet.

• ResNet-34:
  • Uses residual blocks (Identity and Convolutional) that add the input x to the output of a block F(x).
  • This allows the network to learn residual mappings, which are easier to optimize.
• PlainNet-34:
  • A standard deep CNN with 34 convolutional layers.
  • Does not use any skip connections.

---

🔧 Tech Stack

---

📁 Files Included

• Final_resnet34.ipynb – The complete Jupyter Notebook containing the implementation, training, and evaluation code for both ResNet-34 and PlainNet-34.
• ResNet-Deep-Residual-Learning-for-Image-Recognition.pdf – A presentation summarizing the project's methodology, architecture, and results.

---

🧠 Key Learnings

• Successfully implemented the core architectural components of ResNet, including identity and convolutional skip connections.
• Empirically verified that skip connections mitigate the vanishing gradient problem and allow for the successful training of much deeper networks.
• The comparison with a PlainNet clearly illustrated the "degradation" problem, where adding more layers to a plain network leads to higher training error.

---

📌 Citation

@article{he2016deep,
  title={Deep residual learning for image recognition},
  author={He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian},
  journal={Proceedings of the IEEE conference on computer vision and pattern recognition},
  year={2016}
}