A Big Data project that detects fraudulent bank transactions using Apache Pig for large-scale data processing, PySpark for machine learning, and Tableau for interactive visualization.
Online banking systems generate millions of transactions every day. Detecting fraudulent activities within such huge datasets is difficult using traditional techniques.
This project combines Hadoop ecosystem tools with machine learning to identify suspicious transactions and visualize fraud patterns.
- Data preprocessing
- Fraud transaction filtering using Apache Pig
- Fraud analysis on Hadoop
- Machine Learning using PySpark Logistic Regression
- Fraud prediction
- Tableau dashboards
- Fraud statistics and reporting
- Python
- Apache Hadoop
- Apache Pig
- PySpark
- Pandas
- Tableau
- HDFS
The project uses a synthetic bank transaction dataset containing over 2500 transactions.
Dataset includes:
- Transaction ID
- Amount
- Transaction Type
- Customer ID
- Merchant ID
- Location
- Fraud Label
- Load CSV into HDFS
- Clean and preprocess data
- Analyze fraud using Apache Pig
- Train Logistic Regression model using PySpark
- Generate fraud predictions
- Visualize results using Tableau
Algorithm:
- Logistic Regression
Evaluation Metric:
- AUC Score
Achieved Performance:
AUC = 0.9213
Bank-Transaction-Fraud-Detection
│
├── data
├── src
├── report
├── images
├── outputs
├── README.md
└── requirements.txt
Clone the repository
git clone https://github.com/yourusername/Bank-Transaction-Fraud-Detection.gitInstall dependencies
pip install -r requirements.txtRun
python src/bank.py✔ Fraud detection using PySpark
✔ Regional fraud analysis using Pig
✔ Interactive Tableau dashboards
✔ Fraud transaction reporting
- Apache Spark Streaming
- Kafka Integration
- Random Forest and XGBoost models
- Real-time fraud detection
- Web dashboard deployment
Arushi