ER Tool — Engagement Rate Tool

Scrape TikTok and Instagram profiles by username and month to measure engagement rate. Collects views, likes, comments, and ER per post.

Features

• TikTok — scrapes all posts for a given month, includes views, likes, comments, shares, and engagement rate. Optional account-level ER based on last 30 posts.
• Instagram — scrapes posts for a given month via a real browser session (Playwright). Includes post type (Photo / Video / Reel / Carousel), views (videos/reels), likes, and comments.
• Runs a hidden Chrome browser in the background — no visible popup.
• Session is saved after first Instagram login (paste sessionid once).
• Export results as CSV per user or combined.

Requirements

• Python 3.10+
• Playwright browsers installed

Setup

pip install -r requirements.txt
playwright install chromium

Usage

Web app (recommended)

streamlit run app.py

Open http://localhost:8501, select a source, enter usernames (one per line, no @), pick a month, and click Scrape.

CLI (TikTok only)

python harvester.py username1 username2 --month 2026-03
python harvester.py roihan_jnr --month 2026-04 --output april.csv

Flag	Description
--month / -m	Month to scrape in YYYY-MM format (required)
--output / -o	CSV output file (default: tiktok_YYYY_MM.csv)
--sessions / -s	Number of parallel browser sessions (default: 1)

Instagram setup

Instagram requires a sessionid cookie from a logged-in browser session.

1. Open instagram.com in Chrome while logged in
2. Open DevTools → Application → Cookies → https://www.instagram.com
3. Copy the value of the sessionid cookie
4. Paste it into the app when prompted

The session is saved to ~/.config/ig_scraper_session.json and reused automatically. Use the Log out button in the app to clear it.

Important notes

Instagram account safety

Instagram actively detects and bans accounts that show unusual automation behavior. Do not use your personal or main business Instagram account. Create a dedicated burner account specifically for scraping. If that account gets banned or challenged, simply create a new one — your real account stays safe.

• Scraping too many accounts in a short time increases ban risk. Space out your requests.
• If Instagram triggers a security challenge (email/phone verification), the session will be invalidated and you will need to re-login with a fresh sessionid.
• TikTok scraping does not require any account — the browser session is anonymous.

Output fields

Field	TikTok	Instagram
Date	✓	✓
Post Type	—	✓ (Photo / Video / Reel / Carousel)
Views	✓	✓ (videos/reels only)
Likes	✓	✓
Comments	✓	✓
Shares	✓	—
View Eng. Rate %	✓	✓
Caption	✓	✓
Followers	✓	✓