Required Data Sources

[splimon]

Required Data Sources

Datasets:

• Github Gold Standard Dataset from Figshare: Manually labeled sentiment data (ID, Polarity, Text) from GitHub pull request and commit comments. Contains over 7,000 documents relating to polarity of emotions of software developers in GitHub
• GHTorrent 2004 MySQL Database Dump from Internet Archive: Historical GitHub metadata including projects, users, commits, and commit_comments tables
• GHTorrent Database Schema Diagram from Internet Archieve: Visual reference for table relationships and column names of the GHTorrent 2004 MySQL Database Dump

Code Repositories:

• sentiment_classifier: Main Python ML pipeline for sentiment analysis
• sentiment_github_dataset: Repo for Github Gold Standard Dataset preparation work
• Kaiaulu: R package for software repository analysis and data extraction

Reference Materials:

• Kaiaulu Issue #226: Data source integration reference (outdated but useful context)
• sentiment_classifier PR #4: Previous ICS 496 student's work creating train_or_predict.py script
• Kaiaulu PR #347: Haotian's original sentiment analysis implementation (R and Python integration)
• Kaiaulu Issue #346 - Architectural discussion on R/Python code organization
• Sentiment Analysis Cheatsheet: Documentation of current sentiment analysis workflow