Skip to content

mohsohail-567/IntelSOC

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
backend
backend
 
 
data
data
 
 
reports
reports
 
 
static
static
 
 
templates
templates
 
 
uploads
uploads
 
 
.gitignore
.gitignore
 
 
README.md
README.md
 
 
main.py
main.py
 
 
requirements.txt
requirements.txt
 
 
run.bat
run.bat
 
 

Repository files navigation

IntelSOC - Log Based Threat Detection System

A comprehensive Security Operations Center (SOC) platform that detects, analyzes, and responds to cybersecurity threats in real-time. IntelSOC leverages advanced threat intelligence and pattern matching to identify malicious activities, generate professional incident reports, and provide actionable insights for security teams.

🎯 Features

Core Threat Detection

  • Real-time Log Analysis: Process and analyze security logs for suspicious patterns
  • Multi-vector Threat Detection: Identify various attack types (brute force, SQL injection, DDoS, unauthorized access, etc.)
  • Intelligent Alerting: Generate severity-based alerts (Critical, High, Medium, Low)
  • IP Reputation Tracking: Maintain and manage IP blocklists
  • Attack Metrics: Track attack trends and statistics over time

Report Generation & Download

  • Professional PDF Reports: Generate comprehensive incident reports with executive summaries
  • One-Click Download: Download latest report directly from dashboard
  • Report Management: View, organize, and download all historical reports
  • Executive Summary: Key metrics for management briefings
  • Detailed Threat Analysis: Complete breakdown of detected threats
  • Incident Response Documentation: Automated response actions and analyst guidance

Dashboard & Visualization

  • Live Metrics: Real-time threat statistics and alerts
  • Charts & Analytics: Visual representation of threat distribution
  • Alert Feed: Latest security incidents and alerts
  • Quick Access: One-button report download from main dashboard

🚀 Installation

Prerequisites

  • Python 3.7 or higher
  • pip (Python package manager)
  • Windows, Linux, or macOS

Step 1: Install Dependencies

# Navigate to project directory
cd ThreatDetection

# Install required packages
pip install -r requirements.txt

Step 2: Verify Installation

# Check Python version
python --version

# Verify Flask is installed
python -c "import flask; print('Flask version:', flask.__version__)"

# Verify ReportLab is installed
python -c "import reportlab; print('ReportLab version:', reportlab.__version__)"

Step 3: Run the Application

Windows:

run.bat

Linux/Mac:

python app.py

Manual Start (All Platforms):

python app.py

Expected Output

* Serving Flask app 'backend.app'
* Debug mode: on
* Running on http://127.0.0.1:5000

Access IntelSOC at: http://localhost:5000

📖 How to Use

1. Dashboard Overview

When you open IntelSOC, you'll see:

  • Metrics Cards: Total alerts, critical incidents, high-severity attacks, blocked IPs
  • Severity Distribution: Pie chart showing threat distribution
  • Attack Types: Bar chart showing attack frequency
  • Recent Alerts: Feed of latest detected threats
  • SOC Incident Report: Quick download button for latest report

2. Uploading Logs

  1. Click "Upload Logs" in the navigation bar
  2. Select a log file (text format)
  3. IntelSOC analyzes the logs for threats
  4. Detected threats appear on dashboard

Example Log Format:

[2024-01-15 10:23:45] Login attempt from 192.168.1.100
[2024-01-15 10:24:12] Multiple failed passwords detected from 10.0.0.50
[2024-01-15 10:25:33] SQL injection attempt detected in query parameter
[2024-01-15 10:26:01] DDoS traffic spike from 203.0.113.42

3. Viewing Alerts

  1. Click "Alerts" in the navigation bar
  2. View all detected security incidents
  3. See severity level, detection time, and alert details
  4. Each alert shows:
    • Alert ID and timestamp
    • Source IP and attack type
    • Severity classification
    • Detection confidence

4. Generating Reports

Option A: Quick Download from Dashboard

  1. Scroll down to "SOC Incident Report" card
  2. Click "⬇️ Download Latest"
  3. Latest report PDF downloads automatically

Option B: From Reports Page

  1. Click "Manage Reports" button on dashboard OR navigate to: http://localhost:5000/reports
  2. Click "Generate Incident Report" button
  3. System generates PDF with current threat data
  4. Report appears in "Latest Report" section
  5. Click download button

Option C: View Report History

  1. Go to Reports page
  2. See "All Reports" table listing every generated report
  3. Each row shows:
    • Report filename
    • Generation date and time
    • File size
    • Download button
  4. Download any historical report

5. Understanding the Report

Each PDF incident report contains:

Executive Summary

  • Critical incident count
  • High-severity incident count
  • Total attacks detected
  • IP addresses blocked

Incident Statistics

  • Breakdown by severity level
  • Metrics in professional table format

Detected Threats

  • Attack types identified
  • Attack frequency and distribution

Incident Response

  • Blocked IP addresses and block reasons
  • Automated response actions

SOC Analyst Guidance

  • Investigation steps to follow
  • Escalation criteria
  • Documentation requirements

🏗️ Platform Architecture

Backend Components

app.py - Main Flask Application

  • HTTP request handling
  • Route management
  • API endpoints

threat_engine.py - Threat Detection Engine

  • Pattern matching for threats
  • Log analysis algorithms
  • Alert generation
  • Attack classification

report_generator.py - Report Generation Module

  • PDF creation with ReportLab
  • Report formatting and styling
  • File management

json_handler.py - Data Management

  • JSON file operations
  • Data persistence
  • Alert/attack storage

email_service.py - Email Integration

  • Alert notifications (optional)
  • Report distribution

Frontend Components

Dashboard (index.html)

  • Main interface
  • Metrics display
  • Quick report download
  • Navigation

Reports Page (reports.html)

  • Report management UI
  • Generate/download interface
  • Report history listing
  • Status updates

Alerts Page (alerts.html)

  • Alert viewing interface
  • Detailed alert information
  • Filtering and sorting

Upload Page (upload.html)

  • Log file upload interface
  • File selection
  • Upload progress

Styling (static/css/style.css)

  • Dark SOC-themed design
  • Responsive layout
  • Professional appearance

JavaScript (static/js/dashboard.js)

  • Real-time metric updates
  • Interactive charts
  • Report download functionality
  • User feedback

Data Storage

data/alerts.json

  • Detected security incidents
  • Alert details and timestamps

data/attacks.json

  • Attack type records
  • Attack frequency tracking

data/blocklist.json

  • Blocked IP addresses
  • Block reasons and timestamps

data/stats.json

  • Statistical information
  • Metrics and aggregations

reports/ folder

  • Generated PDF reports
  • Timestamped filenames

🔌 API Endpoints

Dashboard Metrics

GET /api/dashboard/metrics

{
  "total_alerts": 42,
  "critical_count": 3,
  "high_count": 8,
  "medium_count": 15,
  "low_count": 16,
  "total_blocked_ips": 12,
  "attacks": {...}
}

Report Generation

POST /api/reports/generate

  • Generates new incident report
  • Returns success status and filename

Get Latest Report

GET /api/reports/latest

{
  "success": true,
  "filename": "IntelSOC_Incident_Report_20240115_143022.pdf",
  "size": 45678,
  "timestamp": "2024-01-15 14:30:22"
}

List All Reports

GET /api/reports/list

{
  "success": true,
  "reports": [
    {
      "filename": "IntelSOC_Incident_Report_20240115_143022.pdf",
      "size": 45678,
      "timestamp": "2024-01-15 14:30:22"
    }
  ]
}

Download Report

GET /api/reports/download/<filename>

  • Downloads PDF file directly
  • Example: /api/reports/download/IntelSOC_Incident_Report_20240115_143022.pdf

Get Alerts

GET /api/alerts

  • Returns list of all alerts

Get Attacks

GET /api/attacks

  • Returns attack statistics

Get Blocklist

GET /api/blocklist

  • Returns blocked IP addresses

🎓 Example Workflows

Workflow 1: Daily Monitoring

  1. Open dashboard each morning
  2. Check metric cards for overnight activity
  3. Review alert feed
  4. Download latest report for team briefing
  5. Share PDF with security team

Workflow 2: Incident Response

  1. Receive security alert notification
  2. Go to Alerts page for details
  3. Analyze threat data on dashboard
  4. Generate incident report
  5. Document in PDF for incident response team
  6. Archive report for compliance

Workflow 3: Log Analysis

  1. Receive new security logs
  2. Click "Upload Logs"
  3. Select and upload log file
  4. Wait for analysis (2-5 seconds)
  5. View detected threats on dashboard
  6. Review in Alerts section
  7. Download report with findings

Workflow 4: Compliance Reporting

  1. Go to Reports page
  2. View all reports for compliance period
  3. Download reports needed for audit
  4. Combine PDFs into audit package
  5. Submit to compliance team
  6. Archive for records

🛡️ Security Features

  • Threat Validation: Each detected threat is verified against known patterns
  • IP Reputation: Maintained blocklist of malicious IPs
  • File Security:
    • Filename validation on downloads
    • Directory traversal prevention
    • Secure file serving with proper headers
  • Audit Logging: All operations logged for compliance
  • Data Persistence: Secure JSON storage of alerts and threats

⚙️ Configuration

Default Settings

  • Flask Debug Mode: ON (development)
  • Update Interval: 10 seconds
  • Report Format: PDF
  • Storage Location: /reports/ directory

Environment Variables

Create .env file for sensitive configuration:

FLASK_ENV=development
FLASK_DEBUG=True
REPORT_FOLDER=./reports

🐛 Troubleshooting

Issue: Port 5000 already in use

Solution:

# Use different port
python app.py --port 5001

# Or find process using port 5000 and kill it

Issue: "ReportLab not found"

Solution:

pip install reportlab==4.0.4

Issue: Dashboard not updating

Solution:

  1. Refresh browser (Ctrl+F5)
  2. Check browser console for errors (F12)
  3. Restart application: python app.py

Issue: Upload logs not working

Solution:

  • Ensure log file is in text format
  • Check file permissions
  • Verify uploads folder exists
  • Check logs match expected format

Issue: Reports folder not created

Solution:

# Create manually
mkdir reports

# Or generate a report (creates folder automatically)

Issue: Download reports not working

Solution:

  1. Check /reports folder exists
  2. Verify app has write permissions
  3. Refresh browser
  4. Check Python console for errors
  5. Check browser console (F12)

📊 Understanding Threat Levels

Critical (🔴)

  • Active exploitation attempts
  • Data breach indicators
  • Unauthorized system access
  • Action: Immediate investigation and response

High (🟠)

  • Multiple failed authentication attempts
  • Known exploit patterns
  • Suspicious command execution
  • Action: Investigate within 1 hour

Medium (🟡)

  • Unusual traffic patterns
  • Policy violations
  • Suspicious but not confirmed threats
  • Action: Monitor and investigate

Low (🟢)

  • Benign suspicious activity
  • Informational alerts
  • Policy deviations
  • Action: Log and monitor

📈 Metrics Explained

Total Alerts: Number of security incidents detected Critical Count: High-priority incidents requiring immediate action High Count: Important threats needing investigation Medium Count: Moderate-priority alerts for monitoring Low Count: Informational alerts Blocked IPs: Number of IP addresses in blocklist

🔄 Data Flow

Logs Uploaded → Threat Detection Engine → Alerts Generated
                                        ↓
                              Dashboard Display
                                        ↓
                        Report Generation on Demand
                                        ↓
                            PDF Download & Archive

📝 File Structure

ThreatDetection/
├── app.py                  # Main application (DO NOT EDIT)
├── main.py                 # Entry point
├── requirements.txt        # Python dependencies
├── run.bat                 # Windows startup script
├── README.md              # This file
│
├── backend/
│   ├── app.py             # Flask application
│   ├── threat_engine.py    # Threat detection logic
│   ├── report_generator.py # PDF generation
│   ├── json_handler.py     # Data management
│   └── email_service.py    # Email integration
│
├── static/
│   ├── css/
│   │   └── style.css       # Dashboard styling
│   └── js/
│       └── dashboard.js    # Frontend JavaScript
│
├── templates/
│   ├── index.html          # Dashboard page
│   ├── alerts.html         # Alerts page
│   ├── reports.html        # Reports management
│   └── upload.html         # Log upload page
│
├── data/
│   ├── alerts.json         # Alert records
│   ├── attacks.json        # Attack statistics
│   ├── blocklist.json      # Blocked IPs
│   └── stats.json          # System metrics
│
├── reports/                # Generated PDF reports
└── uploads/                # Uploaded log files

🚀 Performance Tips

  1. Clean Up Old Uploads: Regularly delete processed logs from uploads/ folder
  2. Archive Reports: Move old reports to external storage periodically
  3. Monitor Storage: Ensure disk space is available for reports
  4. Browser Cache: Clear cache if dashboard appears stale
  5. Database Maintenance: JSON files are compact but should be pruned occasionally

📞 Support & Tips

Getting More Help

  • Check browser console for errors: Press F12
  • Review Python console output for backend errors
  • Verify file permissions on reports/ and uploads/ folders
  • Ensure all dependencies installed: pip list

Tips for Best Results

  • Upload logs regularly for comprehensive threat tracking
  • Generate reports weekly for compliance documentation
  • Review alerts feed daily for quick threat assessment
  • Use Reports page to manage and archive incidents
  • Export and backup reports for permanent records

Next Steps

  1. Start application: python app.py
  2. Upload sample logs
  3. Review detected threats
  4. Generate your first report
  5. Download and review PDF

🎉 You're Ready!

IntelSOC is now installed and ready to use. Start monitoring your security threats and generating professional incident reports.

Next: Open http://localhost:5000 in your browser to begin!

IntelSOC - Log Based Threat Detection System
Professional Security Operations Center Platform

About

Threat intelligence + SOC operations

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages