Market Data Library

⚠️ Disclaimer

This software is provided for educational and research purposes only. It is not affiliated with, endorsed by, or in any way associated with Barchart, CNN, CoinMarketCap, Alternative.me, Messari, or any other data provider.

Use of this software may violate the terms of service of the data sources. The authors and contributors are not responsible for any misuse, legal consequences, or damages resulting from the use of this software.

Commercial use of this software and the data it provides is strictly prohibited.

By using this software, you agree to comply with all applicable laws and the terms of service of the data providers.

Introduction

A Python library for accessing market data from various stocks and cryptocurrency data providers. This library provides a unified, type-safe interface to multiple data sources with built-in rate limiting, authentication management, and async support.

📋 Table of Contents

• Features
• Data Sources
• Project Structure
• Installation
• Quick Start
• Usage Examples
• Development Setup
• Production Deployment
• API Reference
• Contributing
• Known Issues

✨ Features

• Multiple Data Providers: Access stocks, options, and crypto data from various sources
• Type Safety: Full type hints with py.typed marker for mypy compatibility
• Async/Await: Asynchronous API for high-performance data fetching
• Rate Limiting: Built-in rate limiting to respect API constraints
• Authentication Management: Automatic session management and re-authentication
• Web Scraping Support: Selenium integration with stealth mode for web-based data sources
• Docker Ready: Configurable for both local and containerized deployments
• Connection Pooling: Efficient HTTP client with connection reuse

📊 Data Sources

Stocks

Provider	Data Types	Authentication	Status
Barchart	Stock prices, Options data, Options expirations, Most active options, Open interest changes	Session-based	✅ Working
CNN Fear & Greed Index	Market sentiment indicator	None	✅ Working

Cryptocurrency

Provider	Data Types	Authentication	Status
CoinMarketCap (CMC)	Crypto prices, Historical data, Market cap, Sector analysis, Top search ranks	None (public API)	✅ Working
Alternative.me	Crypto fear & greed index	None	✅ Working
Messari	Various crypto data	API Key	⚠️ Not working (API changes)

🏗️ Project Structure

market-data-library/
├── market_data_library/          # Main library package
│   ├── crypto/                   # Cryptocurrency data providers
│   │   ├── alternativeme/        # Alternative.me fear & greed
│   │   └── cmc/                  # CoinMarketCap integration
│   ├── stocks/                   # Stock market data providers
│   │   ├── barchart/             # Barchart stocks & options
│   │   │   ├── auth/             # Authentication & rate limiting
│   │   │   ├── options/          # Options data services
│   │   │   └── stocks/           # Stock price services
│   │   └── cnn_fear_greed/       # CNN Fear & Greed Index
│   ├── http/                     # Shared HTTP client infrastructure
│   │   ├── http_client.py        # Async HTTP client wrapper
│   │   └── common.py             # Common HTTP utilities
│   └── util/                     # Common utilities
│       ├── date_util.py          # Date/time helpers
│       └── logger.py             # Logging configuration
├── example/                      # Usage examples (mirrors library structure)
│   ├── crypto/
│   │   ├── alternativeme/
│   │   └── cmc/
│   └── stocks/
│       ├── barchart/
│       └── cnn_fear_greed/
├── .github/                      # GitHub configuration & documentation
│   ├── copilot-instructions.md   # Universal engineering practices
│   └── project.md                # Project-specific guidelines
├── pyproject.toml                # Poetry configuration
├── setup.py                      # Package setup (legacy)
└── README.md                     # This file

🚀 Installation

Prerequisites

• Python 3.10 or higher
• Poetry (recommended) or pip
• Chrome browser (for web scraping features)

Using Poetry (Recommended)

# Clone the repository
git clone https://github.com/hanchiang/market_data_api.git
cd market-data-library

# Install dependencies
poetry install

# Activate virtual environment
poetry shell

Using pip

# Clone the repository
git clone https://github.com/hanchiang/market_data_api.git
cd market-data-library

# Create virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies from pyproject.toml
pip install -e .

Setting PYTHONPATH

# Linux/Mac
export PYTHONPATH=$(pwd)

# Windows (PowerShell)
$env:PYTHONPATH = (Get-Location).Path

# Windows (CMD)
set PYTHONPATH=%cd%

🎯 Quick Start

Fetch CNN Fear & Greed Index

import asyncio
from market_data_library.stocks.cnn_fear_greed.cnn_fear_greed import CNNService

async def main():
    # Initialize service
    service = CNNService(remote_mode=False)  # Use local Chrome
    
    # Fetch data
    data = await service.get_fear_greed_index()
    
    print(f"Current Fear & Greed: {data.fear_and_greed.score}")
    print(f"Rating: {CNNService.map_fear_greed_to_text(data.fear_and_greed.score)}")

asyncio.run(main())

Fetch Crypto Fear & Greed Index

import asyncio
from market_data_library.crypto.alternativeme.alternativeme import AlternativeMeService

async def main():
    service = AlternativeMeService()
    
    # Get last 10 days of data
    data = await service.get_fear_greed_index(days=10)
    
    print(f"Current value: {data.data[0].value}")
    print(f"Classification: {data.data[0].value_classification}")
    
    await service.cleanup()

asyncio.run(main())

Fetch Options Data from Barchart

import asyncio
from market_data_library.stocks.barchart.options.options import OptionService

async def main():
    service = OptionService()
    
    # Get options for SPY
    options = await service.get_options_for_ticker(symbol='spy')
    print(f"Found {len(options)} options")
    
    # Get expiration dates
    expirations = await service.get_options_expirations_for_ticker(symbol='spy')
    print(f"Available expirations: {expirations}")
    
    # Get most active options
    active = await service.get_most_active_options()
    print(f"Most active options: {active}")
    
    await service.cleanup()

asyncio.run(main())

Fetch CoinMarketCap Data

import asyncio
from market_data_library.crypto.cmc.cmc import CMCService

async def main():
    service = CMCService()
    
    # Get latest Bitcoin quote
    quote = await service.get_latest_quote(id='1')  # 1 = Bitcoin
    print(f"Bitcoin price: ${quote.data['1'].quote['USD'].price}")
    
    # Get sector list
    sectors = await service.get_sector_list()
    print(f"Number of sectors: {len(sectors)}")
    
    # Get top search ranks
    top_searches = await service.get_top_search_rank(num=10)
    print(f"Top crypto searches: {top_searches}")
    
    await service.cleanup()

asyncio.run(main())

📚 Usage Examples

For comprehensive usage examples, see example/README.md.

🛠️ Development Setup

Environment Configuration

1. Install development dependencies:

poetry install --with dev

2. Install pre-commit hooks (if available):

pre-commit install

3. Configure IDE for type checking:

Enable mypy in your IDE for real-time type checking.

Development Workflow

# Format code
poetry run ruff format .

# Lint code
poetry run ruff check .

# Type checking
poetry run mypy market_data_library/

# Run examples
poetry run python example/stocks/cnn_fear_greed/main.py

Adding a New Data Provider

1. Create provider directory under market_data_library/stocks/ or market_data_library/crypto/
2. Create __init__.py, main service file, and type.py for data models
3. Implement service class with async methods
4. Add corresponding example in example/ directory
5. Update this README with provider information

Example structure:

from market_data_library.http.http_client import HttpClient
from typing import List
from dataclasses import dataclass

@dataclass
class MyDataModel:
    field1: str
    field2: int

class MyService:
    def __init__(self):
        self.client = HttpClient(base_url="https://api.example.com")
    
    async def get_data(self) -> List[MyDataModel]:
        res = await self.client.get('/endpoint')
        return await res.json()
    
    async def cleanup(self):
        await self.client.close()

🚀 Production Deployment

Docker Configuration

For web scraping features (CNN Fear & Greed), configure remote WebDriver:

from market_data_library.stocks.cnn_fear_greed.cnn_fear_greed import CNNService

# Remote mode (Docker/Kubernetes)
service = CNNService(
    remote_mode=True,
    server_host='http://chrome:4444'  # Selenium Grid endpoint
)

data = await service.get_fear_greed_index()

Docker Compose Example

version: '3.8'
services:
  chrome:
    image: selenium/standalone-chrome:latest
    ports:
      - "4444:4444"
    shm_size: 2gb
  
  app:
    build: .
    depends_on:
      - chrome
    environment:
      - CHROME_HOST=http://chrome:4444

Environment Variables

# Chrome WebDriver
CHROME_HOST=http://chrome:4444

# Logging level
LOG_LEVEL=INFO

# Rate limiting (if overriding defaults)
BARCHART_RATE_LIMIT=60  # requests per minute

Performance Considerations

• Connection Pooling: HTTP clients reuse connections automatically
• Rate Limiting: Built-in rate limiters prevent API throttling
• Async Operations: Use asyncio.gather() for parallel requests
• Resource Cleanup: Always call await service.cleanup() when done

import asyncio

async def fetch_multiple():
    service1 = AlternativeMeService()
    service2 = CMCService()
    
    try:
        # Parallel execution
        crypto_fear, sectors = await asyncio.gather(
            service1.get_fear_greed_index(),
            service2.get_sector_list()
        )
        return crypto_fear, sectors
    finally:
        # Cleanup
        await service1.cleanup()
        await service2.cleanup()

📖 API Reference

Core Services

CNNService

• get_fear_greed_index() -> CnnFearGreedIndex
• map_fear_greed_to_text(value: float) -> dict

AlternativeMeService

• get_fear_greed_index(days: int = 1) -> AlternativeMeFearGreed

OptionService (Barchart)

• get_options_for_ticker(symbol: str, ...) -> dict
• get_options_expirations_for_ticker(symbol: str) -> dict
• get_most_active_options() -> dict
• get_change_in_options_interest(change_dir: str) -> dict

CMCService

• get_sector_list() -> List[CMCSector]
• get_crypto_list() -> dict
• get_homepage_listings_and_spotlight() -> CMCHomepageListingsAndSpotlight
• get_latest_quote(id: str) -> CMCLatestQuote
• get_ohlcv_historical(id: int, interval: str) -> CMCOHLCVHistorical
• get_coin_detail(id: int) -> CMCCoinDetail
• get_top_search_rank(data_type: int, timeframe: str, num: int) -> CMCTopSearchRank

For detailed API documentation, see the docstrings in each service class.

🤝 Contributing

We welcome contributions! Please follow these guidelines:

1. Read the documentation: Check .github/copilot-instructions.md and .github/project.md
2. Follow code style: Use type hints, async/await, and proper error handling
3. Write examples: Add usage examples for new features
4. Test your changes: Ensure existing examples still work
5. Update documentation: Keep README and docstrings current

Code Style Guidelines

• Type hints: All functions must have type annotations
• Async/Await: All I/O operations should be async
• Error handling: Use try-finally for resource cleanup
• Logging: Use module-level loggers with appropriate context
• Naming: snake_case for functions, PascalCase for classes
• Private methods: Prefix with underscore _method_name

Pull Request Process

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Make your changes with proper commits
4. Test thoroughly with examples
5. Update documentation
6. Submit pull request with clear description

⚠️ Known Issues

• Messari API: Currently non-functional due to upstream API changes
• selenium-stealth: Doesn't support Remote WebDriver mode (use remote_mode=False with stealth)
• Rate Limits: Some providers have strict rate limiting (60 req/min for Barchart)
• Web Scraping: CNN Fear & Greed relies on DOM structure which may change

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Data provided by Barchart, CNN, CoinMarketCap, and Alternative.me
• Built with aiohttp, Selenium, and dacite

📞 Contract

• Issues: GitHub Issues
• Discussions: GitHub Discussions