Skip to content

Web scraping with Selenium and ProxyMesh, storing the data in MongoDB, showing a list on a webpage.

Notifications You must be signed in to change notification settings

shhiivvaam/barsaati

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Barsaati

Overview

Barsaati is a project aimed at scraping Twitter trends using Selenium WebDriver. This README provides detailed instructions on setting up the project, including resolving issues related to ChromeDriver installation and configuration.

Prerequisites

  • Node.js (v12 or later)
  • npm (Node Package Manager)
  • Google Chrome browser

Installation

  1. Clone the Repository:

    git clone https://github.com/shhiivvaam/barsaati.git
    cd barsaati
    
  2. Install Dependencies:

    npm install
    
  3. Download ChromeDriver:

    • Go to the ChromeDriver download page.
    • Download the version of ChromeDriver that matches your Google Chrome version.
    • Extract the downloaded file and note the path to chromedriver.exe.
  4. Set ChromeDriver Path:

    • Add the path to chromedriver.exe to your system's PATH environment variable, or
    • Specify the path directly in your code (see Configuration section).

Configuration

Selenium WebDriver Configuration

In your Selenium script (e.g., selenium-script.js), configure the path to ChromeDriver:

const {Builder, By, Key, until} = require('selenium-webdriver');
const chrome = require('selenium-webdriver/chrome');

let service = new chrome.ServiceBuilder('path/to/your/chromedriver.exe').build();
chrome.setDefaultService(service);

let driver = new Builder().forBrowser('chrome').build();

Running the Application

  1. Start the Server :
node backend/index.js
  1. Access the Application : Open your browser and navigate to http://localhost:3000.

Thankyou for visiting!!

About

Web scraping with Selenium and ProxyMesh, storing the data in MongoDB, showing a list on a webpage.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published