Scrapper

flowtask.components.ProductCompetitors.scrapper

ProductCompetitors

ProductCompetitors(loop=None, job=None, stat=None, **kwargs)

Bases: FlowComponent, SeleniumService, HTTPService

Product Competitors Scraper Component

Overview: Pluggable component for scraping product information from competitors (BestBuy and Lowes).

Properties: - url_column (str): Name of the column containing URLs to scrape (default: 'url') - account_name_column (str): Name of the column containing retailer name (default: 'account_name') - product_id_column (str): Name of the column containing product IDs (default: 'product_id') - competitors (list): List of competitor brands to search for (e.g. ['Insignia', 'TCL', 'LG', 'Sony', 'Samsung'])

close async

close()

Clean up resources.

run async

run()

Execute scraping for all URLs in the DataFrame.

start async

start(**kwargs)

Initialize the component and validate required parameters.