Scrapy

Scrapy: Basic Commands

  1. Create Project

scrapy startproject EXAMPLE /home/Scripts/scrapy/EXAMPLE

 

2 . Create Project inside the spiders folder

with the format: example_spider.py

 

3.  Start testing with the console.

scrapy shell 'http://quotes.toscrape.com'

 

And execute the XPath commands

https://devhints.io/xpath

 

To run the spider execute the followng command on the folder.

scrapy crawl <spider name> -o file.csv -t csv

Leave a Reply

Your email address will not be published. Required fields are marked *