Exceptions

Built-in Exceptions reference

Here’s a list of all exceptions included in Scrapy and their usage.

DropItem

exception scrapy.exceptions.DropItem

The exception that must be raised by item pipeline stages to stop processing an Item. For more information see Item Pipeline.

CloseSpider

exception scrapy.exceptions.CloseSpider(reason='cancelled')

This exception can be raised from a spider callback to request the spider to be closed/stopped. Supported arguments:

Parameters:reason (str) – the reason for closing

For example:

def parse_page(self, response):
    if 'Bandwidth exceeded' in response.body:
        raise CloseSpider('bandwidth_exceeded')

IgnoreRequest

exception scrapy.exceptions.IgnoreRequest

This exception can be raised by the Scheduler or any downloader middleware to indicate that the request should be ignored.

NotConfigured

exception scrapy.exceptions.NotConfigured

This exception can be raised by some components to indicate that they will remain disabled. Those components include:

  • Extensions
  • Item pipelines
  • Downloader middlewares
  • Spider middlewares

The exception must be raised in the component constructor.

NotSupported

exception scrapy.exceptions.NotSupported

This exception is raised to indicate an unsupported feature.