Built-in Exceptions reference¶
Here’s a list of all exceptions included in Scrapy and their usage.
- exception scrapy.exceptions.CloseSpider(reason='cancelled')¶
This exception can be raised from a spider callback to request the spider to be closed/stopped. Supported arguments:
reason (str) – the reason for closing
def parse_page(self, response):
if "Bandwidth exceeded" in response.body:
- exception scrapy.exceptions.DontCloseSpider¶
This exception can be raised in a
spider_idle signal handler to
prevent the spider from being closed.
- exception scrapy.exceptions.DropItem¶
The exception that must be raised by item pipeline stages to stop processing an Item. For more information see Item Pipeline.
- exception scrapy.exceptions.IgnoreRequest¶
This exception can be raised by the Scheduler or any downloader middleware to indicate that the request should be ignored.
- exception scrapy.exceptions.NotConfigured¶
This exception can be raised by some components to indicate that they will remain disabled. Those components include:
The exception must be raised in the component’s
- exception scrapy.exceptions.NotSupported¶
This exception is raised to indicate an unsupported feature.
New in version 2.2.
- exception scrapy.exceptions.StopDownload(fail=True)¶
fail boolean parameter controls which method will handle the resulting
fail=True(default), the request errback is called. The response object is available as the
responseattribute of the
StopDownloadexception, which is in turn stored as the
valueattribute of the received
Failureobject. This means that in an errback defined as
def errback(self, failure), the response can be accessed though
fail=False, the request callback is called instead.
In both cases, the response could have its body truncated: the body contains
all bytes received up until the exception is raised, including the bytes
received in the signal handler that raises the exception. Also, the response
object is marked with
"download_stopped" in its
fail is a keyword-only parameter, i.e. raising
StopDownload(True) will raise