在使用scrapy-redis报

Unhandled error in Deferred:

Traceback (most recent call last):
  File "D:\Programes\Python\Python36\lib\site-packages\scrapy\crawler.py", line 194, in crawl
    return self._crawl(crawler, *args, **kwargs)
  File "D:\Programes\Python\Python36\lib\site-packages\scrapy\crawler.py", line 198, in _crawl
    d = crawler.crawl(*args, **kwargs)
  File "D:\Programes\Python\Python36\lib\site-packages\twisted\internet\defer.py", line 1613, in unwindGenerator
    return _cancellableInlineCallbacks(gen)
  File "D:\Programes\Python\Python36\lib\site-packages\twisted\internet\defer.py", line 1529, in _cancellableInlineCallbacks
    _inlineCallbacks(None, g, status)
--- <exception caught here> ---
  File "D:\Programes\Python\Python36\lib\site-packages\twisted\internet\defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "D:\Programes\Python\Python36\lib\site-packages\scrapy\crawler.py", line 88, in crawl
    yield self.engine.open_spider(self.spider, start_requests)
builtins.TypeError: __init__() got an unexpected keyword argument 'encoding'

是由于版本问题造成的,在scrapy_redis\defaults.py和connect.py中将encoding注释

REDIS_PARAMS = {
    'socket_timeout': 30,
    'socket_connect_timeout': 30,
    'retry_on_timeout': True,
    #'encoding': REDIS_ENCODING,
}

更多推荐