python - pyspider调试的时候没有任何问题,点run就报编码问题

问题描述

调试的时候没有任何问题,点run就报编码问题。同样2个采集就这一个老报错,另外一个完全没问题

taskid

d7221a2be620c4ef60e874a1d93e79d1

lastcrawltime

1499144004.75187 (20 minutes ago)

updatetime

1499144004.7518892 (20 minutes ago)

exetime

1499144014.7518687 (20 minutes ago)

track.fetch 1.32ms

{ 'content': '', 'encoding': null, 'error': '’ascii’ codec can’t encode character ’uff09’ in position 94: ordinal not in range(128)', 'headers': {}, 'ok': false, 'redirect_url': null, 'status_code': 599, 'time': 0.0013222694396972656}

track.process 0.83ms

’ascii’ codec can’t encode character ’uff09’ in position 94: ordinal not in range(128) = self.gen.throw(*exc_info) File '/root/workspaces/pyspider3/lib/python3.5/site-packages/pyspider/fetcher/tornado_fetcher.py', line 378, in http_fetchresponse = yield gen.maybe_future(self.http_client.fetch(request)) File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/gen.py', line 1055, in runvalue = future.result() File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/concurrent.py', line 238, in resultraise_exc_info(self._exc_info) File '<string>', line 4, in raise_exc_info File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py', line 214, in _process_queuecurl.info['headers']) File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py', line 306, in _curl_setup_requestfor k, v in request.headers.get_all()]) Exception: ’ascii’ codec can’t encode character ’uff09’ in position 94: ordinal not in range(128){ 'exception': '’ascii’ codec can’t encode character ’uff09’ in position 94: ordinal not in range(128)', 'follows': 0, 'logs': ' = self.gen.throw(*exc_info)n File '/root/workspaces/pyspider3/lib/python3.5/site-packages/pyspider/fetcher/tornado_fetcher.py', line 378, in http_fetchnresponse = yield gen.maybe_future(self.http_client.fetch(request))n File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/gen.py', line 1055, in runnvalue = future.result()n File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/concurrent.py', line 238, in resultnraise_exc_info(self._exc_info)n File '<string>', line 4, in raise_exc_infon File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py', line 214, in _process_queuencurl.info['headers'])n File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py', line 306, in _curl_setup_requestnfor k, v in request.headers.get_all()])n Exception: ’ascii’ codec can’t encode character ’uff09’ in position 94: ordinal not in range(128)n', 'ok': false, 'result': null, 'time': 0.0008292198181152344}

schedule

{ 'age': 10, 'exetime': 1499144014.7518687, 'retried': 3}

process

{ 'callback': 'index_page'}

fetch

{}

问题解答

headers设置有问题,我删掉直接秒好.

#!/usr/bin/env python# -*- encoding: utf-8 -*-

看看你的代码前两行是这个吧