网页爬虫 - Python requests爬虫，如何post payload里的数据，谢谢！

问题描述

我用requests做post请求的时候，怎么用这个payload呢？data数据要怎么构造form-data的数据，我会。

import requestsimport jsonurl=’http://tx3.cbg.163.com/cgi-bin/equipquery.py?act=show_overall_search’html = requests.post(url,data=data)

问题解答

根据楼上两位解答，经过修改已经顺利爬出数据

# -*- coding: utf-8 -*-import requestsimport sysimport jsonreload(sys)sys.setdefaultencoding(’utf-8’)def get_tx3cbg(): url = ’http://tx3.cbg.163.com/cgi-bin/search.py’ headers = {’Content-Type’:’application/x-www-form-urlencoded; charset=UTF-8’,’Cookie’:’overall_search_equips_cookie=%7C89_212365%7C23_211970%7C9_204469%7C2_156154%7C192_28326%7C176_74442%7C15_480910%7C15_477817%7C2_156122%7C10_156067%7C10_133849%7C15_481140%7C78_425212%7C12_165185%7C161_222125; usertrack=ZUcIhliu8JNB01gjA64jAg==; _ntes_nnid=f90c5442c5912e2adc252657cb12d457,1487860049616; _ntes_nuid=f90c5442c5912e2adc252657cb12d457; NTES_CMT_USER_INFO=35428995%7C%E5%BE%A1%E5%89%91%E6%8A%8A%E9%85%92%E5%90%AC%E7%A7%8B%E9%9B%A8%7Chttps%3A%2F%2Fsimg.ws.126.net%2Fe%2Fimg5.cache.netease.com%2Ftie%2Fimages%2Fyun%2Fphoto_default_62.png.39x39.100.jpg%7Cfalse%7CZmFuZ3dlaXJlbjg0M0AxNjMuY29t; UM_distinctid=15b0f71ba430-0e2166208-581f3413-1fa400-15b0f71ba4415a; vjuids=28a3aef4a.15b1fcdeaec.0.5e87609c; vjlast=1490887240.1490887240.30; vinfo_n_f_l_n3=c384af10aeab8944.1.0.1490887240441.0.1490887272552; P_INFO=fangweiren843@163.com|1491746466|1|blog|11&16|zhj&1491645222&gamebbs#zhj&330500#10#0#0|157600&0|gamebbs&tx2&mail163&blog|fangweiren843@163.com; _ga=GA1.2.69335350.1487859926; latest_views=15_466493-89_212755-1_217134-19_190513; __utma=187553192.69335350.1487859926.1492699270.1492699270.1; __utmz=187553192.1492699270.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __oc_uuid=65296ee0-25d7-11e7-be4c-213889b11650; last_login_serverid=19; area_td_id=1; overall_sid=000bEiuYYQVly-GWAHJ3BgbcS4ypMSxBavIhk6yVALj’,’Referer’:’http://tx3.cbg.163.com/cgi-bin/equipquery.py?act=show_overall_search’,’User-Agent’:’Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36’ } payload = {’act’:’overall_search_role’,’order_by’:’’,’page’:’1’,’other_arg’:’’,’price_min’:’50000’,’price_max’:’60000’ } html = requests.post(url,data=payload,headers=headers).text html = json.loads(html) print html get_tx3cbg()回答2：

将 request payload 的数据改造成 dict，传给data

常查文档！http://docs.python-requests.o...

>>> payload = {’key1’: ’value1’, ’key2’: ’value2’}>>> r = requests.post('http://httpbin.org/post', data=payload)>>> print(r.text){ ... 'form': { 'key2': 'value2', 'key1': 'value1' }, ...}