python requests模块和 urllib.request模块
例子:
>>> r=requests.get("http://www.baidu.com") >>> r.status_code 200 >>> r.encoding 'ISO-8859-1' >>> r.apparent_encoding 'utf-8' >>> r.text '<!DOCTYPE html>\r\n<!--STATUS OK--><html> <head><meta http-equiv=content-type content=text/html;charset=utf-8>ipt> <a href=//www.baidu.com/more/ name=tj_briicon class=bri >æ\x9b´å¤\x9a产å\x93\x81</a> </div> </div> </div> <div >>> r.encoding='utf-8' >>> r.text '<!DOCTYPE html>\r\n<!--STATUS OK--><html> <head><meta http-equiv=content-type content=text/html;charset=utf-8><meta http-equiv=X-UA-Compatible content=IE=Edge><meta chref=http://s1.bdstatic.com/r/www/cache/bdorz/baidu.min.css="h读</a> <a href=http://jianyi.baidu.com/ class=cp-feedback>意见反馈</a> 京ICP证030173号 <img src=//www.baidu.com/img/gs.gif> </p> </div> </div> </div> </body> </html>\r\n'
参考:https://blog.csdn.net/pittpakk/article/details/81218566
Python3中urllib合并了Python2中的urllib和urllib2.
比如urllib2.Request和urllib2.urlopen 在python3中对应urllib.reques中的 Request和urlopen
对应兼容python2和python3的导入方法可以参考如下方法:
try:
from urllib.request import Request, urlopen
except:
from urllib2 import Request, urlopen
对于request,urlopen 使用方法
headers = {"Content-Type":"aplication/json"}
url = "http://www.abc.com/post"
req = request(url, headers=headers)
此处也可以使用add_header方法来添加header
data = {"a":"1"}
res = urlopen(req, data=json.dumps(data).encode("utf-8"))
参考网址(proxy, cookie): https://blog.csdn.net/qq_43546676/article/details/88777227