我使用请求库来捕获重定向的URL。让我用以下代码演示这一点:
import requests
try:
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36',}
response = requests.get('https://www.mooc-list.com/go.php?courseId=3502', timeout=3, headers=headers)
response.raise_for_status()
except requests.exceptions.HTTPError as errh:
print ("Http Error:",errh)
except requests.exceptions.ConnectionError as errc:
print ("Error Connecting:",errc)
except requests.exceptions.Timeout as errt:
print ("Timeout Error:",errt)
except requests.exceptions.RequestException as err:
print ("Oops : Something Else",err)我得到了以下输出:
错误连接: HTTPSConnectionPool(host='hub0.ecolearning.eu',port=443):最大重试超过url: /course/ with一步步-2ed/(由ConnectTimeoutError(“hub0.ecolearning.eu超时连接”)引起)
但是,当我试图通过执行print(response.url)打印URL时,会出现一个名称错误,如下所示
NameError:未定义名称“响应”
这基本上意味着当连接失败时不会初始化response对象,因此我无法捕获URL历史记录或重定向。
我不介意连接失败,但我想提取URL重定向。这件事有什么好转吗?
谢谢!:)
发布于 2018-10-29 02:08:20
如果我们不能打开重定向到的url。我们可以尝试从http头中找到Location。所以我选择在requests上停止自动重定向,并建立一个新的重定向器。
import requests
def Final_location(url):
try:
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36',}
response = requests.get(url, timeout=3 , allow_redirects=False , headers=headers)
except requests.exceptions.HTTPError as errh:
print ("Http Error:",errh)
return url
except requests.exceptions.ConnectionError as errc:
print ("Error Connecting:",errc)
return url
except requests.exceptions.Timeout as errt:
print ("Timeout Error:",errt)
return url
except requests.exceptions.RequestException as err:
print ("Oops : Something Else",err)
return url
if response.headers.get("Location"):
return Final_location(response.headers.get("Location"))
else:
return response.url
#Location = Final_location(response.headers.get("Location")) if response.headers.get("Location") else response.url
#return Location
print(Final_location('https://www.mooc-list.com/go.php?courseId=3502'))输出:
Error Connecting: HTTPSConnectionPool(host='hub0.ecolearning.eu', port=443): Max retries exceeded with url: /course/smooc-step-by-step-2ed/ (Caused by ProxyError('Cannot connect to proxy.', timeout('_ssl.c:817: The handshake operation timed out',)))
https://hub0.ecolearning.eu/course/smooc-step-by-step-2ed/#发布于 2020-04-04 16:52:13
def ReqGet():
try :
resp = requests.get(url, params=params, headers = headers,timeout=3)
return resp
except Exception as e
return e
res = ReqGet()
print(res.request.url)Error对象有最后一个重定向请求,您可以从该请求获取url。
https://stackoverflow.com/questions/53032125
复制相似问题