requests Module function : Impersonate a browser to send a request .
Use :(requests Module coding process )
- Guide pack import requests
- Appoint url
- Initiate request (get,post)
- Get the data of the response ( page )
- Persistent storage ( Store response data )
Case study : Crawl the page data of Sogou homepage :
# Crawling search dog home page import requests url = "http://www.sogou.com" #step2: Initiate request response=requests.get(url=url) #step3: Get response data .text Return response data of string type page_text = response.text print(page_text) #step4: Persistent storage with open('./sogou.html','w',encoding='utf-8') as fp: fp.write(page_text) print(" End of crawling data ")