Sec-Art: A Beginners Guide to requests Library in Python : How to Use requests library in Python

Most existing Python modules for sending HTTP requests are extremely verbose and cumbersome. Python’s builtin urllib2 module provides most of the HTTP capabilities you should need, but the api is thoroughly broken. It requires an enormous amount of work (even method overrides) to perform the simplest of tasks.
Requests is an Apache2 Licensed HTTP library, written in Python. It is designed to be used by humans to interact with the language. This means we don’t need have to manually add query strings to URLs, or form-encode your POST data. Requests will allow to send HTTP/1.1 requests using Python. With it, we can add content like headers, form data, multi-part files, and parameters via simple Python libraries. It also allows to access the response data of Python in the same way.

Features :

Keep-Alive & Connection Pooling
Sessions with Cookie Persistence
Browser-style SSL Verification
Basic/Digest Authentication
Elegant Key/Value Cookies
Automatic Decompression
Unicode Response Bodies
Multipart File Uploads
Connection Timeouts
Thread-safety
HTTP(S) proxy support

Installation :

To install requests library use the below command

 sudo apt-get install python-requests

or install with pip

 pip install requests

 easy_install requests

HTTP GET requests :

Start python interactive shell, import requests library by 'import requests' and let start

 re = requests.get('http://sec-art.net')

Now all the information about our request is now stored in a Response object re. Now use the below commands for request details

 re.status_code # to check the status code

We can also check the status code with requests built-in lookup codes

 re.status_code == requests.codes.ok

 re.encoding  # to check encoding property
 re.url   # return rul
 re.text  # shows the response body if it is text (parse it as unicode) 
 re.content  # show the reponse body
 re.headers  # show all the headers as key->value pairs
 re.cookies  # show all cookies

we can also use other HTTP request types, like PUT, DELETE, HEAD and OPTIONS

 r = requests.put("http://httpbin.org/put")
 
 r = requests.delete("http://httpbin.org/delete")
 
 r = requests.head("http://httpbin.org/get")
 
 r = requests.options("http://httpbin.org/get")

Now to see individual headers

 re.headers['Content-Length']  # show the content length
 re.headers['Content-Type'] # show the content type

for other headers just put the name of the header name.

Sending requests with Parameters :

First we need to create a dictonary of parameters

 p = {'name' : 'Frank Castle', 'Age' : '40', 'Occupation' : 'Self-Employed'}
 re = requests.get('http://google.com', params=p)

We can check the url by

 re.url
 u'http://www.google.com/?Age=40&name=Frank+Castle&Occupation=Self-Employed'

Another example

 P = { 'search_query' : 'latest movies'}
 re = requests.get('https://youtube.com', params=p)

 re.url
 u'https://www.youtube.com/?search_query=latest+movies'

HTTP POST Requests :

 import requests
 re = requests.post('https://dvwa.com/Login.php', data={'username':'Frank', 'password':'12345', 'Submit':'Login'})

The post data can be sent as dictonary named data. To send post param in json format

 import json
 import requests
 payload = {'username':'Frank', 'password':'12345', 'Submit':'Login'}
 re = requests.post('https://dvwa.com/Login.php', data=json.dumps(payload))

Sending Custom Headers :

 import requests
 headers = {'Custom1': 'test101', 'Custom2': 'exploit', 'Custom3': 'hello_world'}
 re = requests.post('https://requestb.in/135oaie1', headers=headers)

Sending Custom Cookies :

 import requests
 cookies = {'Custom1': 'test101', 'Custom2': 'exploit', 'Custom3': 'hello_world'}
 re = requests.post('https://requestb.in/135oaie1', cookes=cookies)

Session Objects

To maintain the same session within different requests

 import requests
 session1 = requests.Session()
 re1 = session1.get('http://facebook.com/somepage')      # first request 
 re2 = session1.get('http://facebook.com/someotherpage') # second request

Timeouts

To set timeout in requests

 import requests
 re =  requests.get('http://sec-art.net/', timeout=3)

 re =  requests.get('http://sec-art.net/', timeout=0.001)

For more information visit below links :

http://docs.python-requests.org/en/master/