Request Module Python

Request Module Python

requests
Requests allow you to send HTTP/1.1 requests. You can add headers, form data, multipart files, and parameters with simple Python dictionaries, and access the response data in the same way.
>>> import requests
#Let's try to get a webpage http://hackaholic.info
>>> r = requests.get("http://hackaholic.info")

Now, we have a Response object called r. We can get all the information we need from this object.
What about the other HTTP request types: PUT, DELETE, HEAD and OPTIONS? These are all just as simple:
>>> r = requests.put('http://httpbin.org/put', data = {'key':'value'})
>>> r = requests.delete('http://httpbin.org/delete')
>>> r = requests.head('http://httpbin.org/get')
>>> r = requests.options('http://httpbin.org/get')

Head option:
The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response. The metainformation contained in the HTTP headers in response to a HEAD request SHOULD be identical to the information sent in response to a GET request. This method can be used for obtaining metainformation about the entity implied by the request without transferring the entity-body itself. This method is often used for testing hypertext links for validity, accessibility, and recent modification.
The response to a HEAD request MAY be cacheable in the sense that the information contained in the response MAY be used to update a previously cached entity from that resource. If the new field values indicate that the cached entity differs from the current entity (as would be indicated by a change in Content-Length, Content-MD5, ETag or Last-Modified), then the cache MUST treat the cache entry as stale.

Passing Parameters In URLs
You often want to send some sort of data in the URL’s query string. e.g. httpbin.org/get?key=val. Requests allows you to provide these arguments as a dictionary of strings, using the params keyword argument.
>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.get('http://httpbin.org/get', params=payload)
>>>print(r.status_code)
200
>>> print(r.text)
{"args":{"key1":"value1","key2":"value2"},"headers":{"Accept":"*/*","Accept-Encoding":"gzip, deflate","Connection":"close","Host":"httpbin.org","User-Agent":"python-requests/2.18.4"},"origin":"182.75.82.78","url":"http://httpbin.org/get?key1=value1&key2=value2"}
>>> print(r.url)
http://httpbin.org/get?key1=value1&key2=value2

Note that any dictionary key whose value is None will not be added to the URL’s query string.

You can also pass a list of items as a value:
>>> payload = {'key1': 'value1', 'key2': ['value2', 'value3']}
>>> r = requests.get('http://httpbin.org/get', params=payload)
>>> print(r.url)
http://httpbin.org/get?key1=value1&key2=value2&key2=value3

Response Content
We can read the content of the server’s response
>>> r = requests.get('https://api.github.com/events')
>>> r.text

When you make a request, Requests makes educated guesses about the encoding of the response based on the HTTP headers. The text encoding guessed by Requests is used when you access r.text. You can find out what encoding Requests is using, and change it, using the r.encoding property:

JSON Response Content
There’s also a builtin JSON decoder, in case you’re dealing with JSON data:
>>> r.json()

Custom Headers
If you’d like to add HTTP headers to a request, simply pass in a dict to the headers parameter. For example, we didn’t specify our user-agent in the previous example:
>>> url = 'https://api.github.com/some/endpoint'
>>> headers = {'user-agent': 'my-app/0.0.1'}
>>> r = requests.get(url, headers=headers)

More complicated POST requests
Typically, you want to send some form-encoded data — much like an HTML form. To do this, simply pass a dictionary to the data argument. Your dictionary of data will automatically be form-encoded when the request is made:

Note: params form the query string in the URL, data is used to fill the body of a request (together with files). GET and HEAD requests have no body.

>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.post("http://httpbin.org/post", data=payload)
>>> print(r.text)

We can also use tuples. Example
>>> payload_tuples = [('key1', 'value1'), ('key1', 'value2')]

POST a Multipart-Encoded File
Requests makes it simple to upload Multipart-encoded files:

How to get mime-type (content-type of a file)
$ file –mitme-type filename

$ file --mime-type spam.log
spam.log: text/plain

Mime-types: Click Here

>>> url = 'http://httpbin.org/post'
>>> files = {'file': open('report.txt', 'rb')}
>>> r = requests.post(url, files=files)

You can set the filename, content_type and headers explicitly:
files = {'file':('test.txt', open('report.txt','rb'),'text/plan')}

In the event you are posting a very large file as a multipart/form-data request, you may want to stream the request. By default, requests does not support this, but there is a separate package which does – requests-toolbelt. You should read the toolbelt’s documentation for more details about how to use it.

Response Status Codes
We can check the response status code:

>>> r = requests.get('http://httpbin.org/get')
>>> r.status_code
200

Requests also comes with a built-in status code lookup object for easy reference: requests.codes

If we made a bad request (a 4XX client error or 5XX server error response), we can raise it with Response.raise_for_status():

>>> r = requests.get('http://httpbin.org/status/404')
>>> r.status_code
404
>>> r.raise_for_status()
Traceback (most recent call last):
File "requests/models.py", line 832, in raise_for_status
raise http_error
requests.exceptions.HTTPError: 404 Client Error

If status_code for r is 200, when we call raise_for_status() we get: None.

Redirection and History
By default Requests will perform location redirection for all verbs except HEAD.We can use the history property of the Response object to track redirection.The Response.history list contains the Response objects that were created in order to complete the request. The list is sorted from the oldest to the most recent response.

For example, GitHub redirects all HTTP requests to HTTPS:
>>> r = requests.get('http://github.com')
>>> r.url
'https://github.com/'
>>> r.status_code
200
>>> r.history
[]

you can disable redirection handling with the allow_redirects parameter:
>> r = requests.get('http://github.com', allow_redirects=False)
>>> r.status_code
301
>>> r.history
[]

Timeouts
You can tell Requests to stop waiting for a response after a given number of seconds with the timeout parameter. Nearly all production code should use this parameter in nearly all requests. Failure to do so can cause your program to hang indefinitely:

>>> requests.get('http://github.com', timeout=0.001)
Traceback (most recent call last):
File "", line 1, in
requests.exceptions.Timeout: HTTPConnectionPool(host='github.com', port=80): Request timed out. (timeout=0.001)

Note: timeout is not a time limit on the entire response download; rather, an exception is raised if the server has not issued a response for timeout seconds (more precisely, if no bytes have been received on the underlying socket for timeout seconds). If no timeout is specified explicitly, requests do not time out.

Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *