Categories
Programming Python

How to get data from twitter using Tweepy in Python?

To start working on Python you need to have Python installed on your PC. If you haven’t installed python. Go to the Python website and get it installed.

After installing Python set up your twitter account if you don’t have one already. Next, go to Developer Page and apply for the Developer Access, then fill the form and accept the developer agreement.

You can create the developer account according to your area of interest. I have created using Student.

After Submission, you’ll receive an email for a confirmation.

Confirmation Email

After confirming the email. Your account will be in a review phase and you’ll receive an email of approval after the review.

Note:

Approval time may vary from one to another.

Once Your Application gets accepted. Create an App to access the tweets.

After, creating the App Successfully go to Keys and Tokens in your app to access the keys.

Access Twitter API in Python

Install Tweepy by using the following command if you haven’t installed it already.

pip install tweepy

Once you have installed Tweepy. Import the relevant libraries to your Python file.

import csv
import tweepy as tw
import time
Code language: JavaScript (javascript)

To access the data of tweets you need to have 4 keys from the Twitter app page. You can get the keys from the Key Access Token Tab and define them in your python file as below.

consumer_key= 'your_consumer_key'
consumer_secret= 'your_consumer_secret'
access_token= 'your_access_token'
access_token_secret= 'your_access_token_secret'

auth = tw.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tw.API(auth, wait_on_rate_limit=True)
Code language: PHP (php)

Append the twitter data to an existing file by using the method open(file, mode) as shown below :

Parameter mode values

"r"Β – Read – Default value. Opens a file for reading, error if the file does not exist

"a"Β – Append – Opens a file for appending, creates the file if it does not exist

"w"Β – Write – Opens a file for writing, creates the file if it does not exist

"x"Β – Create – Creates the specified file, returns an error if the file exists

csvFile = open('Filename.csv', 'a')
csvWriter = csv.writer(csvFile)
Code language: JavaScript (javascript)

Next, you need to implement Tweepy cursor to fetch the tweets via a for loop to fetch 1000 tweets.

  • ID
  • User Name
  • Text
  • Created at
  • User Location

search_terms ='*'
# IF YOU WANT TO USE MULTIPLE KEYWORDS THEN USE OR IN BETWEEN AS : search_terms = 'bishrulhaq OR BH'

count= 0
for tweet in tw.Cursor(api.search,
                       q=search_terms,
                       since='2020-05-01', until='2020-05-10',
                       count=5000,
                       result_type='recent',
                       include_entities=True,
                       monitor_rate_limit=True,
                       wait_on_rate_limit=True,
                       lang="en").items():
    try:
        count = count + 1
        print ("No of Tweet: %d" %count)

        csvWriter.writerow([tweet.id, tweet.user.screen_name.encode('utf8'), tweet.text.encode('utf-8'), tweet.user.location.encode('utf8')])
        # change the count values as the number of tweets you need to fetch
        if count == 10000:
            break

    except IOError:
        time.sleep(60)
        continue

print ("Total Tweets Fetched %d" %count)
csvFile.close()

Code language: PHP (php)

Finally, the code will look like this,

import csv
import tweepy as tw
import time

# BH | Bishrul Haq

consumer_key= 'your_consumer_key'
consumer_secret= 'your_consumer_secret'
access_token= 'your_access_token'
access_token_secret= 'your_access_token_secret'

auth = tw.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tw.API(auth, wait_on_rate_limit=True)

csvFile = open('extracted_tweets.csv', 'a')
csvWriter = csv.writer(csvFile)

search_terms ='*' 
# IF YOU WANT TO USE MULTIPLE KEYWORDS THEN USE OR IN BETWEEN AS : search_terms = 'bishrulhaq OR BH'

count= 0

for tweet in tw.Cursor(api.search,
                       q=search_terms,
                       since='2020-05-01', until='2020-05-10',
                       count=5000,
                       result_type='recent',
                       include_entities=True,
                       monitor_rate_limit=True,
                       wait_on_rate_limit=True,
                       lang="en").items():
    try:
        count = count + 1
        print ("No of Tweet: %d" %count)

        csvWriter.writerow([tweet.id, tweet.user.screen_name.encode('utf8'), tweet.text.encode('utf-8'), tweet.user.location.encode('utf8')])
        # change the count values as the number of tweets you need to fetch
        if count == 10000:
            break

    except IOError:
        time.sleep(60)
        continue

print ("Total Tweets Fetched %d" %count)
csvFile.close()
Code language: PHP (php)

Categories
Python

Getting Started with Django

Django is an open-source, high-level and free Web framework created using Python. It follows the Model View Template(MVT) architectural pattern. In this tutorial you’ll learn to setup a Django 2.2 project from scratch.

Why Django?

Django is one of the best framework available today. It acts as a quick solution for web development and delivers high-quality code with better security. Django takes it very seriously and helps to avoid SQL injection, cross-site scripting etc. Let’s look at some of the core features in Django

  1. Faster : It’s considered to be one of the fastest framework built by python language.It encourages rapid development with a clean and pragmatic design.
  2. More Packages : Django comes with many components that helps to develop faster and easier.No need of downloading any components separately as Django installs all the extras, packages and the related dependencies.
  3. Versatile : Django is used to develop all sorts of applications – from basic development to complex development. Therefore, Django is extremely versatile in all fields.
  4. Secure : Django is as secure as any web framework can be. It provides tools to prevent common mistakes causing security problems.
Installing Django

#Note

Make sure you have installed Python in your PC. Click onΒ Install PythonΒ If you haven’t installed Python on your PC/ Mac.

Type the following command at your terminal to install Django on your PC/ Mac

pip install Django==2.2
Starting a new Django application

Now that we have completed the basic setup, to start Django application. Let’s start the Project.

django-admin startproject sampleProject

After initiating the project. Type the following command to get into the project directory

cd sampleProject

After creating the project, you will find a list of files inside the project directory as below.

|-- sampleProject
|   |-- manage.py
|   |-- sampleProject
|   |   |--_init_.py
|   |   |--settings.py
|   |   |--urls.py
|   |   |--wsgi.py
  1. _init_.py : Init just tells the python that this is to be treated like a python package.
  2. settings.py : Manages all the settings of your project.
  3. urls.py : The main controller which maps it to the website.
  4. wsgi.py : Serves as an entry point for WSGI compatible web servers.
|-- sampleProject
|   |-- manage.py
|   |-- sampleProject
|   |-- app
|   |   |--migrations
|   |   |  |--_init_.py
|   |   |--_init_.py
|   |   |--admin.py
|   |   |--apps.py
|   |   |--models.py
|   |   |--tests.py
|   |   |--views.py

Type the following command at your terminal to create the list of files inside the app as shown above.

python manage.py startapp app
Code language: CSS (css)

Now add the following code to the views.py file which will return a httpResponse

app/views.py

from django.http import HttpResponse
def index(request):
 return HttpResponse("<h1 st
Code language: JavaScript (javascript)

To map the view with the URL, add the following code in urls.py

sampleProject/urls.py

from django.contrib import admin
from django.urls import path
from app.views import index

urlpatterns = [
    path('admin/', admin.site.urls),
    path('',index),
]
Code language: JavaScript (javascript)

Now It’s time to run the application, To start the server simply type the command show below at your terminal.

python manage.py runserver
Code language: CSS (css)

Feel free to drop your thoughts at the comment section!