Skip to content
TAPoR Text Analysis Portal for Research

Tools

Sort by Name Rating Date ↓

Search

TWINT - Twitter Intelligence Tool

Twint is an advanced Twitter scraping tool written in Python that allows for scraping Tweets from Twitter profiles without using Twitter's API. Twint utilizes Twitter's search operators to let you scrape Tweets from specific users, scrape Tweets relating to certain topics, hashtags & trends, or sort out sensitive information from Tweets like e-mail and phone numbers. I find this very useful, and you can get really creative with it too. Twint also makes special queries to Twitter allowing you to also scrape a Twitter user's followers, Tweets a user has liked, and who they follow without any authentication, API, Selenium, or browser emulation. Recently it looks like Twitter has found a way to limit the number of tweets scraped using twint. Twint also has a visualizer and a desktop app is under development.

Rated 0.0 out of 5.

Capture

MassMine

MassMine is a command line tool designed for researchers to simplify the collection and use of data from online sources such as social media networks. Geared toward novices and power users alike. Designed for use on personal computers or on headless servers. Handles the acquisition of API data while managing rate limits imposed by these services. Makes setting up custom data requests easy, allowing for flexible deployment for individualized research needs. Creates and manages databases of collected data automatically. This means that all tweets, trends, posts, etc. related to your research questions are saved permanently on your local hard drive. Designed for long-running data collection projects. MassMine makes it easy to create your own database of social media content spanning days, months, or years.

Rated 0.0 out of 5.

Natural Language Processing

Tokenization and Extraction of Twitter Data

This notebook is part of a greater series of Juypter Notebook structured around Twitter Tweet analysis. This particular notebook will look at tokenization of extracting key features from a tweet text. This notebook also serves as one of the introductory notebook for TATR as tokenization and extraction are fundamental features for any analysis.

Rated 0.0 out of 5.

Analysis

Panda and CSV of Tweets

TATR: Panda and CSV of Tweets This notebook is part of a greater series of Juypter Notebook structured around Twitter Tweet analysis. This particular notebook will look at loading in a CSV (comma-separated values) file into a panda datastructure. Serving as one of the introductory coding notebook in the series, this notebook aims to teach and showcase how to load and save tweets into CSV files.

Rated 0.0 out of 5.

Visualization

Panda Dataframe Manipulation

TATR: This notebook is part of a greater series of Juypter Notebook structured around Twitter Tweet analysis. This particular notebook will look at Panda dataframe manipulation. It looks closely at panda index, groupby, and count features of panda. This notebook aims to teach these concepts and how they interact with csv.

Rated 0.0 out of 5.

Visualization

Graphing Twitter Data

TATR: Graphing This notebook is part of a greater series of Juypter Notebook structured around Twitter Tweet analysis. This particular notebook will look at graphing. From saving the resulting graph to organization. This notebook will provide the a framework that can be expanded to suit your needs.

Rated 0.0 out of 5.

Visualization

General Inquirer Category

TATR: This notebook is part of a greater series of Juypter Notebook structured around Twitter Tweet analysis. This particular notebook will look at categorizing text using the General Inquirer Categories released by Harvard (http://www.wjh.harvard.edu/~inquirer/spreadsheet_guide.htm). This notebook will showcase how to break down the General Inquirer Category into a dictionary and how it can be tailored to fit your needs.

Rated 0.0 out of 5.

Visualization

Finding Popular Hashtags

TATR: This notebook is part of a greater series of Juypter Notebook structured around Twitter Tweet analysis. This particular notebook will look at finding the most popular hashtag for a given date. This notebook will showcase two main features with the end goal of finding the most popular hashtag by date. The first feature is collapsing all the tokenized hashtag of a date into a larger set. The second feature is to save the "tokenized list" into a csv and load it back such that Panda dataframes can still treat this as a list. This notebook will also provide the a framework that can be expanded to suit your needs.

Rated 0.0 out of 5.

Natural Language Processing

Using Regular Expressions to Clean Up Web Pages

A beginner's notebook on using Regex to strip website content of tags.

Rated 0.0 out of 5.

Natural Language Processing

Using Regular Expressions to Clean a Text

Techniques on using regular expressions to clean up a text for textual analysis.

Rated 0.0 out of 5.