Our blog

Twitter gives MIT researchers access to every single public tweet

Source: theverge.com Tags search, social media, unstructured data

Twitter is opening up its vault for researchers at MIT's Media Lab, and it's giving them $10 million to try and glean some insights from the deluge of tweets. The team of researchers, working under the title "Laboratory of Social Machines," will have access to Twitter's highly valuable "fire hose" of live public tweets, as well as archives of every public 140-character-or-less utterance posted to the service since its founding in 2006. With the raw data, the group intends to "create new platforms for both individuals and institutions to identify, discuss, and act on pressing societal problems," according to a statement from MIT.

.

.

The group's goals are fairly vague at this point, but the researchers appear to be focusing on magnifying consensus on social media platforms like Twitter, where it can be difficult to draw conclusions from hundreds of thousands of opinions. As the Laboratory of Social Machines explains, "Pattern discovery and data visualization will be explored to reveal interaction patterns and shared interests in relevant social systems." The group adds: "collaborative tools and mobile apps will be developed to enable new forms of public communication and social organization." Among those tools could be ways for journalists to more systematically tap into the fire hose of tweets and identify trends.

"IT’S HARDER TO HARNESS [TWITTER] INTO SUSTAINED CHANGE."

.

Deb Roy, an associate professor at MIT's Media Lab who'll lead the group, tells Bloomberg Businessweek that one particular area of interest is to understand why public forums like Twitter don't frequently foster productive change. "It’s better at disrupting or stopping things, or having your voice heard," he says. "It’s harder to harness that into sustained change."

.

This is hardly the first time data analysts or academic institutions have shown interest in tapping into Twitter's massive stores of data for research purposes. However, it is the first time Twitter itself has financially supported one of those efforts. In a statement, Twitter CEO Dick Costolo said that the move is an "opportunity to go deeper into research to understand the role Twitter and other platforms play in the way people communicate."

Tags

• ai/ artificial intelligence
• analytics
• big data
• competitive intelligence
• content analytics
• darpa
• deep web
• future
• ibm
• intelligence
• investigations
• knowledge base
• search
• security
• social media
• unstructured data
• visualization
• Watson