Study

NTT DATA creates a high-speed, big data–based analysis of the relationship between stock prices and
social media sentiments with the Intel® Xeon® processor E7 v2 family
Challenges
• High-speed identication of positive and negative tweets in 35 months of social media data
containing tens of billions of tweets
• Build an analysis system for providing a real-time Twitter* sentiment index
Solutions
• Leverage the high-performing Intel® Xeon® processor E7 v2 family to develop a big data
analytic solution
• Develop a Twitter sentiment index for nancial markets
Impact
• Validated the usefulness and value of a social media sentiment index for nancial services
companies using historical social media data, and completed sentiment analysis in one day
• Validated the ability to provide a real-time, distributed social media analytic system with the Intel
Xeon processor E7 v2 family
Using Real-Time Analysis to Build a Financial
Sentiment Index with the Intel® Xeon® Processor E7 v2 Family
“As hardware advances,
analyses that were unthinkable
10 years ago come into reach.
We intend to continue working
with Intel to supply services
with high added value to
our customers.”
– Keiichirō Nakagawa,
Director,
NTT DATA Mathematical Systems, Inc.
Financial Sentiment Analysis of a
Complete Twitter Data Set
NTT DATA Corporation, a member of NTT
Group, operates a system solutions business,
while its aliate NTT DATA Mathematical
Systems Inc. is involved in consulting, as well
as business analytics package development
and contracting. The two companies have
jointly developed a Twitter sentiment index
solution for nancial markets. This solution
is comprised of systems based on the Intel
Xeon processor E7 v2 family. It extracts
stock-related tweets from Twitter data and
assesses marketing strength or weakness.
The system demonstrates the usefulness
of social media sentiments in actual
market trading.
Information delivery services that use Twitter
or other social network service data are
operating in the United States, where use of
big data for nancial analysis is widespread—
but there is also growing demand in Japan
for such services. NTT DATA signed a contract
with Twitter, Inc. in September 2012 for
the acquisition and resale of all Twitter
data tweeted in Japanese or within Japan.
Customers for this data include marketing
companies and the news media. “When
considering new information services using
Twitter data, we directed our attention at
nancial markets where interest is high,
and so we decided to develop this index,”
explains Keiichirō Nakagawa, a director at
NTT DATA Mathematical Systems.
Calculated from all the Twitter data in
Japan, the Twitter sentiment index is a
numerical indicator of positive or negative
sentiments expressed in tweets relating to the
stock market.
The rst step in analyzing the Twitter data is
to scan the complete data set, consisting of
several tens of billions of tweets, for those
tweets that contain stock-related terms such
as “Nikkei” or company names such as “NTT
DATA.” After narrowing down the Twitter data
to about one percent of its initial size, the
second step brings this total down another
two-thirds by identifying those tweets that
contain positive or negative terms, such as
“The Nikkei is rising” or “It seems too good to
be true.” The Twitter sentiment index is then
dened as positive or negative accordingly.
“We needed a faster IT platform for the
positivity/negativity identication step in
which a text-knowledge extraction technique
developed by NTT Group is applied to the
large number of tweets,” explains Nakagawa.
The companies performed a test by extracting
stock-related tweets from 35 months of
Twitter data from January 2011 to November
2013, and then analyzing their relationship
with stock market indicators. The analysis
found a statistically signicant correlation
Case Study
Intel® Xeon® Processor E7 v2 Family
Big Data
Financial Services