Twitter Analysis with Hadoop on Hortonworks Sandbox 2.5

ANALYSING AND VISUALISING TWEETS WITH APACHE NIFI AND HDP SEARCH (SOLR, BANANA DASHBOARD) VIDEO TUTORIAL SUPPLEMENT.

PRE-REQUISITES 

  1. Hortonworks Sandbox HDP, with the following services installed: 
    • Apache NiFi
    • Solr
    • Download Here
  1. NiFi twitter template
  1. Twitter application
    • Consumer Key
    • Consumer Secret
    • Access Token
    • Access Token Secret

COMMANDS USED IN TUTORIAL:

su solr

cp -r /opt/lucidworks-hdpsearch/solr/server/solr/configsets/data_driven_schema_configs /opt/lucidworks-hdpsearch/solr/server/solr/configsets/tweet_configs

vi /opt/lucidworks-hdpsearch/solr/server/solr/configsets/tweet_configs/conf/solrconfig.xml

/ParseDateFieldUpdateProcessorFactory

<str>EEE MMM d HH:mm:ss Z yyyy</str>

Press ‘esc’ key on your keyboard

:wq

cd /opt/lucidworks-hdpsearch/solr/server/solr-webapp/webapp/banana/app/dashboards/

mv default.json default.json.orig

wget https://raw.githubusercontent.com/abajwa-hw/ambari-nifi-service/master/demofiles/default.json

/opt/lucidworks-hdpsearch/solr/bin/solr create -c tweets -d tweet_configs -s 1 -rf 1 -p 8983

exit

http://localhost:8983/solr/banana/index.html

q: language_s:en

sort: screenName_sasc

rows: 150

fl: screenName_s,text_t

wt: csv

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s