Real time twitter Analysis with Hadoop on Hortonworks Sandbox 2.5

ANALYSING AND VISUALISING TWEETS WITH APACHE NIFI AND HDP SEARCH (SOLR, BANANA DASHBOARD) VIDEO TUTORIAL SUPPLEMENT.

PRE-REQUISITES 

  1. Hortonworks Sandbox HDP, with the following services installed: 
    • Apache NiFi
    • Solr
    • Download Here
  1. NiFi twitter template
  1. Twitter application
    • Consumer Key
    • Consumer Secret
    • Access Token
    • Access Token Secret

COMMANDS USED IN TUTORIAL:

su solr

cp -r /opt/lucidworks-hdpsearch/solr/server/solr/configsets/data_driven_schema_configs /opt/lucidworks-hdpsearch/solr/server/solr/configsets/tweet_configs

vi /opt/lucidworks-hdpsearch/solr/server/solr/configsets/tweet_configs/conf/solrconfig.xml

/ParseDateFieldUpdateProcessorFactory

<str>EEE MMM d HH:mm:ss Z yyyy</str>

Press ‘esc’ key on your keyboard

:wq

cd /opt/lucidworks-hdpsearch/solr/server/solr-webapp/webapp/banana/app/dashboards/

mv default.json default.json.orig

wget https://raw.githubusercontent.com/abajwa-hw/ambari-nifi-service/master/demofiles/default.json

/opt/lucidworks-hdpsearch/solr/bin/solr create -c tweets -d tweet_configs -s 1 -rf 1 -p 8983

exit

http://localhost:8983/solr/banana/index.html

q: language_s:en

sort: screenName_sasc

rows: 150

fl: screenName_s,text_t

wt: csv

Advertisements