Monday, 15 July 2013

Learning to Process Big Data with MapReduce and Hadoop - Hands-On Exercises

I recently ran a hands-on session at the SICSA Summer School on Big Data Information Visualisation on MapReduce and Hadoop. The aim was to teach students how to process Big Data in a fast scalable manner using the  MapReduce programming paradigm. The students started with a simple word counting program, learned its structure and how to deploy it both locally and to a Amazon Web Services Elastic MapReduce Cluster. They then learned how to modify MapReduce programs to find top hashtags in Twitter data and design output formats to target another application, i.e. Twitter search using the open source Terrier Information Retrieval platform.

The session was highly successful and as such I am making the exercise sheets and source code freely available. The exercise sheets assume that you have some familiarity with Java programming, are running either Linux or Mac and if you want to use Elastic MapReduce then you have already created an account (if not then I suggest you run through the video tutorials available here).

Download the Lab Code-Base Now
Download the Exercise Sheets Now
  • This is a PDF containing the three exercise sheets.

95 comments:

Seyed Jalil Kazemitabar said...

Hello,

The executable file hadoop in hadoop-1.1.2/bin generates an error massage as follows :

Exception in thread "main" java.lang.NoClassDefFoundError: &

I receive this for and commands. How can I fix this?

Thanks

sundara rami reddy said...

Really good piece of knowledge, I had come back to understand regarding your website from my friend Sumit, Hyderabad And it is very useful for who is looking for HADOOP.I browse and saw you website and I found it very interesting.Thank you for the good work, greetings.
Hadoop Training in hyderabad

jack wilson said...

Your posts is really helpful for me.Thanks for your wonderful post. I am very happy to read your post. It is really very helpful for us and I have gathered some important information from this blog.


Big Data Training in Chennai

ramya parvathaneni said...

Hi,
The best information and we are providing online training with all modules
hadoop online training
with real time

Hadoop online training said...

Hi,
Thanks for nice data provided easy way to learn training with real time on
hadoop online training
through the experienced experts

Jhon Mick said...

Uniqe informative article and of course True words, thanks for sharing. Today I see myself proud to be a hadoop professional with strong dedication and will power by blasting the obstacles. Thanks to Hadoop Training in Chennai

dhanamlakshmi palu said...

Your posts is really helpful for me.Thanks for your wonderful post. I am very happy to read your post. AWS Training in chennai | AWS Training chennai | AWS course in chennai

Suranka VMware said...

Thanks for sharing this valuable information to our vision. You have posted a trust worthy blog keep sharing.
VMWare Training in chennai | VMWare Training chennai | VMWare course in chennai

surangacloud said...

Nice article i was really impressed by seeing this article, it was very interesting and it is very useful for me..
Cloud Computing Training in chennai | Cloud Computing Training chennai | Cloud Computing Course in chennai | Cloud Computing Course chennai

JpaSolutions.in said...

Fortunately, Apache Hadoop is a tailor-made solution that delivers on both counts, by turning big data insights into actionable business enhancements for long-term success. To know more, visit Big data Training Bangalore

Jhon Mick said...

Thanks for this guide.

Hadoop training chennai

Paul Miller said...

I was struggling a lot with mapreduce concept but this was very useful to me. Thanks for sharing.

Hadoop Training Chennai
Hadoop Training in Chennai

Rohini T said...

Excellent post, I agree with you 100%! I’m always scouring the oracle for new information and learning whatever I can, and in doing so I sometimes leave comments on blogs.Oracle Training In Chennai

Rohini T said...

if possible share with us...Very useful information is providing by ur blog.here is a way to find Oracle Training In Chennai

Rohini T said...

i wondered keep share this sites .if anyone wants realtime training Greens technolog chennai in Adyar visit this blog..performance tuning training In Chennai and more Oracle Training In Chennai

Rohini T said...

once again sharing this informative blog .Datastage training In Chennai It uses a graphical notation to construct data integration solutions and is available in various versions may visit greens technology chennai in adyar Greens Technologys Training In Chennai

Rohini T said...

As your information sybase very nice its more informative and gather new ideas implemnted thanks for sharing this blog sybase training In Chennai if want to get details now Greens Technologies Training In Chennai

Rohini T said...

i gain the knowledge of Java programs easy to add functionalities play online games, chating with others and industry oriented coaching available from greens technology chennai in Adyar may visit.Core java training In Chennai

Rohini T said...

I have read your blog and I got very useful and knowledgeable information from your blog. It’s really a very nice article Greens Technologies Training In Chennai

Pooja Doss said...

Oracle DBA Training in Chennai
Thanks for sharing this informative blog. I did Oracle DBA Certification in Greens Technology at Adyar. This is really useful for me to make a bright career..

Pooja Doss said...

Whatever we gathered information from the blogs, we should implement that in practically then only we can understand that exact thing clearly, but it’s no need to do it, because you have explained the concepts very well. It was crystal clear, keep sharing..
Websphere Training in Chennai

Pooja Doss said...

Data warehousing Training in Chennai
I am reading your post from the beginning, it was so interesting to read & I feel thanks to you for posting such a good blog, keep updates regularly..

Pooja Doss said...

Selenium Training in Chennai
Wonderful blog.. Thanks for sharing informative blog.. its very useful to me..

Pooja Doss said...

Oracle Training in chennai
Thanks for sharing such a great information..Its really nice and informative..

Pooja Doss said...

I have read your blog and i got a very useful and knowledgeable information from your blog.You have done a great job.
SAP Training in Chennai

Pooja Doss said...

This information is impressive..I am inspired with your post writing style & how continuously you describe this topic. After reading your post,thanks
for taking the time to discuss this, I feel happy about it and I love learning more about this topic
Android Training In Chennai In Chennai

Pooja Doss said...

Pretty article! I found some useful information in your blog, it was awesome to read,thanks for sharing this great content to my vision, keep sharing..
Unix Training In Chennai

Pooja Doss said...

I found some useful information in your blog, it was awesome to read, thanks for sharing this great content to my vision, keep sharing..
SalesForce Training in Chennai

Pooja Doss said...

There are lots of information about latest technology and how to get trained in them, like Best Hadoop Training In Chennai have spread around the web, but this is a unique one according to me. The strategy you have updated here will make me to get trained in future technologies Hadoop Training in Chennai By the way you are running a great blog. Thanks for sharing this blogs..

Dinju Thomas said...

Oracle Training in chennai | Oracle D2K Training In chennai
This information is impressive; I am inspired with your post writing style & how continuously you describe this topic. After reading your post, thanks for taking the time to discuss this, I feel happy about it and I love learning more about this topic..

Dinju Thomas said...

Thanks for Information Oracle Apps Technical is a collection of a bunch of collected applications like accounts payables, purchasing, inventory, accounts receivables, human resources, order management, general ledger and fixed assets, etc which have its own functionality for serving the business
Oracle Apps Training In Chennai

Dinju Thomas said...

This information is impressive; I am inspired with your post writing style & how continuously you describe this topic. After reading your post, thanks for taking the time to discuss this, I feel happy about it and I love learning more about this topic..
Selenium Training in Chennai | QTP Training in Chennai

Neethu Sharma said...

I am very impressed with the article I have just read,so nice.......
QTP Training In Chennai | Selenium Training in Chennai | Oracle Training in Chennai

Neethu Sharma said...

Excellent information with unique content and it is very useful to know about the information based on blogs.
Hadoop Training In Chennai | oracle apps financials Training In Chennai | advanced plsql Training In Chennai

Andrew Son said...

Learning new technology would give oneself a true confidence in the current emerging Information Technology domain. With the knowledge of big data the most magnificent cloud computing technology one can go the peek of data processing. As there is a drastic improvement in this field everyone are showing much interest in pursuing this technology. Your content tells the same about evolving technology. Thanks for sharing this.

Hadoop Training in Chennai | Big Data Hadoop Training in Chennai | Hadoop Course in Chennai | Hadoop training institutes in chennai

Jhon Mick said...

Truely a very good article on how to handle the future technology. This content creates a new hope and inspiration within me. Thanks for sharing article like this. The way you have stated everything above is quite awesome. Keep blogging like this. Thanks :)

Software testing training in chennai | Software testing training institutes in chennai | Software testing training institute in chennai

Arjun kumar said...

I have finally found a Worth able content to read. The way you have presented information here is quite impressive. I have bookmarked this page for future use. Thanks for sharing content like this once again. Keep sharing content like this.

Software testing training in chennai | Software testing training institutes in chennai | Manual testing training in Chennai

Arjun kumar said...

I have finally found a Worth able content to read. The way you have presented information here is quite impressive. I have bookmarked this page for future use. Thanks for sharing content like this once again. Keep sharing content like this.

Software testing training in chennai | Software testing training | Manual testing training in Chennai

Jhon Abraham said...

Testing is the best in Information Technolgohy compared to anyother domain. Thanks for sharing your knowledge via this content. You have presented it very well. Thank you once again. I have bookmarked this page for future use. Keep blogging article like this.

Selenium training in chennai | Best selenium training institute in chennai | Manual testing training in Chennai

for IT the said...

Great Article
Java Online Course | Java EE Training


Java Training Institutes in Chennai | java j2ee training institutes in chennai | Java Training in Chennai | J2EE Training in Chennai | Java Course in Chennai


Java Interview Questions | Java Training Institutes | IT Technical Articles

Mervin Parmar said...

Using big data analytics may give the companies many fruitful results, the findings can be implemented in their business decisions so as to minimize their risk and to cut the costs.
hadoop training in chennai|big data training|big data training in chennai

Raghu Sharma said...

Cloud computing is the next big thing, through cloud the users have the liberty to use a shared network. The companies can focus on core business parts rather than investing heavily on infrastucture.
cloud computing training in chennai|cloud computing courses in chennai|cloud computing training

Divit said...


Hey, nice site you have here! Keep up the excellent work!

Hadoop Certification in Chennai

Dhivya Shree said...

Salesforce.com is an american company which offfers CRM based cloud services and it is loved globally for it quality services
salesforce training in chennai|salesforce training institute in chennai | salesforce course in chennai

prathap kumar said...

Interesting Article

Java Training in Chennai | Online Java Training

Dhivya Shree said...

SAS stands for statistical analysis system which is a analysis tool developed by SAS institute and with the help of this tool data driven decisions can be taken which is helpful for the bsuiness.
SAS training in Chennai | SAS course in Chennai | SAS training institute in Chennai

Arjun kumar said...
This comment has been removed by the author.
Jhon Abraham said...

Nice article. Hadoop is a Java-based programming framework.Your article gives more knowledge.
Thanks
Hadoop courses in chennai | Hadoop Training in chennai




Arjun kumar said...
This comment has been removed by the author.
Arjun kumar said...

Excellent article. Hadoop is a cloud based tool.It give more information about massive storage and it helps to improve our skills. Hadoop provides more job opportunities.To achieve a great career join with us. Hadoop Training Chennai
Regards,
Arjun

Bay Max said...

Your post is really useful to all.Engineers have a good career in today's IT industry.Update regularly.
Thanks
Hadoop Training chennai | Hadoop Training in chennai



Arthur Mac said...

It is good and easy to learn.Your online training is really useful.It is more useful for IT professionals.Keep blogging always.
Regards,
Hadoop Training in Chennai | Bigdata Hadoop Training in Chennai

Deeksha Dawar said...

I read lot of articles and really like this article. This information is definitely useful for everyone in daily life. Fantastic job.ERP Training in noida

caroline jesi said...

In near future, big data handling and processing is going to the future of IT industry. Thus taking Hadoop Training in Chennai | Big Data Training in Chennai will prove beneficial for talented professionals.

mythily mythu said...
This comment has been removed by the author.
mythily mythu said...
This comment has been removed by the author.
mythily mythu said...

Nice...
informatica training in chennai

1croreprojects said...
This comment has been removed by the author.
1croreprojects said...

Thanks for sharing this valuable information.
ieee java projects in chennai
ieee dotnet projects in chennai
mba projects in chennai
be projects in chennai
ns2 projects in chennai
mca projects in chennai
bulk projects in chennai
bsc projects in chennai
msc projects in chennai

Iqbal Dlk said...

very nice and informative blog
dot net training in chennai
java training in chennai

ramesh rams said...

big data hadoop online training placement
hadoop admin online training
Learn job oriented Big data hadoop Online training and Certification Course and become Expert in big data hadoop .

ch.abhishek reddy said...

Thanks for information
online training

Rekha J said...

This is my first visit to your blog, your post made productive reading, thank you. dot net training in chennai

A1trainings said...

hadoop online training by real-time experts visit A1trainings

Hadoop training in india


Kaviya R said...

This blog is very informative about the concepts involved in hadoop and its scope in future. Interesting concepts on its architectute and syllabus which are covered by big data hadoop training institute in Chennai that is functioning effectively.

yasar said...

very nice and informative blog
big data projects chennai
mobile computing projects chennai
cloud computing projects chennai
secure computing projects chennai

Sgraph Infotech said...

Big data and map reduce is an interesting concepts to know and this article had helped me to know more about it
thank you for posting it.

Sgraph Infotech Bangalore

jahan bdreamz said...

This information is impressive; I am inspired with your post writing style & how continuously you describe this topic.
Thanks for sharring..,

Hadoop Training Training in chennai

isabellaJoseph said...

Amazing post.Thanks for sharing that information.We are giving more Dot Net Training in Chennai for corporates which employees of their company will be benefits with our training.Get more details courses details......... Selenium Training in Chennai
Hadoop Training in Chennai

Shreeja K said...

Informative article, just what I was looking for.seo services chennai

raj kumar said...

Hello Admin,
I really enjoyed while reading your article, the information you have mentioned in this post was damn good. Keep sharing your blog with updated and useful information.
Regards,
Hadoop Training in Bangalore

Amar G said...

very nice

raj kumar said...
This comment has been removed by the author.
raj kumar said...

I have read your blog and i got a very useful and knowledgeable information from your blog.You have done a great job . If anybody want to learn Hadoop Training in Bangalore.


Hadoop Training in Bangalore

soumya Reddy said...

Nice Blog, very good information Hadoop admin Online Training Bangalore

soumya Reddy said...

Check it through Hadoop admin Online Training Hyderabad for more info.

nutana meka said...

Check it once through for Hadoop admin Online Training Bangalore for more info.

nishanth said...

Nice Blog Thank you for sharing such an good blog.
Devops Training in Bangalore
itEanz

Unknown said...

excellent blog python training in bangalore

Papumoni Gogoi said...

Thanks for posting this Type of Blog Keep itup
Iot Training in Bangalore
iteanz

Unknown said...

Great information. Thank you for sharing

Unknown said...

Very nice post. Awesome article... Really helpful

mojah said...

Nice Blog
Iot Training in Bangalore
Iteanz

Swathi Samala said...

The article is very easy to under stand Hadoop Admin Online Training Bangalore

Lucy singh said...

Thanks for sharing such a valuable information.This post is very useful for me.
Big Data and Hadoop Administrator Certification Training
.Thanks lots for sharing them with us.

Unknown said...

usefull article. Thanks for sharing

Praveen said...

Nice Interview Questions
SQL Interview Questions
BPO Interview Questions
C Interview Questions
Spring Interview Questions
Iot Interview Questions
HTML5 Interview Questions

Gokul Ravi said...

very nice blog it was useful

Gokul Ravi said...

very nice blog It was useful

dushyant sharma said...

Webtrackker is the best web development company in Noida
Website Development Company in Noida,
Website Development Company in Noida sector 64,
Website Development Company in Noida sector 63,
Website Development Company in Noida sector 62,
Website Development Company in Noida sector 58,
Website Development Company in Noida sector 55,



dushyant sharma said...

Webtrackker is the best web development company in Noida

Website Development Company in Noida sector 5,
Website Development Company in Noida sector 37,
Website Development Company in Noida sector 2,
Website Development Company in Noida sector 18,
Website Development Company in Noida sector 15,

sunil kumar dubey said...

Delhi Education Institute,Deiedu

online mba in India,
mba course in India

correspondence mba in india

distance mba in india

Mba in fast track

Distance education in india

Engineering courses in india

Engineering degree in india

Unknown said...

very informative blog

Unknown said...

Really nice post.

Unknown said...

Good blog. Thank you for sharing.

Post a Comment

newer post older post Home