Append files in Hadoop?

Can I really append a file in HDFS OR do i need to always replace the file? http://hadoop4mapreduce.blogspot.com/2012/08/two-methods-to-append-content-to-file.html

Compression in Hadoop

I am currently researching various options to do compression in Hadoop and found the following articles useful:

• http://comphadoop.weebly.com/index.html
  
• Slide share = http://www.slideshare.net/Hadoop_Summit/kamat-singh-june27425pmroom210cv2
  
• Splitable LZO = http://blog.cloudera.com/blog/2009/11/hadoop-at-twitter-part-1-splittable-lzo-compression/
  

My project needs are = should have minimum performance impact + should give decent compression gains. It looks like my choice is splitable LZO.

Hadoop 2.0 Yarn Explained

http://www.ibm.com/developerworks/library/bd-hadoopyarn/

Cascading for Workflows in Hadoop

Cascading allows workflows in Hadoop with lingual that makes sql based pipelining

MapReduce in Eclipse

Good article on how to setup eclipse for MapReduce development http://www.thecloudavenue.com/2012/10/debugging-hadoop-mapreduce-program-in.html

Hadoop Summit Gartner Presentation - Hadoop Adoption

HDInsight Getting Started Link

I have been reading up on Microsoft HDInsight which is a new Hadoop Distribution in the market. It has a cloud service that is very easy to use and has a unique feature that allows one to save the data on low cost storage and later bring the Hadoop cluster back online when needed. Here is a getting started on this... http://www.windowsazure.com/en-us/manage/services/hdinsight/get-started-hdinsight/#header-7

Hadoop Summit 2013

The annual Hortonwork's Hadoop Summit is just a few days away and should showcase latest product releases and roadmap from the Hadoop Ecosystem. Look out for recorded video sessions and slides in this website. http://hadoopsummit.org/san-jose/

Big Data on CNBC

Good to see BigData and Hadoop getting limelight. This is a platform change for several IT teams and will transforms the business.

Enterprise Information Management Approach

Information management and enterprise architecture

View more PowerPoint from nvvrajesh

Datameer Analytics on Big Data

This allows visualization on Hadoop and seems to be the best out there doing this

Datameer Analytics Solution

View more presentations from templedf

Pentaho ETL software for Hadoop

Pentaho ETL tool now makes it really easy to build MapReduce jobs for Hadoop

http://www.youtube.com/watch?v=KZe1UugxXcs&feature=colike

Open Source ETL comparision

I agree to some of the analysis here. The future for open source integration looks bright, the competition is good. Pentaho and Talend are front runners. Talend has the advantage of being deployed onto any Java server but Pentaho has more users and is slightly easier to use.

Open Source ETL vs Commercial ETL

View more presentations from JonathanL

Oracle's BI Strategy and OBIEE Overview

Oracle business intelligence overview

View more presentations from nvvrajesh

Doing BI the Agile Way

BI the Agile Way

View more presentations from nvvrajesh

Hadoop Links

thanks to Sameer Farooqui from Linkedin..

++ MapReduce Framework ++
Great 1 hour video introduction: http://nosqltapes.com/video/understanding-mapreduce-with-mike-miller

Read the famous 2004 paper from Google that kicked off the MapReduce revolution. This is a very readable paper that can be digested in about 2 - 3 hours:http://research.google.com/archive/mapreduce.html

Here's a 33 minute video on what kinds of simple things you can do with MapReduce:
http://www.cloudera.com/videos/mapreduce_algorithms

Google's MapReduce course:
http://code.google.com/edu/parallel/mapreduce-tutorial.html


++ Beginner Hadoop ++
Excellent beginner's video on understanding Hadoop, MapReduce and HDFS:
http://www.cloudera.com/protected/?resource=introduction-to-apache-mapreduce-and-hdfs

Understanding the Hadoop ecosystem:
http://www.cloudera.com/protected/?resource=apache-hadoop-ecosystem


++ HDFS ++
An easy 2-3 hour read about Hadoop's distributed File System:
http://www.aosabook.org/en/hdfs.html


++ Labs ++
Install VirtualBox on your laptop, get an Ubuntu Virtual Machine going and follow this excellent tutorial to install your first Hadoop node:
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/

Then use this to scale your cluster to multiple nodes:
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/

Run a MapReduce job in Python on your cluster:
http://www.michael-noll.com/tutorials/writing-an-hadoop-mapreduce-program-in-python/


++ Bonus ++
Dynamo/Cassandra is a good alternative to Hadoop/HBase and is worth as least being familiar with: http://nosqltapes.com/video/understanding-dynamo-with-andy-gross

Hadoop real time

http://www.slideshare.net/parallellabs/sigmod-realtime-hadooppresentation

http://www.rhipe.org/home

Oracle Strategy on Endeca

This is a good presentation on strategy...

• Endeca allows business analyst to explore data
• Endeca has hooks to Hadoop and has a smart cube builder
• Oracle bought this last year and has plans to integrate these tools...

http://www.oracle.com/us/corporate/acquisitions/endeca/general-presentation-517133.pdf

Hadoop... short presentation

Hadoop For Enterprises

View more presentations from nvvrajesh

Good link for Oracle DBA related tips