Ravi Raj

The best way to do the projects on BigData & Hadoop is to leverage the open data sets available across the web. These datasets are available across domains like IMDB datasets, NY Taxi datasets, Wikipedia data sets etc. These are available in different sizes as well (from few mbs to 100s of gbs). Understanding the datasets structure and then defining the problem you want to pursue becomes very essential in planning to do the project. Another important factor to consider is - what module you want to use (like Hive, Spark or MapReduce) or any other related projects like Kafka, Flume you want to use to execute the project in. Also what is the infrastructure that is available at your dispose (like only VM or a cluster on AWS or Azure) helps in identifying and executing the project. Hope it gives a high level idea. Ravi