Hadoop Notebook

Hadoop 测试

MRUnit unit test Mapper and Reducer classes independently run in memory, single-threaded PipelineMapReduceDriver

LocalJobRunner single-threaded, and only one Reducer can be started conf.set("mapred.job.tracker", "local"); conf.set("fs.default.name", "file:////"); FileSystem fs = FileSystem.getLocal(conf);

MiniMRCluster, MiniYarnCluster, MiniDFSCluster multithreaded