HIVE_HOME/bin EOF cat >> ~/.bash_profile <<'EOF' export JAVA_HOME=/srv/java export HADOOP_HOME=/srv/apache-hadoop [root@localhost ~]$ su - hadoop [hadoop@localhost ~]$ /srv/apache-hadoop/bin/hdfs dfs -mkdir -p /user /hive/warehouse [hadoop@localhost ~]$ /srv/apache-hadoop/bin/hdfs dfs -mkdir -p /tmp/hive [hadoop@localhost ~]$ /srv/apache-hadoop/bin/hdfs dfs -chmod g+w /user/hive/warehouse [hadoop@localhost ~]$ /srv/apache-hadoop 访问 Hive 启动 Hadoop [hadoop@localhost ~]$ /srv/apache-hadoop/sbin/start-all.sh This script is Deprecated
@hadoop01 conf]$ vim sqoop-env.sh # 在hadoop的cdh版本中,把4个组件(commom、hdfs、mapreduce、yarn)安装到4个不同的目录中 # 而apache-hadoop 的4个组件都在一个目录下 # 根据自己的版本配置这里的环境变量,我是apache-hadoop,所以都用HADOOP_HOME即可 export HADOOP_COMMON_HOME=/home/hadoop
dfs -put ~/alice.txt /test-path hdfs dfs -ls /test-path more ops, view https://sparkbyexamples.com/apache-hadoop
/本地测试,正式环境需要注释掉// Date date = new Date();//保证输出的目录不重复 args = new String[]{"D:\\ideawork\\bigdata\\apache-hadoop \\src\\main\\resources\\mapreduce\\input\\word.txt", "D:\\ideawork\\bigdata\\apache-hadoop\\src\\main /本地测试,正式环境需要注释掉// Date date = new Date();//保证输出的目录不重复 args = new String[]{"D:\\ideawork\\bigdata\\apache-hadoop \\src\\main\\resources\\mapreduce\\input\\word.txt", "D:\\ideawork\\bigdata\\apache-hadoop\\src\\main
Hadoop生态圈的Spark(https://www.cloudera.com/products/open-source/apache-hadoop/apache-spark.html),一夜之间成为默认的数据处理引擎
https://www.cloudera.com/products/open-source/apache-hadoop/key-cdh-components.html Hortonworks
3、容量调度器:CapacityScheduler.class,这是 Apache-HADOOP 默认的调度。