mac上hadoop环境的坑,有些不知道怎么搞的就突然好了,碰到的错误就不列举了,网上搜搜还是可以很快搞定的,仅作为备份。

启动ssh:

sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist
ssh localhost

启动hadoop套件,常规方式:

# in hadoop home
sbin/start-dfs.sh
sbin/start-yarn.sh
sbin/mr-jobhistory-daemon.sh start historyserver
# http://localhost:8088/cluster/scheduler
# http://192.168.5.5:19888/jobhistory/app

mac下起yarn要这样参看此文

$HADOOP_PREFIX/sbin/yarn-daemon.sh start nodemanager;
$HADOOP_PREFIX/sbin/yarn-daemon.sh start resourcemanager

还有这个问题:

Exception in thread "main" java.io.IOException: Mkdirs failed to create /var/folders/vq/j0q0rr3x7gg91n_gbks6l2z00000gn/T/hadoop-unjar5762729210584301725/META-INF/license
    at org.apache.hadoop.util.RunJar.ensureDirectory(RunJar.java:128)
    at org.apache.hadoop.util.RunJar.unJar(RunJar.java:104)
    at org.apache.hadoop.util.RunJar.unJar(RunJar.java:81)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:209)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

解决方法,pom中添加:

 <plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-shade-plugin</artifactId>
  <version>2.0</version>
  <configuration>
    <shadedArtifactAttached>true</shadedArtifactAttached>
  </configuration>
  <executions>
    <execution>
      <phase>package</phase>
      <goals>
        <goal>shade</goal>
      </goals>
      <configuration>
        <filters>
          <filter>
            <artifact>*:*</artifact>
            <excludes>
              <exclude>META-INF/*.SF</exclude>
              <exclude>META-INF/*.DSA</exclude>
              <exclude>META-INF/*.RSA</exclude>
              <exclude>META-INF/LICENSE*</exclude>
              <exclude>license/*</exclude>
            </excludes>
          </filter>
        </filters>
      </configuration>
    </execution>
  </executions>
</plugin>

也有这样的方法,这个我没试。