Hadoop集群配置

《Hadoop集群配置》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。

1、Hadoop集群配置1准备工作1.1配置主机名修改每台服务器上的/etc/hosts文件，添加主机名配置。192.168.0.75master192.168.0.76slave1192.168.0.78slave2192.168.0.79slave31.2安装JDK1.3创建运行Hadoop程序的用户在系统中添加运行Hadoop程序的用户hadoop，并修改其登录密码为hadoop。[root@bgu0004~]#useraddhadoop[root@bgu0004~]#passwdhadoopChangingpass

2、wordforuserhadoop.NewUNIXpassword:BADPASSWORD:itisbasedonadictionarywordRetypenewUNIXpassword:passwd:allauthenticationtokensupdatedsuccessfully.1.4配置SSH无密钥验证配置为了实现无口令SSH登录，需要在主机上配置公私密钥方式登录。此时需要通过root帐号登录，修改每个主机上的/etc/ssh/sshd_config文件，去掉RSAAuthenticationyes和Pubk

3、eyAuthenticationyes前面的“#”。RSAAuthenticationyesPubkeyAuthenticationyes#AuthorizedKeysFile.ssh/authorized_keys然后使用1.3节中创建的hadoop重新登录系统，在每台主机上运行ssh-keygen-trsa命令生成本地的公私密钥对。[hadoop@bgu0001~]$ssh-keygen-trsaGeneratingpublic/privatersakeypair.Enterfileinwhichtosavethe

4、key(/home/hadoop/.ssh/id_rsa):Createddirectory'/home/hadoop/.ssh'.Enterpassphrase(emptyfornopassphrase):Entersamepassphraseagain:Youridentificationhasbeensavedin/home/hadoop/.ssh/id_rsa.Yourpublickeyhasbeensavedin/home/hadoop/.ssh/id_rsa.pub.Thekeyfingerprintis:

5、2f:da:4d:f3:00:e2:c3:87:19:b0:50:fe:43:47:08:a7hadoop@bgu0001同样的过程需要在3台slave主机上执行。然后通过scp命令将3台slave主机的公钥拷贝到master主机上。[hadoop@bgu0002~]$scp.ssh/id_rsa.pubmaster:/home/hadoop/.ssh/slave1.pubTheauthenticityofhost'master(192.168.0.75)'can'tbeestablished.RSAkeyfinger

6、printis72:cd:14:f2:a2:d1:f0:e3:9e:a0:db:93:e7:5e:68:ee.Areyousureyouwanttocontinueconnecting(yes/no)?yesWarning:Permanentlyadded'master,192.168.0.75'(RSA)tothelistofknownhosts.hadoop@master'spassword:id_rsa.pub100%3960.4KB/s00:00拷贝之后master主机hadoop用户home目录下.ssh中应

7、该如下所示：[hadoop@bgu0001~]$cd.ssh/[hadoop@bgu0001.ssh]$lsid_rsaid_rsa.pubslave1.pubslave2.pubslave3.pub其中id_rsa.pub内容如下，是用户hadoop在master主机上的公钥；而slave1.pub、slave2.pub和slave3.pub分别是slave主机的公钥。[hadoop@bgu0001.ssh]$catid_rsa.pubssh-rsaAAAAB3NzaC1yc2EAAAABIwAAAQEA7oaJO2

8、WrLGEcWw2sU1+MvNSKnhaRm0ZM8Ee4FO0364gInYm794dX9acgcpIlFyfdHOLKhqDDKT4bUjwD8xaFnHaNXebKJRLPDsDMI+VqVVJ9DxnQaVkA+0+u+kA9KdWKJ2Sxg9eqLnNyHqDfhfiQGACOX8zI9uDcDFmyQINt6V9h