Hadoop集群安装-1

2019-12-17 18:04:45

一、完全分布式模式的安装和配置的具体步骤：

1.配置jdk；2.配置hosts文件；3.建立hadoop运行账号；4.配置ssh免密码连入；

5.下载并解压hadoop安装包；6.配置namenode，修改site文件；7.配置hadoop-env.sh；

8.配置masters和slaves文件；9.向各节点复制hadoop；10.格式化namenode；

11.启动hadoop；12.用jps检验各后台进程是否成功启动

1.配置jdk，分别在各节点配置

1----首先把压缩包解压出来---- 2[root@localhost ~]# tar-zxvf jdk-7u9-linux-i586.tar.gz 3 4----修改目录名---- 5[root@localhost ~]# mv jdk1.7.0_09/jdk1.7 6 7----在/etc/profile文件中添加下面几行---- 8[root@localhost ~]# vi/etc/profile 910export JAVA_HOME=/jdk1.711export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib12export PATH=$JAVA_HOME/bin:$PATH1314----验证是否已经成功安装jdk1.7----15[root@localhost ~]# java-version16java version "1.7.0_09"17Java(TM) SE Runtime Environment (build1.7.0_09-b05)18Java HotSpot(TM) Client VM (build23.5-b02, mixed mode)

2.配置hosts文件，分别在各节点配置

1[root@localhost 123]# cat/etc/hosts2# Donotremove the following line,or various programs3# that require network functionality will fail.4127.0.0.1 localhost.localdomain localhost5::1 localhost6.localdomain6 localhost667192.168.1.151 node18192.168.1.152 node29192.168.1.153node3

3.建立hadoop运行账号，分别在各个节点进行配置

1[root@localhost ~]# useradd jack2[root@localhost ~]# passwd jack3Changing passwordforuser jack.4New UNIX password: 5BAD PASSWORD: itis too short6Retype new UNIX password: 7passwd:all authentication tokens updated successfully.8[root@localhost ~]# id jack9uid=500(jack) gid=500(jack) groups=500(jack)

4.配置ssh免密码连入

1[jack@node1 ~]$ ssh-keygen-t rsa 2Generatingpublic/private rsakey pair. 3Enterfileinwhichtosavethekey(/home/jack/.ssh/id_rsa): 4Created directory'/home/jack/.ssh'. 5Enter passphrase (emptyfor no passphrase): 6Enter same passphrase again: 7Your identification has been savedin/home/jack/.ssh/id_rsa. 8Yourpublickeyhas been savedin/home/jack/.ssh/id_rsa.pub. 9Thekeyfingerprintis:1065:22:5b:af:69:09:7b:8f:8b:35:f6:b8:69:8c:f0:a1 jack@node11112[jack@node2 ~]$ ssh-keygen-t rsa13Generatingpublic/private rsakey pair.14Enterfileinwhichtosavethekey(/home/jack/.ssh/id_rsa): 15Created directory'/home/jack/.ssh'.16Enter passphrase (emptyfor no passphrase): 17Enter same passphrase again: 18Your identification has been savedin/home/jack/.ssh/id_rsa.19Yourpublickeyhas been savedin/home/jack/.ssh/id_rsa.pub.20Thekeyfingerprintis:21ab:18:29:89:57:82:f8:cc:3c:ed:47:05:b2:15:43:56jack@node22223[jack@node3 ~]$ ssh-keygen-t rsa24Generatingpublic/private rsakey pair.25Enterfileinwhichtosavethekey(/home/jack/.ssh/id_rsa): 26Created directory'/home/jack/.ssh'.27Enter passphrase (emptyfor no passphrase): 28Enter same passphrase again: 29Your identification has been savedin/home/jack/.ssh/id_rsa.30Yourpublickeyhas been savedin/home/jack/.ssh/id_rsa.pub.31Thekeyfingerprintis:3211:9f:7c:81:e2:dd:c8:44:1d:8a:24:15:28:bc:06:78jack@node33334[jack@node1 ~]$ cd .ssh/35[jack@node1 .ssh]$ cat id_rsa.pub> authorized_keys3637[jack@node2 ~]$ cd .ssh/38[jack@node2 .ssh]$ scp id_rsa.pub node1:/home/jack/39The authenticityofhost'node1 (192.168.1.151)'can't be established.40RSA key fingerprint is 51:ac:0e:ec:9c:ec:60:ac:53:19:20:bc:e4:a6:95:64.41Are you sure you want to continue connecting (yes/no)? yes42Warning: Permanently added 'node1,192.168.1.151' (RSA) to the list of known hosts.43jack@node1's password: 44id_rsa.pub3920.4KB/s00:004546[jack@node1 .ssh]$ cat/home/jack/id_rsa.pub>> authorized_keys 4748[jack@node3 ~]$ cd .ssh/49[jack@node3 .ssh]$ scp id_rsa.pub node1:/home/jack/50The authenticityofhost'node1 (192.168.1.151)'can't be established.51RSA key fingerprint is 51:ac:0e:ec:9c:ec:60:ac:53:19:20:bc:e4:a6:95:64.52Are you sure you want to continue connecting (yes/no)? yes53Warning: Permanently added 'node1,192.168.1.151' (RSA) to the list of known hosts.54jack@node1's password: 55id_rsa.pub3920.4KB/s00:005657[jack@node1 .ssh]$ cat/home/jack/id_rsa.pub>> authorized_keys 5859[jack@node1 .ssh]$ ls60authorized_keys id_rsa id_rsa.pub61[jack@node1 .ssh]$ rm id_rsa.pub 62[jack@node1 .ssh]$ scp authorized_keys node2:/home/jack/.ssh/63The authenticityofhost'node2 (192.168.1.152)'can't be established.64RSA key fingerprint is 51:ac:0e:ec:9c:ec:60:ac:53:19:20:bc:e4:a6:95:64.65Are you sure you want to continue connecting (yes/no)? yes66Warning: Permanently added 'node2,192.168.1.152' (RSA) to the list of known hosts.67jack@node2's password: 68authorized_keys11761.2KB/s00:0069[jack@node1 .ssh]$ scp authorized_keys node3:/home/jack/.ssh/70The authenticityofhost'node3 (192.168.1.153)'can't be established.71RSA key fingerprint is 51:ac:0e:ec:9c:ec:60:ac:53:19:20:bc:e4:a6:95:64.72Are you sure you want to continue connecting (yes/no)? yes73Warning: Permanently added 'node3,192.168.1.153' (RSA) to the list of known hosts.74jack@node3's password: 75authorized_keys11761.2KB/s00:0076[jack@node1 .ssh]$ chmod400 authorized_keys 7778[jack@node2 .ssh]$ rm id_rsa.pub 79[jack@node2 .ssh]$ chmod400 authorized_keys 8081[jack@node3 .ssh]$ rm id_rsa.pub 82[jack@node3 .ssh]$ chmod400 authorized_keys 83[jack@node3 .ssh]$ ssh node284The authenticityofhost'node2 (192.168.1.152)'can't be established.85RSA key fingerprint is 51:ac:0e:ec:9c:ec:60:ac:53:19:20:bc:e4:a6:95:64.86Are you sure you want to continue connecting (yes/no)? yes87Warning: Permanently added 'node2,192.168.1.152' (RSA) to the list of known hosts.88Last login: Wed May 15 21:57:50 2013 from 192.168.1.10489[jack@node2 ~]$