Hadoop cluster configuration 1

1. Login to your ubuntu 

2. Start terminal

3. Firt edit your hostname

          command: sudo gedit /etc/hostname

          change name to "master" and save it.

4. command: hostname 

          It will give old name, to get new name restart the service

5. command: sudo service hostname restart

          If you get unknown then type below command

6. sudo service hostname start

7. command: hostname

          Now it will show new name called master 

8.command: sudo gedit /etc/hosts

          If you get any error then use below command

          $sudo vi /etc/hosts

          If you still get some problem then restart your VM

i.e. logoff and login again it will solve the issue and type below command

          $sudo gedit /etc/hosts

9. comment: ubuntu ip address

          #127.0.0.1 ubuntu

          Now get your IP address using command "ifconfig" and

paste in file and add some name

          eg: 198.126.34.9 master

          Save the file and close the file

10. command: hostname

You will able to see master

Now one machine is read

Now check sshd – it will be use to communicate between master and slave

11. command: which ssh

          output will be /usr/bin/ssh if you get some different output then

sshd is not installed in your machine.

You can install using below command

12. commad : sudo apt-get install ssh

          type Y

13. command : sudo apt-get install sshd

14. Check java is installed or not using command:jps

If not you can use below command to install Java

command: sudo apt-get install openjdk-6-jdk

          Y

Michael noll

(http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster)

15. $ sudo apt-get install python-software-properties

16. $ sudo add-apt-repository ppa:ferramroberto/java

17. $ sudo apt-get update

18. $ sudo apt-get install sun-java6-jdk

19. $ sudo update-java-alternatives -s java-6-sun

Start other instance of machine (yash2Ubuntu)

20. Take the IP address of master using command ifconfig

          e.g. 192.168.139.142

          Ping this IP address in slave machine

          Open terminal CTRL+ALT+T

          $ ping 192.168.139.142

          Should able to ping IP address in slave - require to communicate with master

21. Try to copy the IP address of slave and ping it in master

          yash1Ubuntu

          eg: $ ping 192.160.130.140

          It should ping in master to communicate with slave,

then only you can configure hadoop

22. Open Master(need to communicate through some source)

          command: sudo geedit /etc/hosts

          two things you need

          IP address of Master

          eg (

                192.168.139.142 yeshwanth1.yeshwanthsoft.com yeshwanth1 ->Master

                192.168.139.140 yeshwanth2.yeshwanthsoft.com yeshwanth2 ->Slave

          )

          How many nodes you have can be configured here.

          eg: 192.168.139.143 yeshwanth3.yeshwanthsoft.com yeshwanth3 ->Slave

          IP address of Slave

          Number of slaves will be added here

23. Now go to slave here also need to do same things

          $ sudo gedit /etc/hosts or sudo vi /etc/hosts or sudo vim /etc/hosts

(In vi mode you need to press "Insert key" to insert text)

 Add IP address of mater and slave

          eg (

                192.168.139.142 yeshwanth1.yeshwanthsoft.com yeshwanth1 ->Master

                192.168.139.140 yeshwanth2.yeshwanthsoft.com yeshwanth2 ->Slave

          )

Same IP addresses as present in master 

24. Try to ping using name

          $ ping yeshwanth1

          It should able to ping to communicate master

$ sudo apt-get install python-software-properties

$ sudo add-apt-repository ppa:ferramroberto/java

$ sudo apt-get update

$ sudo apt-get install sun-java6-jdk

Y

$ sudo update-java-alternatives -s java-6-sun

25. CTLR+SHIFT+E to open new terminal

How to configure SSH in your system?

26. $ ssh-keygen -t rsa -P "" (it generates password less ssh key) 

27. Second, you have to enable SSH access to your local machine with this newly created key.

$ cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

28. $ ssh localhost

Yes

exit

29. $ ssh master

Yes

          exit

30. $ sudo opt-get install sun-java6-jdk

Y

31. Now goto master "yeshwanth1"

          $ ssh yeshwanth2

          If will ask for password.

          For password less configuration you need to do below things

          open HOME folder and press CTLR+H it shows hidden files.

          Open .ssh folder

          Open id_rsa.pub and copy the text

32. Now goto slave "yeshwanth2"

          open HOME folder and press CTLR+H it shows hidden files.

          open .ssh folder 

          open authorized_key file

          Copy the public key of master

33. $ ssh yeshwanth1

          It will ask for password

          For password less configuration we need to Copy the public key(id_rsa.pub) 

          of slave and past in authorized key(authorized_key) of master. 

34. $ ssh yeshwanth2

          Now you can able connect master to slave without any password.