Apache Ambari provides an end-to-end management and monitoring application for Apache Hadoop . With Ambari, you can deploy and operate a complete Hadoop stack using a graphical user interface (GUI), manage configuration changes, monitor services, and create alerts for all the nodes in your cluster from a central point.
In this article, we will use Apache Ambari (Version 2.2.2.0 ) fromthe Hortonworks Data Platform (HDP) in our discussion.
Hortonworks Data Platform
You can deploy Hortonworks Data Platform (HDP) using either Apache Ambari or not. If you choose not to use Ambari, you can follow the instructions here . However, it will be much easier to deploy Apache Hadoop stack with Ambari (see the instruction here ).
After initial installation and deployment, your Apache Hadoop cluster could still grow and change with use over the time. With Apache Ambari, you can easily and quickly add new services or expand the storage and processing capacity of the cluster.
The ecosystem of Ambari consists of three main components:
Ambari Web Ambari Server Serves as the collection point for data from across the cluster Ambari Agent Run on each host in the cluster to allow the Ambari Server to control it
Ambari Web
Using the Ambari Web UI and REST APIs, you can deploy, operate, manage configuration changes, and monitor services for all nodes in your cluster from a central point.
Ambari Web is a client-side javascript application, which calls the Ambari REST API (accessible from the Ambari Server) to access cluster information and perform cluster operations. A relational database is used to store the information about the cluster configuration and topology.
With Ambari Views , you can customize the Ambari Web UI. Ambari Views offer a systematic way to plug-in UI capabilities to surface custom visualization, management and monitoring features in Ambari Web.
Ambari Server Before starting the Ambari Server, you must set up the Ambari Server once. Setup configures Ambari to talk to the Ambari database, installs the JDK and allows you to customize the user account (default: root) the Ambari Server daemon will run as.
After setup,all the configuration is stored in:
/etc/ambari-server/conf/ ambari.propertiesThen you can run the following commands from the Ambari Server host:
ambari-server start Ifyou reboot your cluster, you must restart the Ambari Server and all the Ambari Agents manually. ambari-server status ambari-server stopOnce started, you can access Ambari using the following URL:
from a web browser.
The start script /usr/sbin/ambari-server is a shell script, that set environment variables and kicks off a python script which kicks off a java process (see details here ).
Trobleshooting
You can start ambari in debug mode to get more detailed output via:
ambari-server start --verbose --debug # or for short ambari-server start -v -gSignificant files/directories:
/var/log/ambari-server/ambari-server.log To monitor Ambari, you do tail -f /var/log/ambari-server/ ambari-server.log /var/lib/ambari-server/resources/ SQL scripts to initialize psql DB
Ambari Agent
Ambari Agents will heartbeat to the master every few seconds and will receive commands from the master in the heartbeat responses. Heartbeat responses will be the only way for master to send a command to the Agent. The command will be queued in the action queue , which will be picked up by the action executioner.
Action executioner will pick the right tool (Puppet, Python, etc) for execution depending on the command type and action type. Thus the actions sent in the hearbeat response will be processed asynchronously at the Agent. The action executioner will put the response or progress messages on the message queue . The Agent will send everything on the message queue to the master in the next heartbeat.
Here are the steps you do to install Ambari Agent manually on RHEL/CentOS/Oracle linux 6:
Install the Ambari Agent on every host in your cluster. yum install ambari-agent Using a text editor, configure the Ambari Agent by editing the ambari-agent.ini file as shown below:vi /etc/ambari-agent/conf/ ambari-agent.ini
[server] hostname= Start the Agent on every host in your cluster. ambari-agent start The Agent registers with the Server on start.The Agent should not die if the master suddenly disappears. It should continue to poll at regular intervals and recover as needed when the master comes back up:
The Ambari Agent should keep all the necessary information it planned to send to the master in case of a connection failure and re-send the information after the master comes back up. It may need to re-register if it was previously in the process of registering.
Troubleshooting
The first thing to do if you run into trouble is to find the logs. Ambari Agent logs can be found at /var/log/ambari-agent/ ambari-agent.logReferences Non-Ambari Cluster Installation Guide