○
A scheduler that is responsible for scheduling the tasks of a framework's job, within the accepted
resources.
○
An executor process that is launched on agent/slave nodes to run the framework's tasks.
●
In addition to the aforementioned components, Urika-GX also supports Marathon and
mrun
(the Cray-
developed application launcher) as ecosystem components of Mesos.
mrun
is built upon Marathon
commands for ease of use and running data secure distributed applications. The
mrun
command sets up
resources for CGE and HPC jobs.
On Urika-GX, all tasks launched directly from Marathon need to be run as user
marathon
, and cannot be run
as any other user ID. If a user tries to launch applications/tasks as non-Marathon user, the application will fail
with error “
Not authorized to launch as
userID
”. This behavior has no impact on Hadoop, Spark,
mrun
and/or CGE jobs.
Role of HAProxy
Requests received on the login nodes for the following services are proxied using HAProxy to the Urika-GX
compute nodes:
●
YARN Resource Manager
●
HDFS NameNode
●
Secondary HDFS NameNode
●
Hadoop Application Timeline Server
●
Hadoop Job History Server
●
Spark History Server
●
Oozie
For services like Mesos Masters and Marathon, while there are 3 instances running, one of them is the active
leader. Requests received by the login node are proxied to the currently active leader. If a leader runs into issues,
one of the backup leaders take over and the requests are proxied to the current leader.
HAProxy can be configured to provide SSL. Some possible solutions are documented in the security section of
"Urika
®
-GX System Administration Guide".
Viewing Mesos Metrics from the CLI
Use the Cray-developed
urika-mesos_metrics
script to view Mesos related details. This script is located in
the
/root/urika-tools/urika-cli
directory on the SMW and needs to be run as root.
Following is a sample output of the
urika-mesos_metrics
script:
#
urika-mesos_metrics
HTTP/1.1 200 OK
Proceeding further...
******* MESOS CLUSTER METRICS **********
Total cpus : 984
Used cpus : 0
Master elected : 1
******* MESOS FRAMEWORK METRICS **********
Frameworks active : 1
Frameworks connected : 1
Frameworks disconnected: 0
Frameworks inactive: 0
Resource Management
S3016
127
