jujubigdata.handlers

jujubigdata.handlers.HDFS
jujubigdata.handlers.HadoopBase
jujubigdata.handlers.YARN

class jujubigdata.handlers.HDFS(hadoop_base)

Bases: object

configure_client()

configure_datanode(host=None, port=None)

configure_hdfs_base(host, port)

configure_namenode(secondary_host=None, secondary_port=None)

configure_secondarynamenode(host=None, port=None)

Configure the Secondary Namenode when the apache-hadoop-hdfs-secondary charm is deployed and related to apache-hadoop-hdfs-master.

The only purpose of the secondary namenode is to perform periodic checkpoints. The secondary name-node periodically downloads current namenode image and edits log files, joins them into new image and uploads the new image back to the (primary and the only) namenode.

create_hdfs_dirs()

format_namenode()

register_slaves(slaves=None)

start_datanode()

start_namenode()

start_secondarynamenode()

stop_datanode()

stop_namenode()

stop_secondarynamenode()

class jujubigdata.handlers.HadoopBase(dist_config)

Bases: object

configure_hadoop()

configure_hosts_file()

Add the unit’s private-address to /etc/hosts to ensure that Java can resolve the hostname of the server to its real IP address. We derive our hostname from the unit_id, replacing / with -.

install(force=False)

install_base_packages()

install_hadoop()

install_java()

Run the java-installer resource to install Java and determine the JAVA_HOME and Java version.

The java-installer must be idempotent and its only output (on stdout) should be two lines: the JAVA_HOME path, and the Java version, respectively.

If there is an error installing Java, the installer should exit with a non-zero exit code.

is_installed()

register_slaves(slaves)

Add slaves to a hdfs or yarn master, determined by the relation name.

Parameters:relation (str) – ‘datanode’ for registering HDFS slaves; ‘nodemanager’ for registering YARN slaves.

run(user, command, *args, **kwargs)

Run a Hadoop command as the hdfs user.

Parameters:

• command (str) – Command to run, prefixed with bin/ or sbin/
• args (list) – Additional args to pass to the command

setup_hadoop_config()

spec()

Generate the full spec for keeping charms in sync.

NB: This has to be a callback instead of a plain property because it is passed to the relations during construction of the Manager but needs to properly reflect the Java version in the same hook invocation that installs Java.

class jujubigdata.handlers.YARN(hadoop_base)

Bases: object

configure_client(host=None, port=None, history_http=None, history_ipc=None)

configure_jobhistory()

configure_nodemanager(host=None, port=None, history_http=None, history_ipc=None)

configure_resourcemanager()

configure_yarn_base(host, port, history_http, history_ipc)

install_demo()

register_slaves(slaves=None)

start_jobhistory()

start_nodemanager()

start_resourcemanager()

stop_jobhistory()

stop_nodemanager()

stop_resourcemanager()