Apache Apex Command Line Interface
Apex CLI, the Apache Apex command line interface, can be used to launch, monitor, and manage Apache Apex applications. It provides a developer friendly way of interacting with Apache Apex platform. Another advantage of Apex CLI is to provide scope, by connecting and executing commands in a context of specific application. Apex CLI enables easy integration with existing enterprise toolset for automated application monitoring and management. Currently the following high level tasks are supported.
- Launch or kill applications
- View system metrics including load, throughput, latency, etc.
- Start or stop tuple recording
- Read operator, stream, port properties and attributes
- Write to operator properties
- Dynamically change the application logical plan
- Create custom macros
Apex CLI Commands
Apex CLI can be launched by running following command
apex
Help on all commands is available via “help” command in the CLI
Global Commands
GLOBAL COMMANDS EXCEPT WHEN CHANGING LOGICAL PLAN:
alias alias-name command
Create a command alias
begin-macro name
Begin Macro Definition ($1...$9 to access parameters and type 'end' to end the definition)
connect app-id
Connect to an app
dump-properties-file out-file jar-file class-name
Dump the properties file of an app class
echo [arg ...]
Echo the arguments
exit
Exit the CLI
get-app-info app-id
Get the information of an app
get-app-package-info app-package-file
Get info on the app package file
get-app-package-operator-properties app-package-file operator-class
Get operator properties within the given app package
get-app-package-operators [options] app-package-file [search-term]
Get operators within the given app package
Options:
-parent Specify the parent class for the operators
get-config-parameter [parameter-name]
Get the configuration parameter
get-jar-operator-classes [options] jar-files-comma-separated [search-term]
List operators in a jar list
Options:
-parent Specify the parent class for the operators
get-jar-operator-properties jar-files-comma-separated operator-class-name
List properties in specified operator
help [command]
Show help
kill-app app-id/app-name [app-id/app-name ...]
Kill an app
launch [options] jar-file/json-file/properties-file/app-package-file [matching-app-name]
Launch an app
Options:
-apconf <app package configuration file> Specify an application
configuration file
within the app
package if launching
an app package.
-archives <comma separated list of archives> Specify comma
separated archives
to be unarchived on
the compute machines.
-conf <configuration file> Specify an
application
configuration file.
-D <property=value> Use value for given
property.
-exactMatch Only consider
applications with
exact app name
-files <comma separated list of files> Specify comma
separated files to
be copied on the
compute machines.
-ignorepom Do not run maven to
find the dependency
-libjars <comma separated list of libjars> Specify comma
separated jar files
or other resource
files to include in
the classpath.
-local Run application in
local mode.
-originalAppId <application id> Specify original
application
identifier for restart.
-queue <queue name> Specify the queue to
launch the application
list-application-attributes
Lists the application attributes
list-apps [pattern]
List applications
list-operator-attributes
Lists the operator attributes
list-port-attributes
Lists the port attributes
set-pager on/off
Set the pager program for output
show-logical-plan [options] jar-file/app-package-file [class-name]
List apps in a jar or show logical plan of an app class
Options:
-exactMatch Only consider exact match
for app name
-ignorepom Do not run maven to find
the dependency
-libjars <comma separated list of jars> Specify comma separated
jar/resource files to
include in the classpath.
shutdown-app app-id [app-id ...]
Shutdown an app
source file
Execute the commands in a file
Commands after connecting to an application
COMMANDS WHEN CONNECTED TO AN APP (via connect <appid>) EXCEPT WHEN CHANGING LOGICAL PLAN:
begin-logical-plan-change
Begin Logical Plan Change
dump-properties-file out-file [jar-file] [class-name]
Dump the properties file of an app class
get-app-attributes [attribute-name]
Get attributes of the connected app
get-app-info [app-id]
Get the information of an app
get-operator-attributes operator-name [attribute-name]
Get attributes of an operator
get-operator-properties operator-name [property-name]
Get properties of a logical operator
get-physical-operator-properties [options] operator-id
Get properties of a physical operator
Options:
-propertyName <property name> The name of the property whose
value needs to be retrieved
-waitTime <wait time> How long to wait to get the result
get-port-attributes operator-name port-name [attribute-name]
Get attributes of a port
get-recording-info [operator-id] [start-time]
Get tuple recording info
kill-app [<app-id/app-name> ...]
Kill an app
kill-container container-id [container-id ...]
Kill a container
list-containers
List containers
list-operators [pattern]
List operators
set-operator-property operator-name property-name property-value
Set a property of an operator
set-physical-operator-property operator-id property-name property-value
Set a property of an operator
show-logical-plan [options] [jar-file/app-package-file] [class-name]
Show logical plan of an app class
Options:
-exactMatch Only consider exact match
for app name
-ignorepom Do not run maven to find
the dependency
-libjars <comma separated list of jars> Specify comma separated
jar/resource files to
include in the classpath.
show-physical-plan
Show physical plan
shutdown-app [app-id ...]
Shutdown an app
start-recording operator-id [port-name] [num-windows]
Start recording
stop-recording operator-id [port-name]
Stop recording
wait timeout
Wait for completion of current application
Commands when changing the logical plan
COMMANDS WHEN CHANGING LOGICAL PLAN (via begin-logical-plan-change):
abort
Abort the plan change
add-stream-sink stream-name to-operator-name to-port-name
Add a sink to an existing stream
create-operator operator-name class-name
Create an operator
create-stream stream-name from-operator-name from-port-name to-operator-name to-port-name
Create a stream
help [command]
Show help
remove-operator operator-name
Remove an operator
remove-stream stream-name
Remove a stream
set-operator-attribute operator-name attr-name attr-value
Set an attribute of an operator
set-operator-property operator-name property-name property-value
Set a property of an operator
set-port-attribute operator-name port-name attr-name attr-value
Set an attribute of a port
set-stream-attribute stream-name attr-name attr-value
Set an attribute of a stream
show-queue
Show the queue of the plan change
submit
Submit the plan change
Examples
An example of defining a custom macro. The macro updates a running application by inserting a new operator. It takes three parameters and executes a logical plan changes.
apex> begin-macro add-console-output
macro> begin-logical-plan-change
macro> create-operator $1 com.datatorrent.lib.io.ConsoleOutputOperator
macro> create-stream stream_$1 $2 $3 $1 in
macro> submit
Then execute the add-console-output macro like this
apex> add-console-output xyz opername portname
This macro then expands to run the following command
begin-logical-plan-change
create-operator xyz com.datatorrent.lib.io.ConsoleOutputOperator
create-stream stream_xyz opername portname xyz in
submit
Note: To perform runtime logical plan changes, like ability to add new operators, they must be part of the jar files that were deployed at application launch time.
kill-app terminates operators in undefined order, causing some operators to not process
data emitted by input operators. Such application can be restarted from last checkpointed
state using launch -originalAppId command. As application is relaunched from checkpointed
state there will be no data loss.
shutdown-app This command is to terminate the application while making sure that all data
emitted by input operators are processed throughout the DAG. Application terminated with
shutdown-app command can not be restarted, as application is considered to be completed
successfully.