Provisioning Config for Component Types

Below document outlines default and override-able params that can be used in the provisioning config for various component types.

Exposed configs for component types

Component type with special parameter are mentioned separately.

Config	Data Type	Mandatory	Schema
deployment_type	string	No	{"enum": ["container", "aws_ec2","gcp_container"], "type": "string"}
params.instance_pool	array	No

Sample provisioning config will look like

[
    {
        "component_name": "my-aerospike",
        "deployment_type": "container"
    }
]

or

[
    {
        "component_name": "my-aerospike",
        "deployment_type": "aws_ec2",
        "params": {
            "instance_pool": ["c5.xlarge"]
        }
    }
]

application

Exposed configs

Config	Data Type	Mandatory	Schema
deployment_type	string	No	{"enum": ["container", "aws_ec2","gcp_container"], "type": "string"}
params.lb_type	string	No	{"enum": ["elb", "alb"], "type": "string"}
params.instance_pool	array	No

Sample provisioning config will look like

[
    {
        "component_name": "my-application",
        "deployment_type": "container"
    }
]

or

[
    {
        "component_name": "my-application",
        "deployment_type": "aws_ec2",
        "params": {
            "instance_pool": ["c5.xlarge"],
            "lb_type": "elb"
        }
    }
]

aerospike

Note - Only supports Aerospike version 6.3.0.7

Exposed configs

Config	Data Type	Mandatory	Schema
deployment_type	string	No	{"enum": ["optimus_aws_ec2"], "type": "string"}
params.aerospike.num_instances	integer	No
params.kafka_outbound.num_instances	integer	No
params.namespace_memory	string	No
params.file	string	No
params.filesize	string	No
params.write_block_size	string	No
params.data_in_memory	string	No
params.remove_ebs_vol	string	No	{ "enum": [ "yes", "no" ], "type": "string" }
params.replication_factor	integer	No	{"type":"integer","minimum":1,"exclusiveMinimum":false}

Spark

Exposed configs

Config	Data Type	Mandatory	Schema
deployment_type	string	No	{ "enum": [ "aws_ec2", "aws_ec2_ha" ], "type": "string" }
params.worker_cleanup_ttl	string	No
params.s3_cron	string	No
params.max_size	string	No
params.on_demand_percentage	string	No
params.spot_allocation_strategy	string	No
params.ebs_root_volume_size	string	No
params.disable_api_termination	string	No
params.num_on_demand_instances	string	No
params.master_instance_pool	array	No	{ "type": "array", "items": { "type": "string" }, "minItems": 1, "maxItems": 1, "uniqueItems": true }
params.master_num_instances	string	No
params.worker_instance_pool	array	No	{ "type": "array", "items": { "type": "string" }, "minItems": 1, "maxItems": 1, "uniqueItems": true }
params.worker_num_instances	string	No
params.history_num_instances	string	No
params.history_instance_pool	string	No	{ "type": "array", "items": { "type": "string" }, "minItems": 1, "maxItems": 1, "uniqueItems": true }
params.spark_ha_route	string	No

To deploy Spark HA, providedeployment_type as aws_ec2_hain the provisioning file.

• you can access the Host using url (spark-$COMPONENT_NAME-multimaster-alb$TEAM_SUFFIX.dream11$VPC_SUFFIX.local). you can also override the Host url using the param spark_ha_route.

• to access the spark UI, use the route from spark exposed configs.

neptune

Exposed configs

Config	Data Type	Mandatory	Schema
deployment_type	string	No	{"enum": ["container", "aws_neptune","gcp_container"], "type": "string"}
params.cluster_parameters	array	No	{"type": "array", "items": {"type": "object", "required": ["name", "value"], "properties": {"name": {"type": "string"}, "value": {}, "apply_immediately": {"enum": ["pending-reboot", "immediate"], "type": "string"}}}}
params.instance_parameters	array	No	{"type": "array", "items": {"type": "object", "required": ["name", "value"], "properties": {"name": {"type": "string"}, "value": {}, "apply_immediately": {"enum": ["pending-reboot", "immediate"], "type": "string"}}}}

postgresql

Exposed configs

Config	Data Type	Mandatory	Schema
deployment_type	string	No	{"enum": ["container", "aws_rds_aurora","gcp_container"], "type": "string"}
params.cluster_parameter_group_name	string	No
params.parameter_group_name	string	No

serverless

Exposed configs

Config	Data Type	Mandatory	Schema
deployment_type	string	No	{"enum": ["container", "aws_serverless"], "type": "string"}
params.extra_envs	array	No	{"type": "array", "items": {"minLength": 1}, "uniqueItems": true}

stepfunction

Exposed configs

Config	Data Type	Mandatory	Schema
deployment_type	string	No	{"enum": ["container", "aws_stepfunction"], "type": "string"}
params.extra_envs	array	No	{"type": "array", "items": {"minLength": 1}, "uniqueItems": true}

Flink

Config	Description	Data Type	Mandatory	Schema
deployment_type	Platform on which the component should be deployed	String	No
params.flink.jobmanager.instance_type	Type of the EC2 instance for jobmanager nodes	String	No
params.flink.taskmanager.instance_type	Type of the EC2 instance for taskmanager nodes	String	No