Component Type reference

Exposed configs in service definition fileThis document outlines usage of various component types and extensible configs.

Odin provides describe action to help understand exact definitions of various entities.

odin describe component-type

Let's explore describe action on component-type

odin describe component-type -h

Usage:  odin list component-type [Options]

Options:
  --name     name of component type (required)   
  --version  of component type (deafult latest) 

To get list of all possible component type use the below command

odin list component-type

Listing all component types
  COMPONENT NAME   | LATEST VERSION | DEPLOYEMNT TYPES               | TOTAL VERSIONS AVAILABLE
-------------------+----------------+--------------------------------+---------------------------
  aerospike        | 5.6.0.15-1.0.0 | aws_ec2, container             | 4
  application      | 1.0.0          | aws_ec2, container             | 1
  cassandra        | 3.11.12-1.0.0  | aws_ec2, container             | 2
  cloudfront       | 1.0.0          | aws_cloudfront                 | 1
  elasticsearch    | 7.5.2-1.0.0    | aws_ec2, container             | 1
  flink            | 1.15.0-1.0.0   | aws_ec2, container             | 2
  kafka            | 2.6.0-1.0.0    | aws_ec2, container             | 1
  kafka-connect    | 5.4.9-1.0.0    | aws_ec2, container             | 2
  kafka-connect-ui | 0.9.7-1.0.0    | aws_ec2, container             | 1
  kafka-zookeeper  | 2.8.1-1.0.0    | aws_ec2, container             | 2
  ksql             | 5.5.12-1.0.0   | aws_ec2, container             | 2
  neptune          | 1.1.1.0-1.0.0  | container, aws_neptune         | 1
  postgresql       | 11.15-1.0.0    | container, aws_rds_aurora      | 1
  rabbitmq         | 3.8.2-1.0.0    | aws_ec2, container             | 1
  rds              | 8.0.23-1.0.0   | container, aws_rds_aurora      | 3
  redis            | 6.2.1-1.0.0    | container, aws_elasticache     | 5
  s3               | 1.0.0          | aws_s3                         | 1
  schema-registry  | 5.2.1-1.0.0    | aws_ec2, container             | 1
  scylladb         | 4.5-1.0.0      | aws_ec2, container             | 1
  serverless       | 1.0.0          | aws_serverless                 | 1
  spark            | 3.1.2-1.0.0    | aws_ec2, container, aws_ec2_ha | 4
  stepfunction     | 1.0.0          | aws_stepfunction               | 1
  voltdb           | 11.4.3-1.0.0   | aws_ec2, container             | 3
  zookeeper        | 3.4.14-1.0.0   | aws_ec2, container             | 1

Command to describe component types
odin describe component-type --name <componentTypeName> --version <componentTypeVersion>

Now to understand config of a specific component-type, use the below command

odin describe component-type --name redis

Output of the above command will be something like this

{ ...definition_json }

list of exposed configs

 CONFIG                    | MANDATORY | DATA TYPE  
----------------------------+-----------+------------

The above output has two component - the exact definition_json for the given component-type used by odin to provision the component and the list of exposed configs that can be used in service definition

Let's deep dive into each component-type

redis

odin describe component-type --name redis

Exposed configs in Service definition

Config	Description	Mandatory	Data Type	Schema
name	Component name	Yes	string
type	Component type	Yes	string
version	Component version	Yes	string
depends_on	List of components which should be provisioned before redis is provisioned	No	array	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}
config.route.private	DNS where redis will be discoverable	No	string	{"type": "string", "pattern": "[a-zA-Z._-]+\${TEAM_SUFFIX}\.dream11\${VPC_SUFFIX}\.local"}

Exposed config in Provisioning file

Config	Description	Mandatory	Data Type	Schema
deployment_type	Where the component should be deployed. Allowed values: container, aws_elasticache	No	string
params.instance_type	Instance type for AWS ElastiCache cluster in which redis will be deployed	No	string
params.node_replicas	Number of read replicas to create per node group for the AWS ElastiCache cluster	No	integer
params.parameter_group_name	Name of the parameter group to associate with the AWS ElastiCache cluster	No	string
params.shards	Number of node groups (shards) to create for the AWS ElastiCache cluster	No	string

Example service definition and provisioning file

// Service definition
{
  "name": "redis-service",
  "version": "1.0.0",
  "team": "devx",
  "components": [
    {
      "type": "redis",
      "name": "demo-redis",
      "version": "6.2.1-1.0.0",
      "config": {
        "route": {
          "private": "demo-redis-${ENV_NAME}.${PRIVATE_HOSTED_ZONE}"
        }
      }
    }
  ]
}

// Provisioning file
[
    {
        "component_name": "demo-redis",
        "deployment_type": "aws_elasticache",
        "params": {
           "instance_type": "cache.t2.medium",
           "node_replicas": 3,
           "parameter_group_name": "some_parameter_group_name",
           "shards": 3
        }
    }
]

application

odin describe component-type --name application

Exposed configs in Service definition

Config	Description	Data Type	Mandatory	Default Value	Schema
name	component name	string	Yes
type	component type	string	Yes
version	component version	string	Yes
config.artifact_name	artifact name of the application	string	Yes		{'type': 'string', 'message': {'pattern': 'artifact name must contain only lower case alphabet, numbers or -'}, 'pattern': '^[a-z][a-z0-9-]*$'}
config.artifact_version	artifact version of the application	string	Yes
config.build_type	base image type on which the application runs	string	Yes
config.build_version	base image version on which the application runs	string	Yes
config.discovery	at what level the application should be discoverable	string	No		{'enum': ['private', 'public', 'both', 'none'], 'type': 'string'}
config.extra_envs	environment variables required while starting the application	object	No
config.healthcheck_protocol	protocal to be used for healthcheck API	string	No	HTTP	{'enum': ['HTTP', 'TCP', 'GRPC'], 'type': 'string'}
config.liveness_endpoint	api endpoint to check application is running	string	No	healthcheck
config.loadbalancer-type	type of load balancer	string	No	alb
config.port	port at which the application is running	integer	No	8080
config.readiness_endpoint	api endpoint for healthcheck	string	No	healthcheck
config.route.private	private DNS endpoint of the application	string	No
config.route.public	public DNS endpoint of the application	string	No
config.start_script_path	path of script to start the application	string	No	.odin/start.sh
config.config_store_directory	directory name in config store to fetch configs from	string	No
depends_on	List of components on which application depends on	array	No		{'type': 'array', 'items': {'type': 'string'}, 'minItems': 1, 'uniqueItems': True}

Exposed config in Provisioning file

Config	Description	Data Type	Mandatory	Default Value	Schema
deployment_type	platform on which the component should be deployed	string	No	container	{'enum': ['container', 'aws_ec2'], 'type': 'string'}
env_variables	environment variables required while starting the application	object	No
params.add_legacy_tags	flag to add legacy tags to the resources	boolean	No	true
params.auto_routing	automatic route traffic from old stack to new stack	boolean	No	true
params.autoscale_cpu_percentage_threshold	cpu percentage which triggers autoscaling	integer	No	50
params.canary	route traffic from old stack to new stack using canary technique	boolean	No	false
params.capacity_type	determines how the instance is launched i.e SPOT or ON_DEMAND	string	No	SPOT	{'enum': ['ON_DEMAND', 'SPOT'], 'type': 'string'}
params.instance_pool	instance type pool for application autoscaling	array	No		{'type': 'array', 'items': {'type': 'string'}, 'message': {'minItems': 'instance_pool must have atleast 1 element'}, 'minItems': 1, 'uniqueItems': True}
params.lb_type	type of load balancer	string	No	alb	{'enum': ['elb', 'alb', 'nlb'], 'type': 'string'}
params.healthcheck_grace_period	Grace period of healthcheck in seconds	integer	No	120
params.extra_ports_config	List of extra ports config	array	No		{ "type": "array", "items": { "type": "object", "required": [ "port" ], "properties": { "port": { "type": "integer" }, "healthcheck_protocol": { "type": "string" }, "readiness_endpoint": { "type": "string" } } } }
params.lcu.external_lb	lcu count for external load balancer	integer	No	1
params.lcu.internal_lb	lcu count for internal load balancer	integer	No	1
params.logs.enabled	enable application logs on datadog	boolean	No	false
params.logs.path	log path location of application for datadog	string	No	/opt/logs/${ARTIFACT_NAME}.log
params.num_instances	number of instances for application autoscaling	integer	No	1
params.num_lbs	number of shards of application	integer	No	1
params.on_demand_base_capacity	number of instances required to be ON_DEMAND capacity. capacity_type should be ON_DEMAND to use this	integer	No	1
params.passive_downscale	downscale passive stack once the deployment is successfull	boolean	No	true
params.replica_count	number of replicas of application pods	integer	No	1
params.resources.limits.cpu	maximum amount of CPU that a container can use	integer	No	1
params.resources.limits.memory	maximum amount of memory that a container can use	string	No	1Gi
params.imdsv2	Use this if you want to opt for imdsv2 This is a non-mandatory config, and takes two possible inputs, "optional" and "required" Use value "required" if you want o opt for imdsv2 for your application component	String	No	optional

Available Build Types

Build Type	Build Version
Java	8
Java	11
Java	17
Java-redis	8-3.2.12
Java-redis	11-3.2.12
java-node	8-14.15.1
Kong	2.2.0
kong	2.8.1
Node	9.0.0
Node	14.15.1
Node	14.15.1-buster-slim
Node	16.15.1
Node	16.15.1-buster-slim
Node-redis	14.15.1-3.2.12
PCI-Java	8
Python	3.8.12
Scala	2.12.8

Example service definition and provisioning file

// Service definition
"components": [
    {
      "type": "application",
      "name": "web-application",
      "version": "1.0.0",
      "config": {
          "artifact_name": "web-app",
          "artifact_version": "1.0.0-SNAPSHOT",
          "build_type": "node",
          "build_version": "14.15.1"
      }
    }
  ]


// Provisioning file
[
    {
        "component_name": "web-application",
        "deployment_type": "container"
    }
]

Environment variables set by odin

Common across EC2 and Containers:

Variable name	Value
SERVICE_NAME	<ARTIFACT_NAME>
ARTIFACT_NAME	<ARTIFACT_NAME>
ARTIFACT_VERSION	<ARTIFACT_VERSION>
VPC_SUFFIX	<VPC_SUFFIX>
ENV	One of [dev, int, load, uat, prod]
DEPLOYMENT_TYPE	One of [ec2, containers]
DISCOVERY_PROVIDER	One of [r53, consul]
CONSUL_DNS_ENDPOINT	<CONSUL_DNS_ENDPOINT>

Specific to EC2:

Variable name	Value
APP_DIR	/var/www/<ARTIFACT_NAME>
NAMESPACE	Config store branch
START_SCRIPT_PATH	As provided in the config. Defaults to ".odin/start.sh"
VAULT_TOKEN	<VAULT_TOKEN>
CONSUL_TOKEN	<CONSUL_TOKEN>
DD_SERVICE	<ARTIFACT_NAME>
DD_VERSION	<ARTIFACT_VERSION>
DD_LOGS_INJECTION	true
DD_RUNTIME_METRICS_ENABLED	true
DD_PROFILING_ENABLED	true
DD_SERVICE_MAPPING	Comma separated list of <COMPONENT_TYPE>:<SERVICE_NAME>-<COMPONENT_TYPE>
DD_AGENT_HOST	localhost

Specific to Containers:

Variable name	Value
APP_DIR	/app/<ARTIFACT_NAME>
AWS_ACCOUNT_ID	<AWS_ACCOUNT_ID>

Aerospike

For optimus aerospike component, please refer Aerospike.

odin describe component-type --name aerospike

Exposed configs in the service definition

Config	Description	Data Type	Mandatory	Schema
name	Component name	string	Yes
type	Component type	string	Yes
version	Component version	string	Yes
depends_on	List of components on which aerospike depends on	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}
config.route.private	Route on which aerospike is discoverable	string	No	{"type": "string", "pattern": "[a-zA-Z._-]+\${TEAM_SUFFIX}\.dream11\${VPC_SUFFIX}\.local"}
config.namespace	Comma separated list of namespaces to create	string	No
config.cluster_type	Type of cluster	string	No	{ "enum": [ "AP", "SC" ], "type": "string" }
config.replication_factor	aerospike replication factor	integer	No
config.kafka_outbound.enabled	Whether kafka outbound is enabled or not.	string	No	Valid values are [true, false]. Default is false
config.kafka_outbound.kafka_host	Host of kafka for the outbound process	string	No
config.kafka_outbound.routing_mode	Specifies how records are routed to the topic	string	No
config.kafka_outbound.topic	Kafka topic to which data is to be written	string	No
config.kafka_outbound.type	Decides whether kafka_outbound is run locally or on remote nodes.	string	No	Valid values are [local, remote]
config.kafka_outbound.xdr_dns	IP address of the remote datacenter	string	No
config.kafka_outbound.namespace_configuration.format_mode	Specifies the serialization format of the data output	string	No
config.kafka_outbound.namespace_configuration.name	Namespace for which this configuration is applicable	string	Conditional	Mandatory if block config.kafka_outbound.namespace_configuration is defined.
config.kafka_outbound.namespace_configuration.routing_mode	Overrides how records are routed to the topic for the namespace	string	No

Exposed configs in the provisioning file

Config	Description	Data type	Mandatory	Schema
deployment_type	Where the component should be deployed.	String	No	Allowed values: container/aws_ec2
params.aerospike.num_instances	Number of instances in the aerospike cluster	Integer	No
params.data_in_memory	Whether to keep a copy of data in memory.	String	No	Valid values [true, false]
params.file	Path of aerospike file	String	No
params.filesize	Size of aerospike file	String	No
params.instance_pool	List of instance-types from which aerospike nodes will be created, takes precendence over instance_type	Array	No
params.kafka_outbound.num_instances	Number of instances of the kafka-outbound process	Integer	No
params.namespace_memory	Comma separated list of in-memory namespaces	String	No
params.remove_ebs_vol	Set to 'yes' if EBS volume should be detached from the EC2 instance.	String	No	Valid values: [yes, no]
params.write_block_size	Size of each I/O block that is written to the disk	String	No

Artifact

odin describe component-type --name artifact

Exposed configs in the service definition

Config	Description	Data Type	Mandatory	Schema
name	Component name	string	Yes
type	Component type	string	Yes
version	Component version	string	Yes
depends_on	List of components on which artifact depends on	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}
config.artifact_name	Artifact name	string	Yes
config.artifact_version	Artifact version	string	Yes

Exposed configs in the provisioning config

Config	Description	Data Type	Mandatory	Schema
deployment_type	Where the component should be deployed.	String	No	Allowed values: aws

Example service definition and provisioning file

// Service definition
"components": [
    {
      "type": "artifact",
      "name": "demo-artifact",
      "version": "1.0.0",
      "config": {
        "artifact_name": "demo-spark-jar",
        "artifact_version": "1.0.0"
      }
    }
  ]


// Provisioning file
[
  {
    "component_name": "demo-artifact",
    "deployment_type": "aws"
  }
]

FAQ

• Make sure the path {BASE_REPOSITORY}/{ARTIFACT_NAME}/{ARTIFACT_VERSION}/ has an artifact in jfrog. BASE_REPOSITORY is d11-repo/com/dream11 for dream11.

  • Example path: https://dreamsports.jfrog.io/artifactory/d11-repo/com/dream11/demo-spark-jar/1.0.0/

• The component makes the artifact discoverable with the below URL

  https://artifactory.dream11.com/{ENV_NAME}/{SERVICE_NAME}/{COMPONENT_NAME}.{FILE_TYPE}

  • Example: Let’s say your environment name is dev-001, the service name is artifact-test, the component name is demo-artifact, and the artifact is artifact-demo.jar i.e. the FILE_TYPE is .jar, then the URL is

    https://artifactory.dream11.com/dev-001/artifact-test/demo-artifact.jar

• The discovery route is not configurable and you cannot override it.

• A spark-submit command using the above URL should look like this:

curl -X POST \
  --header "Content-Type: application/json" \
  --data '{
    "action": "CreateSubmissionRequest",
    "appArgs": ["arg1", "arg2", "arg3"],
    "appResource": "https://artifactory.dream11.com/dev-001/artifact-test/demo-artifact.jar",
    "clientSparkVersion": "3.1.2",
    "mainClass": "com.odin.Analytics",
    "sparkProperties": {
      "spark.jars": "https://artifactory.dream11.com/dev-001/artifact-test/demo-artifact.jar",
      "spark.driver.supervise": "false",
      "spark.app.name": "OdinAnalytics",
      "spark.eventLog.enabled": "true",
      "spark.submit.deployMode": "cluster",
      "spark.master": "spark://<your-spark-master-url>:7077"
    }
  }' \
  http://<your-spark-master-url>:6066/v1/submissions/create

For operations on the artifact component, please refer to this

Cassandra

odin describe component-type --name cassandra

Exposed configs in the service definition

Config	Description	Data Type	Mandatory	Schema
name	Component name	string	Yes
type	Component type	string	Yes
version	Component version	string	Yes
depends_on	List of components on which cassandra depends on	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}
config.ec2-instance-type	Type of the EC2 instance for cassandra nodes	string	No
config.route.private	Route on which cassandra is discoverable	string	No	{"type": "string", "pattern": "[a-zA-Z._-]+\${TEAM_SUFFIX}\.dream11\${VPC_SUFFIX}\.local"}

Exposed configs in the provisioning config

Config	Description	Data Type	Mandatory	Schema
deployment_type	Where the component should be deployed.	String	No	Allowed values: container/aws_ec2
params.instance_pool	List of instance-types from which aerospike nodes will be created, takes precendence over instance_type	Array	No
params.num_instances	Number of instances in the cassandra cluster	Integer	No

cloudfront

odin describe component-type --name cloudfront

Exposed configs

Config	Description	Data Type	Mandatory	Schema	Description/Valid values
name	Component name	string	Yes
type	Component type	string	Yes
version	Component verion	string	Yes
depends_on	List of components which should be provisioned before cloudfront is provisioned	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}
config.default_cache_behavior	A list of default cache behaviors for the CloudFront distribution. The default cache behavior specifies how CloudFront handles requests that don't match any of the other cache behaviors in the distribution	array	Yes	{"type": "object", "items": {"required": ["target_origin_id"], "properties": {"cached_methods": {"type": "array"}, "allowed_methods": {"type": "array"}, "target_origin_id": {"type": "string"}}}, "uniqueItems": true}
config.default_cache_behavior.target_origin_id	The ID of the origin to which requests are forwarded for the cache behavior	string	Yes		Id of one of the origins specified under the origins block
config.origins	A list of origins for the CloudFront distribution. An origin is the location from which CloudFront retrieves the files to distribute	array	Yes	{ "type": "array", "items": { "required": [ "domain_name", "origin_id" ], "properties": { "params": { "type": "object" }, "origin_id": { "type": "string" }, "domain_name": { "type": "string" }, "origin_path": { "type": "string" }, "connection_attempts": { "type": "integer", "maximum": 3, "minimum": 1 }, "connection_timeout": { "type": "integer", "maximum": 10, "minimum": 1 }, "custom_origin_config": { "type": "object", "required": [ "http_port", "https_port", "origin_protocol_policy" ], "properties": { "http_port": { "type": "string" }, "https_port": { "type": "string" }, "origin_read_timeout": { "type": "integer" }, "origin_protocol_policy": { "type": "string" }, "origin_keepalive_timeout": { "type": "integer" } } } } }, "uniqueItems": true }
config.aliases	A list of domain names and subdomain names that you want CloudFront to use to respond to requests for your content	array	No		Alternate domain names
config.default_cache_behavior.allowed_methods	HTTP methods that CloudFront processes and forwards to your custom origin	array	No		DELETE, GET, HEAD, OPTIONS, PATCH, POST, PUT
config.default_cache_behavior.cached_methods	HTTP methods for which CloudFront caches responses by default	array	No		DELETE, GET, HEAD, OPTIONS, PATCH, POST, PUT
config.default_cache_behavior.compress	Whether CloudFront automatically compresses content for web requests that include Accept-Encoding: gzip in the request header	boolean	No
config.default_cache_behavior.default_ttl	The default time in seconds that objects stay in CloudFront caches before CloudFront forwards another request to your custom origin to determine whether the object has been updated	integer	No
config.default_cache_behavior.field_level_encryption_id	The ID of the field-level encryption configuration that you want CloudFront to use for encrypting specific fields of data for a cache behavior	integer	No
default_cache_behavior.forwardedvalues.cookies.forward	Specifies whether you want CloudFront to forward cookies to the origin that is associated with this cache behavior	string	No		all, none, whitelist
config.default_cache_behavior.forwarded_values.cookies.whitelisted_names	A list of cookie names that you want CloudFront to forward to your origin	array	No
config.default_cache_behavior.forwarded_values.headers	A list of HTTP headers that you want CloudFront to forward to your origin	array	No
config.default_cache_behavior.forwarded_values.query_string	Indicates whether you want CloudFront to forward query strings to the origin that is associated with this cache behavior	boolean	No
config.default_cache_behavior.max_ttl	The maximum time in seconds that objects stay in CloudFront caches before CloudFront forwards another request to your custom origin to determine whether the object has been updated	integer	No
config.default_cache_behavior.min_ttl	The minimum time in seconds that objects stay in CloudFront caches before CloudFront forwards another request to your custom origin to determine whether the object has been updated	integer	No
config.default_cache_behavior.smooth_streaming	Indicates whether CloudFront supports smooth streaming for the cache behavior or not	boolean	No
config.default_cache_behavior.viewer_protocol_policy	The protocol that viewers can use to access the files in the origin specified by TargetOriginId when a request matches the path pattern in the cache behavior	string	No		redirect-to-https, allow-all, https-only
config.ordered_cache_behaviours	A list of ordered cache behaviors for the CloudFront distribution. Ordered cache behaviors are used to override the default cache behavior and specify different caching behaviors for specific URL patterns	array	No	{"type":"array","items":{"required":["cached_methods","allowed_methods","target_origin_id","viewer_protocol_policy"],"properties":{"max_ttl":{"type":"integer"},"min_ttl":{"type":"integer"},"compress":{"type":"boolean"},"default_ttl":{"type":"integer"},"cached_methods":{"type":"array"},"allowed_methods":{"type":"array"},"forwarded_values":{"type":"object","required":["cookies","query_string"],"properties":{"cookies":{"type":"object","required":["forward"],"properties":{"forward":{"type":"string"},"whitelisted_names":{"type":"array"}}},"headers":{"type":"array"},"query_string":{"type":"boolean"}}},"target_origin_id":{"type":"string"},"viewer_protocol_policy":{"type":"string"}}},"uniqueItems":true}
config.price_class	Specifies the price class for the CloudFront distribution	string	No
config.restrictions.geo_restriction.restriction_type	The method that you want to use to restrict distribution of your content by country	string	No
config.retain_on_delete	Whether to retain the CloudFront distribution when the resource is deleted	boolean	No
config.viewer_certificate.acm_certificate_arn	The Amazon Resource Name (ARN) of the AWS Certificate Manager SSL certificate that you want CloudFront to use for this distribution	string	No
config.viewer_certificate.cloudfront_default_certificate	Indicates whether to use the default CloudFront certificate or not. If set to true, the default certificate will be used, else, the certificate with the ARN specified in config.viewer_certificate.acm_certificate_arn field will be used	boolean	No
config.viewer_certificate.minimum_protocol_version	The minimum version of the SSL/TLS protocol that you want CloudFront to use for HTTPS connections	string	No
default_cache_behavior.viewer_certificate.ssl_support_method	Specifies how CloudFront serves HTTPS requests	string	No		vip, sni-only
config.web_acl_id	The ID of the AWS WAF web access control list (ACL) associated with the distribution	string	No
config.default_cache_behavior.cache_policy_id	Unique identifier of the cache policy that is attached to the cache behavior. forwarded_values field should not be specified when this field is used.	string	No
config.default_cache_behavior.origin_request_policy_id	Unique identifier of the origin request policy that is attached to the behavior. forwarded_values field should not be specified when this field is used.	string	No

// Service definition
{
  "name": "cloudfront-service",
  "version": "1.0.0",
  "team": "devx",
  "components": [
    {
      "type": "cloudfront",
      "name": "demo-cloudfront",
      "version": "1.0.0",
      "config": {
        "default_cache_behavior": {
          "allowed_methods": ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"],
          "cached_methods": ["GET", "HEAD"],
          "default_ttl": 0,
          "forwarded_values": {
            "cookies": {
              "forward": "none"
            },
            "query_string": false
          },
          "max_ttl": 0,
          "min_ttl": 0,
          "target_origin_id": "******************************",
          "viewer_protocol_policy": "redirect-to-https"
        },
        "viewer_certificate": {
          "cloudfront_default_certificate": false,
          "ssl_support_method": "sni-only",
          "acm_certificate_arn": "**************",
          "minimum_protocol_version": "TLSv1.1_2016"
        },
        "ordered_cache_behaviors": [
          {
            "cached_methods": ["GET", "HEAD"],
            "allowed_methods": ["GET", "HEAD", "OPTIONS"],
            "target_origin_id": "*************************",
            "viewer_protocol_policy": "allow-all",
            "path_pattern": "/monogql*",
            "forwarded_values": {
              "query_string": true,
              "headers": ["Origin", "Referer", "User-Agent", "Accept","Accept-Encoding", "Accept-Language", "Authorization", "CloudFront-Viewer-Country", "x-csrf"],
              "cookies": {
                "forward": "all"
              }
            },
            "field_level_encryption_id": "",
            "smooth_streaming": false,
            "min_ttl": 0,
            "max_ttl": 31536000,
            "default_ttl": 86400,
            "compress": false
          }
        ],
        "origins": [
          {
            "domain_name": "**********************************",
            "origin_id": "*******************************",
            "custom_origin_config": {
              "http_port": "80",
              "https_port": "443",
              "origin_protocol_policy": "http-only",
              "origin_keepalive_timeout": 30,
              "origin_read_timeout": 5,
              "origin_ssl_protocols": ["TLSv1.2", "TLSv1.1", "TLSv1"]
            }
          }
        ]
      }
    }
  ]
}

elasticsearch

odin describe component-type --name elasticsearch

Exposed configs in service definition

Config	Description	Data Type	Mandatory	Schema
config.cluster_name	elasticsearch cluster name	string	No
depends_on	Components that should be deployed before this component.	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}
name	component name	string	Yes
type	component type	string	Yes
version	component version	string	Yes

Exposed configs in provisioning file

Config	Description	Data Type	Mandatory	Schema
params.elastic_ec2_auto_disc_min_master	Auto discovery minimum master count	integer	No
params.on_demand_percentage_above_base_capacity	Desired percent of on-demand instances over the number of base on-demand instances	integer	No
params.hotnode_ebs_root_volume_size	Desired root volume size of the hotnode ebs	integer	No
params.master_ebs_root_volume_size	Desired root volume size of the master node	integer	No
params.mlnode_ebs_root_volume_size	Desired root volume size of the ml node	integer	No
params.warmnode_ebs_root_volume_size	Desired root volume size of the warm node	integer	No
params.hotnode_instance_type	Desired instance type of the hotnode	string	No
params.master_instance_type	Desired instance type of the master	string	No
params.mlnode_instance_type	Desired instance type of the ml node	string	No
params.warmnode_instance_type	Desired instance type of the warm node	string	No
params.num_hotnodes	Desired number of hotnodes	integer	No
params.num_masters	Desired number of master nodes	integer	No
params.num_mlnodes	Desired number of ml nodes	integer	No
params.num_warmnodes	Desired number of warm nodes	integer	No
deployment_type	Where the component should be deployed. Allowed values: container/aws_ec2	string	No	{ "enum": [ "container", "aws_ec2" ], "type": "string" }

// Service definition
"components": [
    {
      "type": "elasticsearch",
      "name": "demo-elasticsearch",
      "version": "7.5.2-1.0.0",
      "depends_on": [
        "demo-component"  // Given this component is also present in service definition
      ],
      "config": {
        "cluster_name":"demo-cluster-name"
      }
    }
  ]


// Provisioning file
[
    {
        "component_name": "demo-elasticsearch",
        "deployment_type": "aws_ec2",
        "params": {
            "elastic_ec2_auto_disc_min_master" : 3,
            "on_demand_percentage_above_base_capacity":4,
            "hotnode_ebs_root_volume_size":100,
            "master_ebs_root_volume_size":100,
            "mlnode_ebs_root_volume_size":100,
            "warmnode_ebs_root_volume_size":50,
            "hotnode_instance_type":"t3.medium",
            "master_instance_type":"t3.medium",
            "mlnode_instance_type":"t3.medium",
            "warmnode_instance_type":"t3.medium",
            "num_hotnodes":2,
            "num_masters":2,
            "num_mlnodes":2,
            "num_warmnodes":2
        }
    }
]

flink

odin describe component-type flink

Exposed configs

Config	Description	Data Type	Mandatory	Schema
name	component name	String	No
type	component type	String	No
version	component version	String	No
config.custom_config	Custom configuration for Flink	object	No
config.route.private	Route on which Flink is discoverable	String	No
depends_on	List of components on which Flink Component depends on	Array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}
parallelism	Execute multiple tasks concurrently, with each task being processed by one or more parallel instances	Integer	No

Note: To add additional configuration in flink-config.yaml, key: value pairs can be passed through component config using custom_config key. eg.

  {
    "type": "flink",
    "name": "custom-flink",
    "version": "1.15.0-1.0.0",
    "config": {
      "custom_config" : {
        "a" : "b"
      }
    }
  }

The sample keys in above example will appear as:\

jobmanager.rpc.address: custom-flink-jobmanager 
taskmanager.numberOfTaskSlots: 2 
blob.server.port: 6124 
jobmanager.rpc.port: 6123 
taskmanager.rpc.port: 6122 
queryable-state.proxy.ports: 6125 
jobmanager.memory.process.size: 1600m 
taskmanager.memory.process.size: 1728m 
parallelism.default: 2 
a: b

kafka

odin describe component-type --name kafka

Exposed configs in Service definition

Config	Description	Mandatory	Data Type	Schema
name	component name	Yes	string
type	component type	Yes	string
version	component version	Yes	string
depends_on	List of components which should be provisioned before kafka is provisioned	No	array	{ "type": "array", "items": { "type": "string" }, "minItems": 1, "uniqueItems": true }
config.route.private	DNS where kafka will be discoverable	No	string	{"type": "string", "pattern": "[a-zA-Z._-]+\${TEAM_SUFFIX}\.dream11\${VPC_SUFFIX}\.local"}
config.zookeeper_endpoint	Endpoint of the ZooKeeper ensemble that Kafka uses for coordination and metadata management	No	string

Exposed configs in Provisioning file

Config	Description	Mandatory	Data Type	Schema
deployment_type	Where the component should be deployed. Allowed values: container, aws_ec2	Yes	string
params.log_retention_check_interval_ms	Interval in milli seconds at which kafka will check for log segments that are eligible for deletion based on the configured log retention policy	No	integer
params.log_retention_hours	Number of hours that Kafka will retain log segments before deleting them based on the log retention policy	No	integer
params.ebs_iops	Number IOPS that the extra EBS block device for kafka should support	No	integer
params.ebs_throughput	Throughput in megabytes per second (MB/s) that the extra ebs block device for kafka should support	No	integer	{ "type": "integer", "minimum": 125, "maximum": 1000 }
params.ebs_volume_size	Size in gibibytes (GiB) of the extra ebs block device for kafka	No	integer
params.instance_pool	List of instances on which kafka will be provisioned	No	array	{ "type": "array", "items": { "type": "string" }, "maxItems": 1, "minItems": 1}
params.num_instances	Number of instances of kafka	No	integer	{ "type": "integer", "minimum": 3 }
params.replicaCount	Number of kafka pods when deployment type is container	No	integer

Example service definition and provisioning file

// Service defintion
{
  "name": "kafka-service",
  "version": "1.0.0",
  "team": "devx",
  "components": [
    {
      "type": "kafka",
      "name": "demo-kafka",
      "version": "2.6.0-1.0.0",
      "config": {
        "route": {
          "private": "demo-kafka-${ENV_NAME}.${PRIVATE_HOSTED_ZONE}"
        }
      }
    }
  ]
}

// Provisioning file
[
    {
        "component_name": "demo-kafka",
        "deployment_type": "aws_ec2",
        "params": {
          "instance_pool": ["t3.medium"],
          "ebs_extra_iops": 3000,
          "ebs_extra_throughput": 125,
          "ebs_extra_volume_size": 500,
          "num_instances": 3,            
          "log_retention_check_interval_ms": 200000,
          "log_retention_hours": 168
        }
    }
]

kafka-connect

odin describe component-type --name kafka-connect

Exposed configs in service definition

Config	Description	Data Type	Mandatory	Schema
name	Component name	string	Yes
type	Component type	string	Yes
version	Component version	string	Yes
config.bootstrap_server	DNS of kafka broker	string	Yes
config.schema_registry_host	DNS of schema registry	string	Yes
config.route.private	DNS where kafka connect will be discoverable	string	No	{"type": "string", "pattern": "[a-zA-Z._-]+\${TEAM_SUFFIX}\.dream11\${VPC_SUFFIX}\.local"}
depends_on	List of components on which kafka connect depends on	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}

Exposed configs in provisioning config

Config	Description	Data Type	Mandatory	Default Value	Schema
deployment_type	Where the component should be deployed. Allowed values: container/aws_ec2	string	No	container
params.instance_pool	List of instances on which kafka connect will be provisioned	array	No	["t2.medium"]	{ "type": "array", "items": { "type": "string" }, "maxItems": 1, "minItems": 1, "uniqueItems": true }
params.num_instances	Number of instances of kafka connect	integer	No	1
params.replicaCount	Number of replicas of the pod to be created	integer	No	1
params.kafka_connect_topic_replication_factor	Number of copies of data to be made	integer	No	1

Example service definition and provisioning file

// Service definition
"components": [
    {
      "type": "kafka-connect",
      "name": "demo-kafka-connect",
      "version": "5.4.9-1.0.0",
      "depends_on": [
        "demo-kafka", // Assuming a component with this name also exists in service definition
        "demo-schema-registry // Assuming a component with this name also exists in service definition
      ],
      "config": {
        "bootstrap_server": "demo-kafka-${ENV_NAME}.${PRIVATE_HOSTED_ZONE}",
        "schema_registry_host": "demo-sr-${ENV_NAME}.${PRIVATE_HOSTED_ZONE}",
        "route": {
          "private": "demo-kafka-connect-${ENV_NAME}.${PRIVATE_HOSTED_ZONE}"
        }
      }
    }
  ]


// Provisioning file
[
    {
        "component_name": "demo-kafka-connect",
        "deployment_type": "aws_ec2",
        "params": {
            "num_instances" : 3,
            "instance_pool": ["t2.medium"],
            "kafka_connect_topic_replication_factor": 3
        }
    }
]

kafka-connect-ui

odin describe component-type --name kafka-connect-ui

Exposed configs in service definition

Config	Description	Data Type	Mandatory	Schema
config.connect_cluster_url	DNS of the kafka connect cluster	string	Yes
depends_on	List of components on which kafka connect ui depends on	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}
name	Component name	string	Yes
type	Component type	string	Yes
version	Component version	string	Yes

Exposed configs in provisioning config

Config	Description	Data Type	Mandatory	Schema
deployment_type	Where the component should be deployed. Allowed values: container/aws_ec2	string	Yes
params.instance_pool	List of instances on which kafka connect ui will be provisioned	array	No	{ "type": "array", "items": { "type": "string" }, "maxItems": 1, "minItems": 1}
params.num_instances	Number of instances of kafka connect ui	integer	No

Example service definition and provisioning file

// Service definition
"components": [
    {
      "type": "kafka-connect-ui",
      "name": "demo-kafka-connect-ui",
      "version": "0.9.7-1.0.0",
      "depends_on": [
        "demo-kafka-connect" // Assuming a component with this name also exists in service definition
      ],
      "config": {
        "connect_cluster_url": "demo-kafka-connect-${ENV_NAME}.${PRIVATE_HOSTED_ZONE}"
      }
    }
  ]


// Provisioning file
[
    {
        "component_name": "demo-kafka-connect-ui",
        "deployment_type": "aws_ec2",
        "params": {
            "num_instances" : 3,
            "instance_pool": ["t3.medium"]
        }
    }
]

kafka-zookeeper

odin describe component-type --name kafka-zookeeper

Exposed configs in service definition

Config	Description	Data Type	Mandatory	Schema
name	Component name	string	Yes
type	Component type	string	Yes
version	Component version	string	Yes
depends_on	List of components which should be provisioned before kafka zookeeper is provisioned	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}
route.private	DNS where kafka zookeeper will be discoverable	string	No

Exposed configs in provisioning file

Config	Description	Data Type	Mandatory	Schema
deployment_type	Where the component should be deployed. Allowed values: container/aws_ec2	string	No
params.kafka.log_retention_check_interval_ms	Interval in milli seconds at which kafka will check for log segments that are eligible for deletion based on the configured log retention policy	integer	No
params.kafka.log_retention_hours	Number of hours that Kafka will retain log segments before deleting them based on the log retention policy	integer	No
params.instance_pool	List of instances on which kafka will be provisioned	array	No	{ "type": "array", "items": { "type": "string" }, "maxItems": 1, "minItems": 1}
params.kafka.ebs_iops	IOPS for extra ebs block device for kafka	integer	No
params.kafka.ebs_throughput	Ebs extra throughput in MiB/s for kafka	integer	No
params.kafka.ebs_volume_size	Ebs extra xvdb volume size in GB	integer	No
params.kafka.num_instances	Number of instances of kafka	integer	No
params.zookeeper.num_instances_per_placement_group	Number of instances per placement group for zookeeper	integer	No
params.zookeeper.num_placement_groups	Number of placement groups for zookeeper	integer	No

Example service definition and provisioning file

// Some code
{
  "name": "odindemo",
  "version": "1.0.0",
  "team": "devx",
  "components": [
    {
      "type": "kafka-zookeeper",
      "name": "demo-kafka-zookeeper",
      "version": "2.6.0-1.0.0",
      "config": {
        "route": {
          "private": "demo-kafka-${ENV_NAME}.${PRIVATE_HOSTED_ZONE}"
        }
      }
    }
  ]
}

// Provisioning file
[
    {
        "component_name": "demo-kafka-zookeeper",
        "deployment_type": "aws_ec2",
        "params": {
          "instance_pool": ["t3.medium"],
          "kafka": {
            "ebs_extra_iops": 3000,
            "ebs_extra_throughput": 125,
            "ebs_extra_volume_size": 500,
            "num_instances": 3,
            "log_retention_check_interval_ms": 200000,
            "log_retention_hours": 168
          },
          "zookeeper": {
            "num_instances_per_placement_group": 1,
            "num_placement_groups": 3
          }
        }
    }
]

ksql

odin describe component-type --name ksql

Exposed configs in Service definition file

Config	Description	Data Type	Mandatory	Schema
name	component name	string	Yes
type	component type	string	Yes
version	component version	string	Yes
config.route.private	private DNS endpoint of the ksql	string	No
config.bootstrap_server	DNS endpoint of kafka	string	Yes
config.schema_registry_host	DNS endpoint of schema registry	string	Yes
depends_on	List of components on which ksql depends on	array	No	{'type': 'array', 'items': {'type': 'string'}, 'minItems': 1, 'uniqueItems': True}

Exposed configs in Provisioning file

Config	Description	Data Type	Mandatory	Schema
deployment_type	platform on which the component should be deployed	string	No	{'enum': ['container', 'aws_ec2'], 'type': 'string'}
params.instance_pool	instance type pool for ksql autoscaling	array	No	{'type': 'array', 'items': {'type': 'string'}, 'message': {'minItems': 'instance_pool must have atleast 1 element'}, 'minItems': 1, 'uniqueItems': True}
params.num_instances	number of instances for ksql autoscaling	integer	No

Example service definition and provisioning file

// Service definition
"components": [
    {
      "type": "ksql",
      "name": "test-ksql",
      "version": "5.2.1-1.0.0",
      "config": {
        "bootstrap_server": "test-kafka.domain.local",
        "schema_registry_host": "test-schema-registry.domain.local"
      }
    }
  ]


// Provisioning file
[
    {
        "component_name": "test-ksql",
        "deployment_type": "container"
    }
]

Neptune

odin describe component-type --name neptune

Exposed configs in service definition file

Config	Description	Data Type	Mandatory	Schema
name	Component name	string	Yes
type	Component type	string	Yes
version	Component version	string	Yes
depends_on	List of components on which neptune depends on	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}
config.route.reader.private	Route on which neptune reader is discoverable	string	No	{"type": "string", "pattern": "[a-zA-Z._-]+\${TEAM_SUFFIX}\.dream11\${VPC_SUFFIX}\.local"}
config.route.writer.private	Route on which neptune writer is discoverable	string	No	{"type": "string", "pattern": "[a-zA-Z._-]+\${TEAM_SUFFIX}\.dream11\${VPC_SUFFIX}\.local"}

Exposed config in provisioning file

Config	Description	DataType	Mandatory	Schema
deployment_type	platform on which the component should be deployed	String	No	Allowed values: [container,aws_neptune]
params.cluster_parameters	cluster parameter config	Object	No	{ "type": "array", "items": { "type": "object", "required": [ "name", "value" ], "properties": { "name": { "type": "string" }, "value": {}, "apply_immediately": { "enum": [ "pending-reboot", "immediate" ], "type": "string" } } } }
params.instance_parameters	instance parameter config	Object	No	{ "type": "array", "items": { "type": "object", "required": [ "name", "value" ], "properties": { "name": { "type": "string" }, "value": {}, "apply_immediately": { "enum": [ "pending-reboot", "immediate" ], "type": "string" } } } }
params.num_instances	Number of instances in the neptune cluster	Integer	No
params.replica_count	Number of replicas in the cluster. Only applicable for env of type containers.	Integer	No

postgresql

odin describe component-type --name postgresql

Exposed configs in Service definition

Config	Description	Data Type	Mandatory	Schema
name	Component name	string	Yes
type	Component type	string	Yes
version	Component version	string	Yes
route.reader.private	DNS where postgresql reader will be discoverable	string	No	{"type": "string", "pattern": "[a-zA-Z._-]+\${TEAM_SUFFIX}\.dream11\${VPC_SUFFIX}\.local"}
route.writer.private	DNS where postgresql writer will be discoverable	string	No	{"type": "string", "pattern": "[a-zA-Z._-]+\${TEAM_SUFFIX}\.dream11\${VPC_SUFFIX}\.local"}
depends_on	List of components which should be provisioned before postgresql is provisioned	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}

Exposed config in Provisioning file

Config	Description	Data Type	Mandatory	Schema
deployment_type	Where the component should be deployed. Allowed values: container, aws_rds_aurora	string	No
params.cluster_parameter_group_name	Cluster parameter group to associate with the RDS cluster	string	No
params.instance_type	Type of instance to use for the RDS cluster instance	string	No
params.parameter_group_name	Parameter group to associate with the RDS cluster instance	string	No
params.replica_count	Number of instances to create for the RDS database cluster	integer	No

rds

odin describe component-type --name rds

Exposed configs in service definition

Config	Description	Data Type	Mandatory	Schema
name	component name	string	Yes
type	component type	string	Yes
version	component version	string	Yes
config.binlog.enable	Enable/Disable binlog	string	No	{"enum": ["Yes", "No"], "type": "string"}
config.route.reader.private	Route on which rds reader is discoverable	string	No	{"type": "string", "pattern": "[a-zA-Z._-]+\${TEAM_SUFFIX}\.dream11\${VPC_SUFFIX}\.local"}
config.route.writer.private	Route on which rds writer is discoverable	string	No	{"type": "string", "pattern": "[a-zA-Z._-]+\${TEAM_SUFFIX}\.dream11\${VPC_SUFFIX}\.local"}
depends_on	List of components on which RDS Component depends on	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}

Exposed configs in provisioning file

Config	Description	Data Type	Mandatory	Schema	Default Value
deployment_type	Platform on which the component should be deployed. Allowed values: container, aws_rds_aurora	string	No	{"enum": ["container", "aws_rds_aurora"], "type": "string"}
params.cluster_parameter_group_name	Parameter group for RDS cluster	string	No
params.engine_version	Refers to the specific release of a database engine software that is used to run the RDS instance	string	No
params.instance_type	Instance type for RDS eg. db.r5.xlarge	string	No		db.r5.large
params.kms_key_id	To specify a KMS key ID for an RDS instance	string	No
params.parameter_group_name	Parameter group for RDS database instances	string	No
params.replica_count	Refers to the number of reader nodes	integer	No		1
params.resources.limits.cpu	Maximum amount of CPU that a container can use	string	No		200m
params.resources.limits.ephemeral-storage	Maximum ephemeral storage for functions such as caching, scratch space, and logs that a container can use	string	No		512Mi
params.resources.limits.memory	Maximum amount of memory that a container can use	string	No		2Gi
params.resources.requests.cpu	Desired amount of CPU that a container can use	string	No		100m
params.resources.requests.ephemeral-storage	Desired amount of ephemeral storage for functions such as caching, scratch space, and logs that a container can use	string	No		256Mi
params.resources.requests.memory	Desired amount of memory that a container can us	string	No		1Gi
params.snapshot_identifier	Snapshot name which needs be used for provisioning the cluster	string	No

Example service config and definition file

// Service definition
"components": [
    {
      "type": "rds",
      "name": "demo-rds",
      "version": "8.0.23-1.0.0",
      "depends_on": [
        "demo-component"  // Given this component is also present in service definition
      ],
      "config": {
        "route": {
          "private": "demo-rds-${ENV_NAME}.${PRIVATE_HOSTED_ZONE}",
          "public": "demo-rds-${ENV_NAME}.${PUBLIC_HOSTED_ZONE}"
        },
        "binlog": {
          "enable": "Yes"
        }
      }
    }
  ]


// Provisioning file
[
    {
        "component_name": "demo-rds",
        "deployment_type": "container",
        "params": {
            "instance_type" : "db.r5.large",
            "replica_count" : 1,
            "resources": {
                "limits": {
                    "cpu": "200m",
                    "memory": "512Mi",
                    "ephemeral-storage": "2Gi"
                },
                "requests": {
                  "cpu": "100m",
                  "memory": "256Mi",
                  "ephemeral-storage": "1Gi"
                }
            }
        }
    }
]

rabbitmq

odin describe component-type --name rabbitmq

Exposed configs in service definition

Config	Description	Data Type	Mandatory	Schema
config.route.private	Private DNS endpoint for the component	string	No	{"type": "string", "pattern": "[a-zA-Z._-]+\${TEAM_SUFFIX}\.dream11\${VPC_SUFFIX}\.local"}
depends_on	List of components that this component depends on	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}
name	Name of the component	string	Yes
type	Type of the component	string	Yes
version	Version of the component	string	Yes

Exposed configs in provisioning file

Config	Description	Data Type	Mandatory	Schema
deployment_type	Where the component should be deployed. Allowed values: container/aws_ec2	string	No	{ "enum": [ "container", "aws_ec2" ], "type": "string" }
params.instance_type	EC2 Instance type for the component	string	No
params.num_instances	Number of instances for the component	integer	No

// Service definition
"components": [
    {
      "type": "rabbitmq",
      "name": "demo-rabbitmq",
      "version": "3.8.2-1.0.0",
      "depends_on": [
        "demo-component"  // Given this component is also present in service definition
      ],
      "config": {
        "route": {
          "private": "demo-rmq-${ENV_NAME}.${PRIVATE_HOSTED_ZONE}"
        }
      }
    }
  ]


// Provisioning file
[
    {
        "component_name": "demo-rabbitmq",
        "deployment_type": "aws_ec2",
        "params": {
            "instance_type" : "t3.medium",
            "num_instances" : 2
        }
    }
]

s3

odin describe component-type --name s3

Exposed configs in service definition

Config	Description	Data Type	Mandatory	Default Value	Schema
name	Component name	string	Yes
type	Component type	string	Yes
version	Component version	string	Yes
config.bucket_name	Name of the s3 bucket to be created. Should be unique	string	Yes
config.access	S3 bucket-level access configuration. Allowed values: private/public	string	No	private	{"enum": ["private", "public"], "type": "string"}
config.force_destroy	Whether all objects in s3 bucket be deleted when undeploying. Allowed values: true/false	string	No	true	{"enum": ["true", "false"]}
config.versioning	Whether to allow versioning of objects in the bucket. Allowed values: Enabled/Disabled	string	No	Enabled	{"enum": ["Enabled", "Disabled"], "type": "string"}

Example service definition and provisioning file

// Service definition
"components": [
    {
      "type": "s3",
      "name": "demo-s3",
      "version": "1.0.0",
      "config": {
        "bucket_name": "demo-s3${TEAM_SUFFIX}",
        "access": "private",
        "force_destroy": "true,
        "versioning": "Enabled"
      }
    }
  ]
  
// s3 components are not present in provisioning file as no config is exposed 

add TEAM_SUFFIX placeholder at the end of the bucket name to have separate buckets for each Environment.

ex: bucket_name${TEAM_SUFFIX}

schema-registry

odin describe component-type --name schema-registry

Exposed configs in service definition

Config	Description	Data Type	Mandatory	Schema
config.bootstrap_server	DNS of kafka broker	string	Yes
config.route.private	DNS where schema registry will be discoverable	string	No
depends_on	List of components which should be provisioned before schema registry is provisioned	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}
name	Component name	string	Yes
type	Component type	string	Yes
version	Component version	string	Yes

Exposed configs in provisioning file

Config	Description	Data Type	Mandatory	Schema
deployment_type	Where the component should be deployed. Allowed values: container/aws_ec2	string	Yes
params.instance_pool	List of instances on which schema registry will be provisioned	array	No	{ "type": "array", "items": { "type": "string" }, "maxItems": 1, "minItems": 1}
params.num_instances	Number of instances of schema registry	integer	No

Example service definition and provisioning file

// Service definition
"components": [
    {
      "type": "schema-registry",
      "name": "demo-schema-registry",
      "version": "5.2.1-1.0.0",
      "depends_on": [
        "demo-kafka"  // Given this component is also present in service definition
      ],
      "config": {
        "bootstrap_server": "demo-kafka-${ENV_NAME}.${PRIVATE_HOSTED_ZONE}",
        "route": {
          "private": "demo-sr-${ENV_NAME}.${PRIVATE_HOSTED_ZONE}"
        }
      }
    }
  ]


// Provisioning file
[
    {
        "component_name": "demo-schema-registry",
        "deployment_type": "aws_ec2",
        "params": {
            "num_instances" : 3,
            "instance_pool": ["t3.medium"]
        }
    }
]

scylladb

odin describe component-type --name scylladb

Exposed configs in service definition

Config	Description	Data Type	Mandatory	Schema
name	Component name	string	Yes
type	Component type	string	Yes
version	Component version	string	Yes
config.cluster_name	Scylladb cluster name	string	No
depends_on	List of components which should be provisioned before scylladb is provisioned	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}
config.route.private	DNS where scylladb will be discoverable	string	No

Exposed configs in provisioning file

Config	Description	Data Type	Mandatory	Schema
deployment_type	Where the component should be deployed. Allowed values: container/aws_ec2	string	Yes
params.instance_pool	List of instances on which scylladb will be provisioned	array	No	{ "type": "array", "items": { "type": "string" }, "maxItems": 1, "minItems": 1}
params.num_instances	Number of instances of scylladb	integer	No
params.num_seed_nodes	Number of seed nodes of scylladb	integer	No

Example service definition and provisioning file

// Service definition
"components": [
    {
      "type": "scylladb",
      "name": "demo-scylladb",
      "version": "4.5-1.0.0",
      "config": {
        "cluster_name": "scylladb",
        "route": {
          "private": "demo-scylla-${ENV_NAME}.${PRIVATE_HOSTED_ZONE}"
        }
      }
    }
  ]


// Provisioning file
[
    {
        "component_name": "demo-scylladb",
        "deployment_type": "aws_ec2",
        "params": {
            "num_instances" : 3,
            "num_seed_nodes": 3,
            "instance_pool": ["i3.2xlarge"]
        }
    }
]

serverless

odin describe component-type --name serverless

Note: You can provide path to custom script file path using start_script_path in config parameters. If provided, the given script will be execute to deploy serverless. The script file should contain serverless deploy command.

Exposed configs in Service definition

Config	Description	Data Type	Mandatory	Schema
name	Component name	string	Yes
type	Component type	string	Yes
version	Component version	string	Yes
config.artifact_name	Artifact name of the serverless	string	Yes
config.artifact_version	Artifact version of the application	string	Yes
config.filename	Serverless config file name	string	No
config.serverless_version	serverless version	string	No
config.start_script_path	path of script to start the application	string	No
depends_on	List of components on which serverless depends on	array	No	{'type': 'array', 'items': {'type': 'string'}, 'minItems': 1, 'uniqueItems': True}

Exposed configs in provisioning file

Config	Description	Data Type	Mandatoy	Schema
deployment_type	Platform on which the component should be deployed	string	No	{'enum': ['aws_serverless'], 'type': 'string'}
params.extra_envs	Environment variables required for serverless	json	No	{'type': 'array', 'items': {'minLength': 1}, 'uniqueItems': True}

Example service definition and provisioning file

// Service definition
"components": [
    {
      "type": "serverless",
      "name": "test-serverless",
      "version": "1.0.0",
      "config": {
          "artifact_name": "test-serverless",
          "artifact_version": "1.0.0-SNAPSHOT"
      }
    }
  ]


// Provisioning file
[
    {
        "component_name": "test-serverless",
        "deployment_type": "aws_serverless"
    }
]

spark

odin describe component-type --name spark

Exposed configs in Service definition

Config	Description	Data Type	Mandatory	Schema
name	Component name	string	Yes
type	Component type	string	Yes
version	Component version	string	Yes
config.route.private	Private DNS endpoint of Spark application	string	No
depends_on	List of components on which spark depends on	array	No	{'type': 'array', 'items': {'type': 'string'}, 'minItems': 1, 'uniqueItems': True}

Exposed configs in Provisioning file

Config	Description	Data type	Mandatory	Schema
deployment_type	Platform on which the component should be deployed	string	No	{'enum': ['aws_ec2', 'aws_ec2_ha'], 'type': 'string'}
params.disable_api_termination	If true, enables EC2 Instance Termination Protection	string	No
params.ebs_root_volume_size	Spark root ebs volume size	string	No
params.master_instance_pool	Spark master cluster instance type	array	No	{'type': 'array', 'items': {'type': 'string'}, 'message': {'maxItems': 'max length of instance_pool can be 1', 'minItems': 'instance_pool must have atleast 1 element'}, 'maxItems': 1, 'minItems': 1, 'uniqueItems': True}
params.master_num_instances	Number of spark master nodes	string	No
params.worker_instance_pool	Spark worker cluster instance type	array	No	{'type': 'array', 'items': {'type': 'string'}, 'message': {'maxItems': 'max length of instance_pool can be 1', 'minItems': 'instance_pool must have atleast 1 element'}, 'maxItems': 1, 'minItems': 1, 'uniqueItems': True}
params.worker_num_instances	Number of spark worker nodes	string	No
params.history_instance_pool	Spark history cluster instance type	array	No	{'type': 'array', 'items': {'type': 'string'}, 'message': {'maxItems': 'max length of instance_pool can be 1', 'minItems': 'instance_pool must have atleast 1 element'}, 'maxItems': 1, 'minItems': 1, 'uniqueItems': True}
params.history_num_instances	Number of spark history nodes	string	No
params.max_size	Number of spark master nodes	string	No
params.num_on_demand_instances	Absolute minimum amount of desired capacity that must be fulfilled by on-demand instances	string	No
params.on_demand_percentage	Percent of on-demand instances over the number of base on-demand instances	string	No
params.s3_cron	Frequency at which the logs gets pushed into s3 from workers	string	No
params.spark_ha_route	Private DNS endpoint of spark ha	string	No
params.spot_allocation_strategy	How to allocate capacity across the Spot pools	string	No
params.worker_cleanup_ttl	Interval, in seconds, at which the worker cleans up old application work dirs on the local machine	string	No
params.resources.limits.cpu	Maximum amount of CPU that a container can use	string	No
params.resources.limits.ephemeral-storage	Maximum amount of ephemeral storage that a container can use	string	No
params.resources.limits.memory	Maximum amount of memory that a container can use	string	No
params.resources.requests.cpu	Desired amount of CPU that a container can use	string	No
params.resources.requests.ephemeral-storage	Desired amount of ephemeral storage that a container can use	string	No
params.resources.requests.memory	Desired amount of memory that a container can us	string	No

Example service definition and provisioning file

// Service definition
"components": [
    {
      "type": "spark",
      "name": "spark-cluster",
      "version": "2.1.0-1.0.0"
    }
  ]


// Provisioning file
[
    {
        "component_name": "spark-cluster",
        "deployment_type": "container"
    }
]

stepfunction

odin describe component-type --name stepfunction

Exposed configs in service definition

Config	Description	Data Type	Mandatory	Default Value	Schema
name	Component name	string	Yes
type	Component type	string	Yes
version	Component version	string	Yes
config.artifact_name	Artifact name of the stepfunction	string	Yes
config.artifact_version	Artifact version of the stepfunction	string	Yes
config.filename	Stepfunction config file name	string	No	main.yml
config.serverless_version	Serverless verision	string	No	2.57.0
depends_on	List of components on which stepfunction depends on	array	No		{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}

Exposed configs in provisioning file

Config	Description	Data Type	Mandatory	Default Value	Schema
deployment_type	Where the component should be deployed. Allowed values: aws_stepfunction	string	No	aws_stepfunction
params.extra_envs	Environment variables required for stepfunction	json	No		{ "type": "array", "items": { "minLength": 1 }, "uniqueItems": true }

Example service definition and provisioning file

// Service definition
"components": [
    {
      "type": "stepfunction",
      "name": "test-stepfunction",
      "version": "1.0.0",
      "config": {
          "artifact_name": "test-stepfunction",
          "artifact_version": "1.0.0-SNAPSHOT"
      }
    }
  ]


// Provisioning file
[
    {
        "component_name": "test-stepfunction",
        "deployment_type": "aws_stepfunction"
    }
]

voltdb

odin describe component-type --name voltdb

Exposed configs in service definition

Config	Description	Data Type	Mandatory	Schema
config.exporters	Desired exporters configuration	array	No	{"type": "array", "items": [{"type": "object", "required": ["target"], "properties": {"type": {"type": "string"}, "target": {"type": "string"}, "enabled": {"type": "boolean"}, "properties": {"type": "array", "items": [{"type": "object", "required": ["name", "value"], "properties": {"name": {"type": "string"}, "value": {"type": "string"}}}]}}}]}
config.importers	Desired importers configuration	array	No	{"type": "array", "items": [{"type": "object", "required": [], "properties": {"type": {"type": "string"}, "format": {"type": "string"}, "enabled": {"type": "boolean"}, "version": {"type": "integer"}, "properties": {"type": "array", "items": [{"type": "object", "required": ["name", "value"], "properties": {"name": {"type": "string"}, "value": {"type": "string"}}}]}}}]}
config.route.private	Desired private DNS route for the component	string	No	{"type": "string", "pattern": "[a-zA-Z._-]+\${TEAM_SUFFIX}\.dream11\${VPC_SUFFIX}\.local"}
depends_on	List of components on which this component depends on	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}
name	Name of the component	string	Yes
type	Type of the component	string	Yes
version	Version of the component	string	Yes

Exposed configs in provisioning file

Config	Description	Data Type	Mandatory	Schema
deployment_type	Where the component should be deployed. Allowed values: container/aws_ec2	string	No	{ "enum": [ "container", "aws_ec2" ], "type": "string" }
params.num_instances	Desired number of instances	integer	No	{ "type": "integer", "minimum": 3 }
params.instance_type	EC2 Instance type for the component	string	No
params.kfactor	Desired Kfactor of voltdb cluster	integer	No
params.sitesPerHost	Desired sitesPerHost of voltdb cluster	integer	No

// Service definition
"components": [
    {
      "type": "voltdb",
      "name": "demo-voltdb",
      "version": "11.4.3-1.0.0",
      "depends_on": [
        "demo-component"  // Given this component is also present in service definition
      ],
      "config": {
        "route": {
          "private": "demo-vltdb-${ENV_NAME}.${PRIVATE_HOSTED_ZONE}"
        },
        "exporters": [
          {
            "type": "kafka",
            "target": "demo-target",
            "enabled": true,
            "properties": [
              {
                "name": "skipinternals",
                "value": "true"
              },
              {
                "name": "topic.prefix",
                "value": "demo-prefix"
              },
              {
                "name": "bootstrap.servers",
                "value": "demo-server${TEAM_SUFFIX}.dream11${VPC_SUFFIX}.local:9092"
              },
              {
                "name": "acks",
                "value": "1"
              }
            ]
          }
        ],
        "importers": [
          {
            "type": "kafka",
            "version": 10,
            "enabled": true,
            "format": "csv",
            "properties": [
              {
                "name": "topics",
                "value": "demo-topic"
              },
              {
                "name": "procedure",
                "value": "demo-procedure.upsert"
              },
              {
                "name": "brokers",
                "value": "demo-broker${TEAM_SUFFIX}.dream11${VPC_SUFFIX}.local:9092"
              },
              {
                "name": "groupid",
                "value": "demo-groupid"
              }
            ]
          }
        ]
      }
    }
  ]


// Provisioning file
[
    {
        "component_name": "demo-voltdb",
        "deployment_type": "aws_ec2",
        "params": {
            "instance_type" : "t3.medium",
            "num_instances" : 2,
            "kfactor": 3,
            "sitesPerHost": 2
        }
    }
]

zookeeper

odin describe component-type --name zookeeper --version <version>

Exposed configs in service definition

Config	Description	Data Type	Mandatory	Schema
config.route.private	DNS where zookeeper will be discoverable	string	No	{"type": "string", "pattern": "[a-zA-Z._-]+\${TEAM_SUFFIX}\.dream11\${VPC_SUFFIX}\.local"}
depends_on	List of components which should be provisioned before zookeeper is provisioned	array	No	{"type": "array", "items": {"type": "string", "minLength": 1}, "uniqueItems": true}
name	Component name	string	Yes
type	Component type	string	Yes
version	Component version	string	Yes

Exposed configs in provisioning file

Config	Description	Data Type	Mandatory	Schema
deployment_type	Where the component should be deployed. Allowed values: container/aws_ec2	string	Yes
params.num_instances_per_placement_group	Number of zookeeper instances per placement group	integer	No
params.num_placement_groups	Number of placement groups in which zookeeper will be provisioned	integer	No
params.replicaCount	Number of zookeeper pods when deployment type is container	interger	No

Example service definition and provisioning file

// Service definition
"components": [
    {
      "type": "zookeeper",
      "name": "demo-zookeeper",
      "version": "3.4.14-1.0.0",
      "config": {
          "route": {
              "private": "demo-zookeeper-${ENV_NAME}.${PRIVATE_HOSTED_ZONE}"
          }
      }
    }
  ]


// Provisioning file
[
    {
        "component_name": "demo-zookeeper",
        "deployment_type": "aws_ec2",
        "params": {
            "num_instances_per_placement_group" : 1,
            "num_placement_group": 3

        }
    }
]