{"id":152,"date":"2023-02-04T11:39:29","date_gmt":"2023-02-04T11:39:29","guid":{"rendered":"https:\/\/mrzebra.co.uk\/code\/?p=152"},"modified":"2023-02-04T11:39:29","modified_gmt":"2023-02-04T11:39:29","slug":"how-to-automate-installation-of-fleet","status":"publish","type":"post","link":"https:\/\/zebra-north.com\/code\/2023\/02\/04\/how-to-automate-installation-of-fleet\/","title":{"rendered":"How to Automate Installation of Fleet"},"content":{"rendered":"\n

Fleet is part of the Elastic Stack, and allows you to manage all your Elastic Agents. Installing it in a fully automated manner is difficult because it requires configuration in Kibana. Here’s how to do it.<\/p>\n\n\n\n\n\n\n\n

Configuring Kibana<\/h2>\n\n\n\n

Kibana’s REST API can be used to set up all the necessary configuration options for the Fleet server.<\/p>\n\n\n\n

A note about SSL: This tutorial includes SSL configuration and supports using certificates signed by your own Certificate Authority (CA), as recommended by the Elasticsearch installation guide. If you are not using SSL then you can remove the --cacert<\/code> options and change https<\/code> to http<\/code>.<\/p>\n\n\n\n

First, we’ll set up some variables. Modify these values to suit your environment:<\/p>\n\n\n\n

# The location of the Certificate Authority file.\n# This is generated by Elasticsearch in the setup container and mounted into the container.\nCA=\/certs\/ca\/ca.crt\n\n# The host and port of the Elasticsearch server.\nELASTIC_HOST=es01:9200\n\n# The username this script uses to connect to Elasticsearch.\nELASTIC_USER=elastic\n\n# The password this script uses to connect to Elasticsearch.\nELASTIC_PASS=elastic\n\n# The URL to the Kibana server, without a trailing slash.\nKIBANA_URL=http:\/\/kibana:5601\n<\/code><\/pre>\n\n\n\n
Service Account Token<\/h3>\n\n\n\n
The Service Account Token is used by Fleet to authorize itself with Elasticsearch.  We will create a token with a random name so that it does not collide with any existing tokens.<\/p>\n\n\n\n
createServiceAccountToken()\n{\n    echo -n 'Creating service account token... '\n\n    # Create a random name for the token so it does not collied with any existing tokens.\n    tokenName=`cat \/proc\/sys\/kernel\/random\/uuid`\n\n    # Create the token.\n    response=`curl \\\n        -u $ELASTIC_USER:$ELASTIC_PASS \\\n        --silent \\\n        --cacert $CA \\\n        -X POST "https:\/\/$ELASTIC_HOST\/_security\/service\/elastic\/fleet-server\/credential\/token\/$tokenName?pretty"`\n\n    # Check that the response was successful.\n    if ! echo "$response" | grep --silent '"created" : true'; then\n        echo 'failed'\n        echo "$response"\n        exit 1\n    fi\n\n    # Parse the token out of the response.\n    token=`echo "$response" | awk '\/value\/ { print $3 }' | sed 's\/"\/\/g'`\n\n    echo "success: $token"\n}<\/code><\/pre>\n\n\n\n
This will store the token in the variable $token<\/code> for use later.<\/p>\n\n\n\n
Agent Policy<\/h3>\n\n\n\n
The Agent Policy configures which integrations (plugins) are associated with the agent.  This will create an empty policy, to which we will later add the Fleet Server integration.<\/p>\n\n\n\n
createAgentPolicy()\n{\n    echo -n 'Creating Fleet server policy... '\n\n    response=`curl \\\n        -u $ELASTIC_USER:$ELASTIC_PASS \\\n        --silent \\\n        --request POST \\\n        --url $KIBANA_URL\/api\/fleet\/agent_policies?sys_monitoring=true \\\n        --header 'content-type: application\/json' \\\n        --header 'kbn-xsrf: true' \\\n        --data '{"id": "docker-fleet-server-policy","name":"Docker Fleet Server Policy","namespace":"default","monitoring_enabled":["logs","metrics"],"is_default_fleet_server": true}'`\n\n    if echo "$response" | grep --silent 'document already exists'; then\n        green 'already exists'\n        return 0\n    fi\n\n    green 'success'\n}<\/code><\/pre>\n\n\n\n
Server Integration<\/h3>\n\n\n\n
The Agent Policy needs the “Fleet Server” integration added to it:<\/p>\n\n\n\n
addServerIntegration()\n{\n    echo -n 'Adding Fleet server integration to policy... '\n\n    response=`curl \\\n        -u $ELASTIC_USER:$ELASTIC_PASS \\\n        --silent \\\n        --request POST \\\n        --url $KIBANA_URL\/api\/fleet\/package_policies \\\n        --header 'content-type: application\/json' \\\n        --header 'kbn-xsrf: true' \\\n        --data '\n        {\n            "policy_id": "fleet-server-policy",\n            "package": {\n                "name": "fleet_server",\n                "version": "1.2.0"\n            },\n            "name": "fleet_server",\n            "description": "",\n            "namespace": "default",\n            "inputs": {\n                "fleet_server-fleet-server": {\n                "enabled": true,\n                "vars": {\n                    "host": "fleet",\n                    "port": [\n                    8220\n                    ],\n                    "custom": ""\n                },\n                "streams": {}\n                }\n            }\n        }'`\n\n    if echo $response | grep --silent 'already exists'; then\n        echo 'already exists';\n        return 0\n    fi\n\n    if echo $response | grep --silent 'error'; then\n        echo 'failed'\n        echo "$response"\n        exit 1\n    fi\n\n    echo 'success'\n}<\/code><\/pre>\n\n\n\n
Server Host<\/h3>\n\n\n\n
The Fleet Server Host setting tells the Elastic Agents the hostname and port on which they can contact the Fleet server.  Make sure to update the URL to match your environment.  The default port for a Fleet server is 8220.<\/p>\n\n\n\n
Note: There is a bug<\/a> in Kibana that will clear the is_default<\/code> option if you try to overwrite an ID that already exists.  If there is a chance that the setting may already exist, you must check first.<\/p>\n\n\n\n
addServerHost()\n{\n    echo -n 'Checking if Fleet server host exists... '\n\n    response=`curl \\\n        -u elastic:elastic \\\n        --silent \\\n        --request GET \\\n        --cacert \/usr\/share\/kibana\/config\/certs\/ca\/ca.crt \\\n        --url http:\/\/kibana:5601\/api\/fleet\/fleet_server_hosts\/fleet-server \\\n        --header 'content-type: application\/json' \\\n        --header 'kbn-xsrf: true'`\n\n    if ! echo $response | grep --silent 'Not Found'; then\n        echo 'already exists'\n        return 0\n    else\n        echo 'does not exist'\n    fi\n\n    echo -n 'Adding Fleet server host... '\n\n    response=`curl \\\n        -u $ELASTIC_USER:$ELASTIC_PASS \\\n        --silent \\\n        --request POST \\\n        --url $KIBANA_URL\/api\/fleet\/fleet_server_hosts \\\n        --header 'content-type: application\/json' \\\n        --header 'kbn-xsrf: true' \\\n        --data '\n        {\n            "id": "fleet-server",\n            "name": "Fleet Server",\n            "is_default": true,\n            "host_urls": [\n                "https:\/\/fleet:8220"\n            ]\n        }'`\n\n    if echo $response | grep --silent 'error'; then\n        echo 'failed'\n        echo "$response"\n        exit 1\n    fi\n\n    echo 'success'\n<\/code><\/pre>\n\n\n\n
Output<\/h3>\n\n\n\n
The Output setting defines where Fleet should send its data to.  In this case, it’s being sent to Elasticsearch.<\/p>\n\n\n\n
setOutput()\n{\n    echo -n 'Setting agent output location... '\n\n    response=`curl \\\n        -u $ELASTIC_USER:$ELASTIC_PASS \\\n        --silent \\\n        --request PUT \\\n        --url $KIBANA_URL\/api\/fleet\/outputs\/fleet-default-output \\\n        --header 'content-type: application\/json' \\\n        --header 'kbn-xsrf: true' \\\n        --data "\n        {\n            \\"name\\": \\"default\\",\n            \\"type\\": \\"elasticsearch\\",\n            \\"is_default\\": true,\n            \\"is_default_monitoring\\": true,\n            \\"hosts\\": [\n                \\"https:\/\/$ELASTIC_HOST\\"\n            ],\n            \\"config_yaml\\": \\"ssl.certificate_authorities: ['\/certs\/ca\/ca.crt']\\"\n        }"`\n\n    if echo "$response" | grep --silent 'document already exists'; then\n        echo 'already exists'\n        return 0\n    fi\n\n    if echo $response | grep --silent 'error'; then\n        echo 'failed'\n        echo "$response"\n        exit 1\n    fi\n\n    echo 'success'\n}<\/code><\/pre>\n\n\n\n
Installing Fleet<\/h2>\n\n\n\n
Now that Kibana has been configured, Fleet can be installed.  Be sure to edit the --url=<\/code> parameter to the hostname of your fleet server.<\/p>\n\n\n\n
installFleet()\n{\n    apt update\n    apt install -y curl\n    curl -L -O https:\/\/artifacts.elastic.co\/downloads\/beats\/elastic-agent\/elastic-agent-8.6.0-linux-x86_64.tar.gz\n    tar xzvf elastic-agent-8.6.0-linux-x86_64.tar.gz\n    cd elastic-agent-8.6.0-linux-x86_64\n\n    .\/elastic-agent install \\\n        --non-interactive \\\n        --fleet-server-es=https:\/\/$ELASTIC_HOST \\\n        --fleet-server-service-token=$token  \\\n        --fleet-server-policy=docker-fleet-server-policy \\\n        --fleet-server-es-ca=\/certs\/ca\/ca.crt \\\n        --fleet-server-cert=\/certs\/fleet\/fleet.crt \\\n        --fleet-server-cert-key=\/certs\/fleet\/fleet.key \\\n        --certificate-authorities=\/certs\/ca\/ca.crt \\\n        --url=https:\/\/fleet:8220\n}<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"
Fleet is part of the Elastic Stack, and allows you to manage all your Elastic Agents. Installing it in a fully automated manner is difficult because it requires configuration in Kibana. Here’s how to do it.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[21],"class_list":["post-152","post","type-post","status-publish","format-standard","hentry","category-elastic-stack","tag-fleet"],"_links":{"self":[{"href":"https:\/\/zebra-north.com\/code\/wp-json\/wp\/v2\/posts\/152","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/zebra-north.com\/code\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/zebra-north.com\/code\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/zebra-north.com\/code\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/zebra-north.com\/code\/wp-json\/wp\/v2\/comments?post=152"}],"version-history":[{"count":1,"href":"https:\/\/zebra-north.com\/code\/wp-json\/wp\/v2\/posts\/152\/revisions"}],"predecessor-version":[{"id":153,"href":"https:\/\/zebra-north.com\/code\/wp-json\/wp\/v2\/posts\/152\/revisions\/153"}],"wp:attachment":[{"href":"https:\/\/zebra-north.com\/code\/wp-json\/wp\/v2\/media?parent=152"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/zebra-north.com\/code\/wp-json\/wp\/v2\/categories?post=152"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/zebra-north.com\/code\/wp-json\/wp\/v2\/tags?post=152"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}