This folder provides guiendence in how to use splunk-ansible in your own environment. The examples here setup a very basic container, that only exposes port 22 and has NOTHING preinstalled (not even ansible). You can follow this exact workflow with baremetal machines / vm's. In this case, first we'll spin up 4 containers to mimic our base baremetal hosts with ssh installed. I've included a docker-compose file to easily build the image, and spin up the stack. ``` docker-compose -f docker-compose.yml up -d ``` This should stand up the full deployment and create all the required networking. Verify they are all running with docker ps: ``` wrapper-example$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1337ac381424 debian_buster_sshd "/usr/sbin/sshd -D" 5 seconds ago Up 3 seconds 0.0.0.0:32772->22/tcp wrapper-example_cluster_master_1 a325a28ba9ea debian_buster_sshd "/usr/sbin/sshd -D" 5 seconds ago Up 4 seconds 0.0.0.0:32771->22/tcp wrapper-example_indexer3_1 88d8ab42bc11 debian_buster_sshd "/usr/sbin/sshd -D" 5 seconds ago Up 4 seconds 0.0.0.0:32770->22/tcp wrapper-example_indexer1_1 29d73413c155 debian_buster_sshd "/usr/sbin/sshd -D" 5 seconds ago Up 4 seconds 0.0.0.0:32769->22/tcp wrapper-example_indexer2_1 2646ede6484a debian_buster_sshd "/usr/sbin/sshd -D" 6 seconds ago Up 5 seconds 0.0.0.0:32768->22/tcp wrapper-example_search_head_1 ``` Next we'll copy our target key in for passwordless login: ``` ssh-copy-id -i ~/.ssh/mykey root@0.0.0.0 -p ``` Make sure to do the above command for all containers. (Assuming you haven't changed the password for root in the Dockerfile, it's set to "**screencast**") Now lets build an ansible inventory to work with our hosts, I personally am using the yaml version, but you can build your inventory however you'd like. See: https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html. I've attached the sample inventory, ansible_inventory.yaml.sample ``` $vi ansible_inventory.yaml all: vars: ansible_user: root children: splunk_search_head: hosts: wrapper-example_search_head_1: ansible_port: 32768 ansible_host: 0.0.0.0 splunk_cluster_master: hosts: wrapper-example_cluster_master_1: ansible_port: 32772 ansible_host: 0.0.0.0 splunk_indexer: hosts: wrapper-example_indexer1_1: ansible_port: 32770 ansible_host: 0.0.0.0 wrapper-example_indexer2_1: ansible_port: 32769 ansible_host: 0.0.0.0 wrapper-example_indexer3_1: ansible_port: 32771 ansible_host: 0.0.0.0 ``` We're now ready to test our first run. Let's test this with the hello-world.playbook: ``` $ ansible-playbook -vv -i ansible_inventory.yaml hello-world.playbook ``` This playbook will install a simple python-minimal instance, and then send "echo hello_world" to the command line, and store the output in a registered var. By running with -vv on ansible-playbook, we'll be able to see that register in the task: ``` TASK [echo hello_world on each host] **************************************************************************************************************************************************************** task path: /Projects/splunk-ansible/wrapper-example/hello-world.playbook:13 changed: [indexer3] => {"changed": true, "cmd": ["echo", "hello_world"], "delta": "0:00:00.002486", "end": "2019-03-05 20:41:17.468653", "rc": 0, "start": "2019-03-05 20:41:17.466167", "stderr": "", "stderr_lines": [], "stdout": "hello_world", "stdout_lines": ["hello_world"]} changed: [indexer1] => {"changed": true, "cmd": ["echo", "hello_world"], "delta": "0:00:00.002334", "end": "2019-03-05 20:41:17.468749", "rc": 0, "start": "2019-03-05 20:41:17.466415", "stderr": "", "stderr_lines": [], "stdout": "hello_world", "stdout_lines": ["hello_world"]} changed: [cluster_master] => {"changed": true, "cmd": ["echo", "hello_world"], "delta": "0:00:00.002773", "end": "2019-03-05 20:41:17.480882", "rc": 0, "start": "2019-03-05 20:41:17.478109", "stderr": "", "stderr_lines": [], "stdout": "hello_world", "stdout_lines": ["hello_world"]} changed: [indexer2] => {"changed": true, "cmd": ["echo", "hello_world"], "delta": "0:00:00.003453", "end": "2019-03-05 20:41:17.484681", "rc": 0, "start": "2019-03-05 20:41:17.481228", "stderr": "", "stderr_lines": [], "stdout": "hello_world", "stdout_lines": ["hello_world"]} ``` As long as the hello_world example had no failures, we're now ready to setup this index cluster. Please note, if you did not manually log into each container ahead of time, you may get a "host key not verified" error for ssh. Either manually login and approve each container's ssh key, or you can add host_key_checking=False to the ansible-playbook commands. Please setup a defaults.yml file (referenced in the docs) and place it inside of our current working directory, next to the playbooks. In the example below, I'm just going to use the auto-generated defaults off of the splunk container to get started. ``` docker run --rm -it splunk/splunk:latest create-defaults > default.yml ``` The supplied play will setup all of "install-splunk-ansible.playbook" will setup all the required prereqs for splunk-ansible, copy the defaults file to /tmp/defaults/default.yml and prep the install of splunk. Run it now using: ``` ansible-playbook -vv -i ansible_inventory.yaml install-splunk-ansible.playbook ``` Grab some coffee, this might take a bit! Once the play finishes for splunk-ansible, we're now ready to embed splunk-ansible as a module. There's a couple of different ways to do this, one you could use the "delegate_to" function of an ansible playbook command, or two, we tell ansible to run in an async method. The install-splunk.playbook, runs in the form of the latter. You can now create different default.yml for each role of splunk, or override the options in your playbook for each group. Once your playbook is configured, run: ``` ansible-playbook -vv -i ansible_inventory.yaml install-splunk.playbook ``` *PLEASE NOTE: If you don't have a machine fast enough to handle 5 instances of splunk in containers starting, you may hit timeouts during the installation!* You should now have a setup splunk instance, configured entirely asynchronously and utilizing splunk-ansible without needing to touch splunk-ansible's inventory directly. You're free now to connect to your searchhead / cluster_master port 8000's that are exposed!