{"_id":"56eb25fa450cf10e00de5675","user":"55c50f4a7c199a2f00665cbf","category":{"_id":"55d3b645f77e6d0d00b1b27b","__v":6,"pages":["55d3b64bf77e6d0d00b1b2a6","55d3b64bf77e6d0d00b1b2a7","55d78faf60fc730d00fc2839","565cc898687b0d0d004101e4","565ccbcdf8bc860d006842ee","56eb25fa450cf10e00de5675","56eb5d051633510e00f884b2"],"project":"55c505b41469ad2500fa2ab7","version":"55d3b644f77e6d0d00b1b273","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2015-08-10T18:45:16.066Z","from_sync":false,"order":2,"slug":"agent","title":"Agent"},"project":"55c505b41469ad2500fa2ab7","__v":119,"parentDoc":null,"version":{"_id":"55d3b644f77e6d0d00b1b273","project":"55c505b41469ad2500fa2ab7","__v":6,"createdAt":"2015-08-18T22:48:36.632Z","releaseDate":"2015-08-18T22:48:36.632Z","categories":["55d3b645f77e6d0d00b1b274","55d3b645f77e6d0d00b1b275","55d3b645f77e6d0d00b1b276","55d3b645f77e6d0d00b1b277","55d3b645f77e6d0d00b1b278","55d3b645f77e6d0d00b1b279","55d3b645f77e6d0d00b1b27a","55d3b645f77e6d0d00b1b27b","55d3b645f77e6d0d00b1b27c","55d3b645f77e6d0d00b1b27d","55d7c2939510f00d007ec6fe","56fac9925df15a20002972a2","56fb2f7668e1d30e00a0b672","583498d411e8af2500f6b334","58e52a180ab7b03b00f4a97a"],"is_deprecated":false,"is_hidden":false,"is_beta":true,"is_stable":true,"codename":"","version_clean":"1.1.0","version":"1.1"},"updates":["591d6e76a266c423002ebb82"],"next":{"pages":[],"description":""},"createdAt":"2016-03-17T21:47:38.021Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":5,"body":"[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"General\"\n}\n[/block]\n**What is Neptune agent and why is it required?**\nNeptune agent is a lightweight piece of software which runs on your servers to run runbooks and report the results to Neptune.io service. We use agents to accomplish this because they provide an easy way to run runbooks without exposing any open ports and without requiring SSH access to your servers.\n\n**Is agent mandatory to use Neptune service?**\nNo. You can use Neptune without installing any agents but you will not be able to run your runbooks as Neptune actions. However, you can use REST API actions, CLI actions, Graph snapshot actions, webpage snapshot actions, etc. \n\nSpecially, AWS, Heroku, DigitalOcean and Softlayer customers can derive lot of value without installing agents, given the power of their CLIs.\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Requirements\"\n}\n[/block]\n**Is there any prerequisite software?**\nThe Neptune agent is a self-contained executable that has no other dependencies.\n\n**What operating systems can I run the agent on?**\nWe currently support all Linux, Darwin, Windows and ARM architectures. See our installation docs for more details.\n\n**Does Neptune need SSH access to my servers?**\nNo – since the Agent runs the runbook directly on server, Neptune does not need SSH access to your server.\n\n**What ports need to be opened in my firewall for Neptune agent to work?**\nNeptune agent only uses HTTPS to talk to Neptune.io service and to AWS SQS hence it only requires 443 port to be opened for outbound connections. \n\n**How much RAM/CPU does the agent consume?**\nThe agent consumes less than 0.1% of CPU and less than 20MB of RAM in general. Unlike monitoring agents that need to keep sending metric information every minute, Neptune agent doesn’t do anything every minute. It only polls queue periodically (that too using SQS long polling technique) and only does something if there is an action that needs to be performed on that host. So, it doesn’t do data gathering automatically unless it is told to so via an action.\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Installation\"\n}\n[/block]\n**How to install/uninstall Neptune agent?**\nPlease see [Agent Installation](doc:agent-installation) for details on agent installation.\n\n**When should I upgrade Neptune agents?**\nOccasionally, agent needs to be upgraded to provide any new functionality or fixes. We recommend you to keep it up to date to take advantage of these improvements. See [Agent-Administration](doc:agent-administration) for the instructions to install/upgrade the agent.\n\n** Does the agent auto-upgrade itself? **\nNo – it doesn’t. However, You can upgrade anytime within a one-liner script.\n\nDoes the agent have any dependencies? \nNo – the agent is written in GO language, so it can be single binary with ZERO dependencies. You need not worry upgrading any software; it just works as a standalone binary. Also, the agent is open source and you can checkout the source at: https://github.com/neptuneio/agent\n\n**Will Neptune agent make any changes to my host configuration?**\nNo – Neptune agent only runs the runbooks it is told to run. However, if your runbook has commands to modify the state of host and if you have given the Agent permissions to run those commands, it's possible that the host state might change. But, we assume you know what you are doing in that case.\n\n**How to troubleshoot Neptune agent failures?**\nNeptune agent logs all the failures to log file so you should be able to find the issues from logs. For convenience, we ship Agent errors to Neptune.io service so that you can see all agent logs from the app itself with a click. Please check servers tab in the app.\n\n**How can I see list of all running Neptune agents?**\nYou can see all the installed agents in Servers tab in Neptune app. You can also see the metadata like host information, start time, IP address, last heartbeat time and logs in the app.\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Permissions and Configuration\"\n}\n[/block]\n**What user does Neptune agent run as? What privileges does it have by default?**\nNeptune agent by default runs as “*neptune*” user. However, you can dictate as what user the agent should run as by setting AGENT_USER during agent installation.\n\nAgent by default will run as a normal user without sudo permissions. To give higher permissions to the agent user do one of the following.\n * Configure agent to run as an alternative user who has more permissions (assuming you have such user configured) \n * Add *neptune* user in a user group that has more privileges\n\nAlternatively, to give sudo permissions to the agent, just reinstall the agent by setting REQUIRE_SUDO=true.\n\n**Does Neptune agent need root permissions?**\nNo - Neptune agent does not need to run as root or sudo user. However, you need root privileges to install the agent since the agent daemon needs to be registered as a service.\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Security\"\n}\n[/block]\n**What if the SQS queue tokens are compromised?**\nFirstly, the SQS queue credentials are sent to Agent in response to registration request over HTTPS only and they are never stored or logged anywhere. On top of it, Neptune uses AWS STS to rotate these tokens once every 4 hours. Agent always syncs up with the latest credentials as part of periodic re-registration protocol which is initiated by Agent always.\n\nAdditionally, the SQS credentials have permissions to only read and delete messages from the queue so no one can post new SQS messages to your action queue using these credentials.\n\n**What if someone snoops and replays the messages to SQS queue?**\nAgent does bunch of checks before processing every action and those checks guard against various security/failure scenarios. The checks are:\n * Agent processes the actions only after verifying the signature and only Neptune.io service has private key to sign messages.\n * Agent verifies that the message is for itself by checking its id in the message. Any corruption observed in message would lead to dropping the action.\n * Agent checks the action creation time and discards stale actions. This guards against the old messages being replayed.\n * Agent has a persistent store to save all the processed events and hence it identifies duplicates and discards them.\n\n**Can a hacker run random runbooks on my host using Neptune agent?**\nLet's say some crazy hacker wants to run random runbooks on your host using Neptune agent. These are all the measures taken against that doing.\n\n * Only Neptune service has credentials to write to SQS queue so Neptune needs to be compromised first.\n * Every action message is signed before writing to SQS (please note that AWS SDK also signs messages using SQS credentials but we are talking about an additional message signing using Neptune.io's key pair here). So, the hacker need to get our private key as well to properly sign messages. Neptune's private key is encrypted using a pass phrase adding an additional layer of security.\n * Agent will process the message only if the message is for itself (containing a guid). That means the hacker need to guess the exact guid of agent.\n * If you are using Github runbook model, action message contains only a pointer to Github runbook, not runbook content. This pointer needs to be guessed correctly.\n * Assuming everything went wrong till here, which is veeeery difficult, hacker can run one of the runbooks that you already have in your Github repo but he cannot run random runbook.\n\n**What if my Neptune API key is compromised?**\nNeptune agent uses API key to register itself with the service. If your Neptune API key is compromised, someone could bring up a new agent and fake that it's one of your agents. However, since agent only has read and delete permissions on SQS queue, they can, at the worst, only delete your action messages causing your actions to be lost. But, they can never execute unwanted runbooks on your machines. We have also measures against faking that agent as one of your agents.\n\n**Who all can access my SQS action queue?**\nAction queues are isolated for each customer. Neptune will have permissions to write to SQS queue and only your agents will have read and delete permissions to the queue. AWS does a wonderful job of making sure one can access a resource only if they have correct permissions.\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Architecture and more\"\n}\n[/block]\n** I am curious. Can you go deeper into how Neptune agent works?**\nHere is an architecture of Neptune agent.\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/q9mKWHxuQD6klljRwlFx_Screen%20Shot%202016-03-20%20at%204.28.57%20PM.png\",\n        \"Screen Shot 2016-03-20 at 4.28.57 PM.png\",\n        \"1194\",\n        \"892\",\n        \"#b68b6e\",\n        \"\"\n      ],\n      \"caption\": \"Neptune Agent architecture\"\n    }\n  ]\n}\n[/block]\nHere is the life cycle of Neptune agent in a nutshell:\n\n* Agent bootstraps with Neptune API key and Github repo details (in case Agent is configured to use Github runbooks)\n* Registers with Neptune.io service to get a unique id and security tokens for action queue (AWS SQS)\n* Starts off threads to do periodic re-registration and heartbeating with Neptune service\n* Polls the SQS queue for messages using long polling technique\n* When an agent receives a SQS message:\n * Check if the message is for self and release the message if not\n * Verify the Neptune signature on message and process further only if signature is verified\n * Double check the message is for self based on a field present in the payload (which is part of signature)\n * Check and discard stale messages\n * If the agent is configured to use Github runbooks, fetch runbook from Github\n * Execute the runbook and publish results back to Neptune service.\n\nPlease see [SQS FAQs](https://aws.amazon.com/sqs/faqs/) for more details about SQS.\n\n**How long does it take for the Agent to start the runbook execution?**\nIn less than couple of seconds after your monitoring tool has fired the alert. Though agent uses pull approach to get the actions from SQS queue, there is no delay because it uses SQS's long polling technique. Long polling helps to cut down the resource utilization for polling when there are no messages but makes sure the messages are immediately received when queue is non-empty. See [SQS Long Polling](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-long-polling.html) for more details.\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Need help?\"\n}\n[/block]\nIf you have any trouble, don’t hesitate to contact our [support](mailto:support:::at:::neptune.io) or you can chat with one of our engineers any time using the help icon at the bottom right of the window in Neptune app.","excerpt":"","slug":"agent-faqs","type":"basic","title":"FAQs"}
[block:api-header] { "type": "basic", "title": "General" } [/block] **What is Neptune agent and why is it required?** Neptune agent is a lightweight piece of software which runs on your servers to run runbooks and report the results to Neptune.io service. We use agents to accomplish this because they provide an easy way to run runbooks without exposing any open ports and without requiring SSH access to your servers. **Is agent mandatory to use Neptune service?** No. You can use Neptune without installing any agents but you will not be able to run your runbooks as Neptune actions. However, you can use REST API actions, CLI actions, Graph snapshot actions, webpage snapshot actions, etc. Specially, AWS, Heroku, DigitalOcean and Softlayer customers can derive lot of value without installing agents, given the power of their CLIs. [block:api-header] { "type": "basic", "title": "Requirements" } [/block] **Is there any prerequisite software?** The Neptune agent is a self-contained executable that has no other dependencies. **What operating systems can I run the agent on?** We currently support all Linux, Darwin, Windows and ARM architectures. See our installation docs for more details. **Does Neptune need SSH access to my servers?** No – since the Agent runs the runbook directly on server, Neptune does not need SSH access to your server. **What ports need to be opened in my firewall for Neptune agent to work?** Neptune agent only uses HTTPS to talk to Neptune.io service and to AWS SQS hence it only requires 443 port to be opened for outbound connections. **How much RAM/CPU does the agent consume?** The agent consumes less than 0.1% of CPU and less than 20MB of RAM in general. Unlike monitoring agents that need to keep sending metric information every minute, Neptune agent doesn’t do anything every minute. It only polls queue periodically (that too using SQS long polling technique) and only does something if there is an action that needs to be performed on that host. So, it doesn’t do data gathering automatically unless it is told to so via an action. [block:api-header] { "type": "basic", "title": "Installation" } [/block] **How to install/uninstall Neptune agent?** Please see [Agent Installation](doc:agent-installation) for details on agent installation. **When should I upgrade Neptune agents?** Occasionally, agent needs to be upgraded to provide any new functionality or fixes. We recommend you to keep it up to date to take advantage of these improvements. See [Agent-Administration](doc:agent-administration) for the instructions to install/upgrade the agent. ** Does the agent auto-upgrade itself? ** No – it doesn’t. However, You can upgrade anytime within a one-liner script. Does the agent have any dependencies? No – the agent is written in GO language, so it can be single binary with ZERO dependencies. You need not worry upgrading any software; it just works as a standalone binary. Also, the agent is open source and you can checkout the source at: https://github.com/neptuneio/agent **Will Neptune agent make any changes to my host configuration?** No – Neptune agent only runs the runbooks it is told to run. However, if your runbook has commands to modify the state of host and if you have given the Agent permissions to run those commands, it's possible that the host state might change. But, we assume you know what you are doing in that case. **How to troubleshoot Neptune agent failures?** Neptune agent logs all the failures to log file so you should be able to find the issues from logs. For convenience, we ship Agent errors to Neptune.io service so that you can see all agent logs from the app itself with a click. Please check servers tab in the app. **How can I see list of all running Neptune agents?** You can see all the installed agents in Servers tab in Neptune app. You can also see the metadata like host information, start time, IP address, last heartbeat time and logs in the app. [block:api-header] { "type": "basic", "title": "Permissions and Configuration" } [/block] **What user does Neptune agent run as? What privileges does it have by default?** Neptune agent by default runs as “*neptune*” user. However, you can dictate as what user the agent should run as by setting AGENT_USER during agent installation. Agent by default will run as a normal user without sudo permissions. To give higher permissions to the agent user do one of the following. * Configure agent to run as an alternative user who has more permissions (assuming you have such user configured) * Add *neptune* user in a user group that has more privileges Alternatively, to give sudo permissions to the agent, just reinstall the agent by setting REQUIRE_SUDO=true. **Does Neptune agent need root permissions?** No - Neptune agent does not need to run as root or sudo user. However, you need root privileges to install the agent since the agent daemon needs to be registered as a service. [block:api-header] { "type": "basic", "title": "Security" } [/block] **What if the SQS queue tokens are compromised?** Firstly, the SQS queue credentials are sent to Agent in response to registration request over HTTPS only and they are never stored or logged anywhere. On top of it, Neptune uses AWS STS to rotate these tokens once every 4 hours. Agent always syncs up with the latest credentials as part of periodic re-registration protocol which is initiated by Agent always. Additionally, the SQS credentials have permissions to only read and delete messages from the queue so no one can post new SQS messages to your action queue using these credentials. **What if someone snoops and replays the messages to SQS queue?** Agent does bunch of checks before processing every action and those checks guard against various security/failure scenarios. The checks are: * Agent processes the actions only after verifying the signature and only Neptune.io service has private key to sign messages. * Agent verifies that the message is for itself by checking its id in the message. Any corruption observed in message would lead to dropping the action. * Agent checks the action creation time and discards stale actions. This guards against the old messages being replayed. * Agent has a persistent store to save all the processed events and hence it identifies duplicates and discards them. **Can a hacker run random runbooks on my host using Neptune agent?** Let's say some crazy hacker wants to run random runbooks on your host using Neptune agent. These are all the measures taken against that doing. * Only Neptune service has credentials to write to SQS queue so Neptune needs to be compromised first. * Every action message is signed before writing to SQS (please note that AWS SDK also signs messages using SQS credentials but we are talking about an additional message signing using Neptune.io's key pair here). So, the hacker need to get our private key as well to properly sign messages. Neptune's private key is encrypted using a pass phrase adding an additional layer of security. * Agent will process the message only if the message is for itself (containing a guid). That means the hacker need to guess the exact guid of agent. * If you are using Github runbook model, action message contains only a pointer to Github runbook, not runbook content. This pointer needs to be guessed correctly. * Assuming everything went wrong till here, which is veeeery difficult, hacker can run one of the runbooks that you already have in your Github repo but he cannot run random runbook. **What if my Neptune API key is compromised?** Neptune agent uses API key to register itself with the service. If your Neptune API key is compromised, someone could bring up a new agent and fake that it's one of your agents. However, since agent only has read and delete permissions on SQS queue, they can, at the worst, only delete your action messages causing your actions to be lost. But, they can never execute unwanted runbooks on your machines. We have also measures against faking that agent as one of your agents. **Who all can access my SQS action queue?** Action queues are isolated for each customer. Neptune will have permissions to write to SQS queue and only your agents will have read and delete permissions to the queue. AWS does a wonderful job of making sure one can access a resource only if they have correct permissions. [block:api-header] { "type": "basic", "title": "Architecture and more" } [/block] ** I am curious. Can you go deeper into how Neptune agent works?** Here is an architecture of Neptune agent. [block:image] { "images": [ { "image": [ "https://files.readme.io/q9mKWHxuQD6klljRwlFx_Screen%20Shot%202016-03-20%20at%204.28.57%20PM.png", "Screen Shot 2016-03-20 at 4.28.57 PM.png", "1194", "892", "#b68b6e", "" ], "caption": "Neptune Agent architecture" } ] } [/block] Here is the life cycle of Neptune agent in a nutshell: * Agent bootstraps with Neptune API key and Github repo details (in case Agent is configured to use Github runbooks) * Registers with Neptune.io service to get a unique id and security tokens for action queue (AWS SQS) * Starts off threads to do periodic re-registration and heartbeating with Neptune service * Polls the SQS queue for messages using long polling technique * When an agent receives a SQS message: * Check if the message is for self and release the message if not * Verify the Neptune signature on message and process further only if signature is verified * Double check the message is for self based on a field present in the payload (which is part of signature) * Check and discard stale messages * If the agent is configured to use Github runbooks, fetch runbook from Github * Execute the runbook and publish results back to Neptune service. Please see [SQS FAQs](https://aws.amazon.com/sqs/faqs/) for more details about SQS. **How long does it take for the Agent to start the runbook execution?** In less than couple of seconds after your monitoring tool has fired the alert. Though agent uses pull approach to get the actions from SQS queue, there is no delay because it uses SQS's long polling technique. Long polling helps to cut down the resource utilization for polling when there are no messages but makes sure the messages are immediately received when queue is non-empty. See [SQS Long Polling](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-long-polling.html) for more details. [block:api-header] { "type": "basic", "title": "Need help?" } [/block] If you have any trouble, don’t hesitate to contact our [support](mailto:support@neptune.io) or you can chat with one of our engineers any time using the help icon at the bottom right of the window in Neptune app.