Open topic with navigation
VictorOps is a real-time incident management and collaboration platform for IT and DevOps teams. VictorOps supports triggering and resolving incidents via email, so you can route AlertSite email alerts to VictorOps to automatically create and resolve incidents based on the status of your AlertSite monitors. This topic explains how to set up the communication between AlertSite and VictorOps.
Note: VictorOps integration is only possible on AlertSite Enterprise plans.
First of all, you need to enable the email endpoint in VictorOps to be able to trigger and resolve incidents via email. To do that:
Select Email Endpoint on the right and click Enable Integration.
This will generate an email address to which you can send email alerts. Note down this address – you will need to specify it in AlertSite.
The email address looks like this:
The $routing_key should be replaced with the key of a team to which you want to route the alerts. For example:
Team routing keys are configured at the bottom of the Settings > Integrations page. For details, see Routing Keys in the VictorOps knowledge base.
If you do not use routing, remove the +$routing_key part (including the plus sign) so that the email looks like this:
When using the VictorOps email endpoint, the email subject must include specific keywords - CRITICAL or PROBLEM to open a new incident, and RESOLVED or OK to resolve an incident. You can customize AlertSite email alerts to include these keywords.
Note: The email subject line must be the same in both the “error” and “clear” templates (apart from the CRITICAL/PROBLEM and RESOLVED/OK words). This is needed for VictorOps to recognize that the “clear” email is related to the incident opened by the “error” email. For example, do not use the $STATUS variable in the subject line, because the status code will be different in the “error” and “clear” alerts, and, in this case, VictorOps will not be able to match these alerts.
To create custom alert templates for VictorOps:
On the list, select the AlertSite Template for Site Error with the delivery method Text.
Subject: [AlertSite] Monitor Alert - $DESCRIP CRITICAL
Note: You can use another subject, but make sure it includes the word CRITICAL and does not include the $STATUS variable.
Select the AlertSite Template for Site Clear with the delivery method Text.
Important: The subject must be exactly the same as in the error template, but with the OK word instead of CRITICAL.
You can see the created templates on the template list:
Next, you need to add the VictorOps email endpoint that you generated earlier as an alert recipient in AlertSite:
Mode – E-mail (text format)
On the Availability Alerts tab, set the option to Alert whenever an error clears. This is needed to automatically resolve incidents in VictorOps when an error clears in AlertSite.
Now, you need to assign your custom alert templates to the VictorOps alert recipient. To do this, you need to create a recipient group that contains the VictorOps recipient, the custom templates, and the monitor whose alerts you want to send to VictorOps:
Click + New Recipient Group.
Under Custom Templates, add the VictorOps “error” and “clear” templates that you created earlier.
Now, the specified monitors will send alerts to VictorOps.
You can view AlertSite alerts in the VictorOps Timeline and Incidents tabs:
Click More info to view the alert contents, including the description of the error occurred:
The team members can then acknowledge the incidents and take action to resolve them. The incidents will also be resolved automatically when a “clear” notification arrives from AlertSite.