In this article
How to maximize value with Jira and AWS Lambda integration
One of our engineers on the TechOps team coined the term “Value++.” It references the shorthand operator for “increment” in various coding languages. It is also a motto for what we should be doing as a team—always adding value.
Here are a few things in our day-to-day operations that have been a serious “value –”
- Answering JIRA tickets that have no description
- “Customer has issue B,” but both the customer name and issue details are omitted from that sentence
- Doing things manually, over and over again
At LogicMonitor, most of the tasks requested of the TechOps team come in the form of JIRA tickets. A new application may be ready for deployment, or a customer account may require a rename. We also have to deal with operational tasks like moving new customer accounts from demo to production environments.
Because LogicMonitor is rapidly growing, we always try to be more efficient by automating ourselves out of work. We decided to automate parts of our DevOps tasks through AWS Lambda functions, API calls, and JIRA tickets. This allows the team to keep track of existing tasks that show up in our queue and spend their time doing more important things.
It’s “Value ++.”
Key takeaways
Understanding projects and issue types for automation
We first had to lock down specific JIRA projects and issue types to differentiate tasks from other items, creating a separate issue type for every task we wanted to automate. This makes things easy to organize and allows us to lock down who can or cannot make specific tickets.
In this blog, we’ll go over one of our simpler use cases: automatically performing an account rename.
Streamlining workflows with straightforward solutions: The simple stupid
This crude Lucidchart (below) shows the basics of what we did. Every 5 minutes, a CloudWatch Event rule triggers a Lambda function. The function will make a JIRA API call to retrieve a list of tickets. Using those tickets, we will grab the necessary information and make subsequent API calls to backend services within LogicMonitor to perform specific actions, such as renames. Lambda will also actively update and close the tickets upon task completion. The first thing we need to do is know what tickets to look for.
Executing JQL queries directly from AWS Lambda
JIRA Query Language (JQL) is one of the most flexible ways to search for issues in JIRA. We use a JQL query with the JIRA REST API to find specific open tickets with issue types of “account rename.” This should return a list of associated tickets.
endpoint = "https://jira_url/rest/api"
jql_issuetype = "issuetype='Account Rename'"
jql_project = "project='TechOps Request'"
status = "status=Open"
jql = ("jql=" + jql_project +
"+AND+" + jql_issuetype +
"+AND+" + status
)
r = session.get(endpoint + "/2/search?" + jql % locals(), headers=headers_jira)
response = json.loads(r.text)
for issues in response["issues"]:
customer = issues["fields"]["customfield_10001"]
target_name = issues["fields"]["customfield_14673"]
Taking the list of open tickets, we need to be able to glean important information out of them, some of them in the form of custom fields.
Automating DevOps tasks with AWS Lambda and JIRA allows teams to focus on innovation rather than repetitive manual processes.
Customizing workflows with Jira’s custom fields
Users create custom fields, which are not by default available in JIRA. For our specific use case, we created a few fields, such as customer name, target name, and rename date. From the code example above, you can see that within the JIRA API, you can not specify just the field’s name; you’ll need to add a customfield_id.
Pro tip:
If you don’t want to look at a page of ugly JSON, you can also use the advanced JIRA search bar and type in the field’s name.
Embracing event-driven automation with AWS Lambda… most of the time
Usually, when we build apps on Lambda, we have components like Lambda functions and event sources. An event source is an AWS service that publishes events for processing by code within a Lambda function. In this case, performing a rename upon JIRA ticket creation could have been handled with a post function and an API Gateway. However, customers have their own maintenance windows and preferred times for an account rename to happen. Sometimes, customers may want their account renamed on Saturday at 4 a.m. during my personal maintenance (sleep) window. As a workaround, we decided to use a CloudWatch event as a lambda scheduler.
today = datetime.datetime.today() - datetime.timedelta(hours=7)
desired_date = datetime.datetime.strptime(issues["fields"]["customfield_16105"].replace("-0700",""), "%Y-%m-%dT%H:%M:%S.%f")
if today > desired_date:
create_rename(customer, target_name)
Our CloudWatch event would run every 5 minutes, triggering our Lambda function. The function will first check if the current time exceeds the value we parsed from the custom field rename date (see code above), and then we will allow the function to continue.
Combining tools to create seamless automation
At this point, we have collected the information we need. We can perform the rename by making API calls to the backend LogicMonitor services, but we won’t show that code in this blog. However, we also want to treat the JIRA ticket as a state file. We don’t want to keep grabbing the same open tickets repeatedly. This is where we want to use another JIRA API call to move the ticket to a different workflow step (e.g., from “Open” to “In Progress”). However, just like custom fields, we need a specific transition id, which you can find by editing your existing project workflow. We can now update the status of our JIRA ticket programmatically:
def changeStatus(key, id):
jira_request = {"transition":{"id": id }, "fields": {"resolution": {"name": "Done"}}}
endpoint = "https://jira_url.com/rest/api"
r = session.post(endpoint + "/2/issue/%(key)s/transitions?expand=transitions.fields" % locals(), data=json.dumps(jira_request), headers=headers_jira)
return r.text
Reducing human errors through intelligent automation: Saving people from people
Customer renames for the team used to be an extremely arduous task. Looking back at the Confluence revision history for our account rename runbook is akin to cleaning out your basement after 20 years. Besides being extremely time-consuming, the process involved halting puppets and, for unknown reasons, executing both a Ruby and a Bash script simultaneously. Sometimes, an application restart was required, but it was not always. As we grow, the only scalable solution is to automate repetitive, manual, and often mind-boggling tasks. It allows us to provide better service for customers and allows us to bypass the mundane to embrace the innovative.
One last tip—and this is the most important part—when we want to automate anything that requires manual input from other people, we have to take human stupidity… uh… error into consideration. Make sure to create validators and conditionals to combat this.
Plus, witty warning messages are a “value++.”
Subscribe to our blog
Get articles like this delivered straight to your inbox