Bulk deletion of attachments using API
There are many use cases that can pop up at any time when being a Jira administrator. Therefore, it is good you acquaint yourself with a better understanding of how you can make the best use of Atlassian API. If you prefer working with scripts then this article is just right for you. This article assumes that you have a basic understanding of making an API call or following instructions to achieve the desired outcome outlined below.
Disclaimer: Please note that deleting attachments is a destructive action that is not recoverable, so take proper care before attempting to use the below method.
This article is going to focus on two use cases that apply the use of the REST API to administer Jira projects in terms of deleting unwanted attachments within issues.
-
Storage space cleanup
-
GDPR compliance
If you have been administrating Jira (Jira Software, Jira Service Management(formerly Jira Service Desk) or Jira Work Management(formerly Jira Core)) for a while you might have come across a time where you’ve stored so much data and storage allocation is almost full and you want to get rid of those old files that are just taking too much space within your cloud instance. This use case employs the use of REST API to perform these tasks. Please note that with automation for Jira, you can perform bulk deletion of attachments by filenames with the limit of 1K issues. However, what you can use this API method factors in additional functions.
-
Delete by users
-
Delete by file extension
-
Delete by file size
-
Delete by date range
-
No issue query limit
All the above is possible with the REST API endpoint provided by Atlassian and by some little scripting, you can get your desired outcome using the API. Also, note that the above method can be used by those administrators who are trying to reduce their attachment size on their instance as well as those administrators who want to automate or even dynamically perform attachment deletion from certain projects at a certain interval. If you would like to automate this process, this API can help you do that just continue reading to find out more.
The other use case surrounds GDPR compliance and how organizations would require to delete personal information from their project. Take, for example, you are an IT Service Desk manager and your organization has a regulation that each customer ticket that is created and has an attachment in it should be deleted within 60 - 90 days. If your project has been running for the past 2 or more years and you’ve got over 40K to 50K tickets. How do you go about finding every attachment and subsequently deleting each one or how do you separate deletion of attachments by certain customers who want their data deleted from your ticket?
Oh! you’re still waiting for an answer. Well, the simple fact is that it is possible to do all that. I will show you how you can achieve all that within your Jira projects and be GDPR compliant in terms of removing user-generated content that contains personal or identifiable details. I’ll also show you the basic usage of this API and how you can even integrate it within your local system (macOS or Linux) to perform automatic attachment deletion on a regular interval using cron jobs.
There are two methods when it comes to deleting attachments using this API. I will simplify the process here to make it as straightforward as possible. It is either you use the file method or the search method. For this use case, we’ll be using python as our scripting language and a library called jiraone which can be downloaded using pip install jiraone or python3 -m pip install jiraone
You should store your credentials in a JSON file as shown below e.g. config.json
{
"user": "prince.nyeche@example.com",
"password": "API-Token-here",
"url": "https://yourinstance.atlassian.net"
}
Using Search Method
If the file argument is provided, the search argument is ignored. If the file argument is not provided the search argument has to be provided else an error will be returned. e.g.
from jiraone import LOGIN, delete_attachments
import json
config = json.load(open('config.json'))
LOGIN(**config)
ext = [".png", ".pdf"]
users = ["5abcXXX", "617bcvXXX"]
size = ">30MB"
jql = {"jql": "project in (COM) ORDER BY Rank DESC"}
delete_attachments(search=jql, extension=ext, by_user=users, by_size=size)
Let me explain to you what each argument does and how you can use it. The above example is one of the ways you can make a request. You can make a request using the below search criteria
# previous expression
key = "COM-120" # a string as issue key
key = "COM-120,TP-15" # a string separated by comma
key = ["COM-120", "IP-18", 10034] # a list of issue keys or issue id
key = {"jql": "project in (COM) ORDER BY Rank DESC"} # a dict with a valid jql
The above will enable you to search for viable issues that has an attachment value. The extension argument can be used as below
# previous expression
ext = ".png" # a string
ext = ".png,.pdf" a string separated by comma
ext = [".png", ".zip", ".csv"] # a list of extensions
You can also use it without the “dot” prefix on the extension but make sure that if the dot is not being used for multiple extensions either by string or list, the dot prefix is not maintained at all. E.g
Valid
# previous expression
ext = [".png", ".zip", ".csv"] # a list of extensions
Valid
# previous expression
ext = ["png", "zip", "csv"] # a list of extensions
ext = "png,zip,pdf" # a string separated by comma
Invalid
# previous expression
ext = [".png", "zip", ".csv"] # a list of extension
In the case of the invalid example, notice that one of the extensions doesn’t have a “dot” prefix! When such happens the file extension is skipped and won’t be deleted in the final execution of the API.
The by_user argument allows you to use accountId to filter the deletion by such users. This argument expects a list of users
# previous expression
users = ["5abcXXX", "617bcvXXX"]
When the user that matches is found, then the deletion will occur.
The by_size argument helps with deletion by byte size. You can specify the limit by using the below format. The acceptable format for by_size uses this mechanism
size = [condition][number][byte type]
-
Condition uses the greater than (>) or lesser than (<) symbols
-
Number could be any digit that you can come up with.
-
Byte type refers to the byte size allocation. Either in kb, mb, gb or blank representing sizes in bytes
# previous expression
size = ">12mb" # greater than 12mb in size
size = "<150mb" # lesser than 150mb in size
size = ">400kb" # greater than 400kb in size
size = "<20000" # lesser than 20000 bytes without the suffix byte type specified
Using the by_date argument within this function helps to determine if and when an attachment should be removed. It uses the initiator's current local time derived from your machine to determine the time and date; down to the last second. Then it compares, that current time to the issue time when the attachment was created and then determine a time delta of the difference. If it can determine that the time period or range is lesser than the DateTime the attachment existed, then it returns true otherwise returns false. You can make the request by performing any of the below tasks.
# previous expression
dates = "3 days ago"
dates = "3 months ago" # you can say 3 months
dates = "15 weeks" # the ago part can be left out and it won't matter.
The accepted format of using this argument is given below and only strings are accepted.
dates = "[number] <space> [time_info] <space> ago"
The ago part is optional (i.e not needed but looks visually pleasing) but the number and time_info part are crucial. These are the expected values for time_info
-
minute or minutes,hour or hours,day or days,week or weeks,month or months,year or years
Depending on the context and which one makes the most accurate depiction in the English language.
# previous expression
dates = "14 hours ago"
dates = "1 year ago"
Besides using the standard way to call this function, you can always mix and match your search criteria using these four arguments. extension, by_user, by_size, by_date
The hierarchy follows the same way as they are arranged above.
E.g. By extension and by size
# previous expression
delete_attachments(search=jql, extension=ext, by_size=size)
E.g. By size only
# previous expression
delete_attachments(search=jql, by_size=size)
E.g. By user and by size
# previous expression
delete_attachments(search=jql, by_user=users, by_size=size)
E.g. By user and by date
# previous expression
delete_attachments(search=jql, by_user=users, by_date=date)
E.g. By date and by size
# previous expression
delete_attachments(search=jql, by_date=date, by_size=size)
I think you get a general idea. This means these four arguments can be used interchangeably and they can be used singularly on their own.
Using File Method
Subsequently, if you do not want to run a search, you can perform an entire export of your filter query from your Jira UI by navigating to your Filter > Advanced issue search, typing your query to get the desired result and click the export CSV all fields.
You do not have to edit the file or change the format in any way. If you’ve exported it as an xlsx file or you’ve modified the file by removing other columns. Please add a delimiter argument and use “;” as the delimiter. Always ensure that the headers are present and not removed. Also, ensure that the “Attachment” and “Issue key” columns are always present in the file.
# previous login statement
ext = [".csv", ".mov", ".png"]
file = "Jira-export.csv"
delete_attachments(file=file, extension=ext)
Example with delimiter parameter.
# previous login statement with variable options
delete_attachments(file=file, extension=ext, delimiter=";")
# You can only filter byextensionwhen using the file method.
Turning on Safe mode
If you just want to test the function without actually deleting any attachments for both the file and search method, you can switch the argument delete into False and that will turn on safe mode. E.g.
# previous login statement with variable options
delete_attachments(file=file, extension=ext, delimiter=";", delete=False)
# result
# Safe mode on: Attachment will not be deleted "jira_workflow_vid.mp4" | Key: COM-701
The same argument is available when you use the search method.
# previous login statement with variable options
delete_attachments(search=jql, delete=False)
# result
# Safe mode on: Attachment will not be deleted "jira_workflow_vid.mp4" | Key: COM-701
When safe mode is on, all filtering is ignored.
Check Point
This API comes with a checkpoint, which means that if for any reason the script stops or is disconnected (i.e due to internet connection or termination of the script). Once you restart the script, it will resume back from whatever iteration it stopped. Even if you’re running a JQL search that has 50K+ issues or more and it stopped at 35K issue during the extraction. It will resume from the exact iteration it stopped and from the marginal record (i.e this is +1 or -1 variation from the relative point it stopped. In most instances it would likely be the exact record.) from which is stopped to complete the information extraction. As long as you provide the answer “yes” when asked to use “the save point data”. Then the script will resume from the last known iteration.
Making a cron script to auto-delete at Intervals
You can configure a simple shell script to run at certain intervals within your local device. The python script should be wrapped within a shell script to run based on your configurations. This is applicable to macOS or Linux users to set up your crontab. On macOS, you might want to go to your security and privacy settings and add the terminal app, and cron “Full disk access” in order to use the below.
Create a bash script. A simple one such as below would do. Assuming that you’ve already created the python script using a similar example above and placed it in a folder within your machine. You can give it any name e.g cronScript.sh ensure that you have permission right on this file. E.g. chmod u+x cronScript.sh
#!/usr/bin/env bash
usr/bin/python delete_attachment.py
Make sure the shell scripts work properly. Then open your terminal and type the below
crontab -e
Setting up cron requires that you familiarize yourself with the format. You can use crontab.guru to get accurate results and generate the expression that you need. Below is just an example.
The above command will open with vim, press i to go to INSERT mode and then enter your expression. e.g.
0,15,30,45 * * * * cd /home/name/Desktop/scripts && ./cronScript.sh
You can use the command pwd to check your present working directory name to the folder where your script is located, which you can use above.
The cronScript.sh is located in a folder called scripts and this folder should contain both your python and your credential files.
Once you’re done saving the expression, hit esc key and type :wq to save and exit the editor. You might want to ensure that you’re running your python executable on the file rather than the python alias. Find your python executable by running which python on your terminal. That way you can use the executable usr/bin/python directly in your shell script. Please note that you can only use python version greater than v3.6.x to operate this script.
To list your crontab configurations, type on your terminal crontab -l
The automation part is done. Now when the time reaches, your script will run and automatically delete the attachment based on your configuration. So if you have customer’s attachment data on tickets that need to be removed within 60 or 90 days after the tickets are closed, now you have a solution to doing such task automatically.
You can use this script to perform your bulk attachment deletions on Jira issues and you can filter the deletion anyhow you want or even integrate it with other systems to get your task done.
And that’s how you can automate the deletion of attachments and be GDPR compliant on your Jira project as an administrator.
Was this helpful?
Thanks!
Prince Nyeche
22 comments