AI Classifier for issue priorities

AI Classifier for issue priorities

Goal

Next, we will develop a practical example: an issue priority classifier for newly created issues: JavaScript Runner will listen for issue.create events and trigger a LangChain script to prioritize the issue based on its description, using natural language processing to analyze context and urgency.

Coding (JS Script)

  • Copy this script below in the JS Console

  • Provide your OpenAI API Key,

  • Save it with the “issue_priority_classifier” name:

image-20250528-220406.png
import { OpenAI } from "@langchain/openai"; const issueId = globalThis.__issueid; //The newly created issue id const JIRA_BASE_URL = jira.getBaseUrl(); const OPENAI_API_KEY = ""; // <- Provide yours const httpHeader = { Accept: "application/json", "Content-Type": "application/json", }; async function getIssue(issueId) { const res = await fetch( `${JIRA_BASE_URL}/rest/api/2/issue/${issueId}?fields=summary,description,priority`, { headers: httpHeader } ); if (!res.ok) throw new Error(`Failed to fetch issue: ${res.status} ${res.statusText}`); const { fields } = await res.json(); return fields; } async function getPriorities() { const res = await fetch( `${JIRA_BASE_URL}/rest/api/2/priority`, { headers: httpHeader } ); if (!res.ok) throw new Error(`Failed to fetch priorities: ${res.statusText}`); return await res.json(); } async function updatePriority(issueId, priorityId) { const res = await fetch( `${JIRA_BASE_URL}/rest/api/2/issue/${issueId}`, { method: "PUT", headers: httpHeader, body: JSON.stringify({ fields: { priority: { id: priorityId } } }), } ); if (!res.ok) throw new Error(`Failed to update priority: ${res.statusText}`); } async function addComment(issueId, commentBody) { const res = await fetch( `${JIRA_BASE_URL}/rest/api/2/issue/${issueId}/comment`, { method: "POST", headers: httpHeader, body: JSON.stringify({ body: commentBody }), } ); if (!res.ok) throw new Error(`Failed to add comment: ${res.statusText}`); } async function classifyPriority(summary, description) { const llm = new OpenAI({ openAIApiKey: OPENAI_API_KEY }); const prompt = `You are an assistant that assigns a Jira ticket priority. Available levels: Blocker, Highest, High, Medium, Low, Lowest, Minor. Summary: ${summary} Description: ${description} Respond with JSON: { "new_priority": "<level>", "reasons": "<brief explanation>" }`; const response = await llm.invoke(prompt); const text = response.trim(); try { return JSON.parse(text); } catch (err) { throw new Error(`Unable to parse OpenAI response: ${text}`); } } try { const { summary, description, priority } = await getIssue(issueId); const { new_priority, reasons } = await classifyPriority(summary, description); if (new_priority !== priority.name) { const priorities = await getPriorities(); const target = priorities.find(p => p.name.toLowerCase() === new_priority.toLowerCase()); if (!target) throw new Error(`Priority '${new_priority}' not found in Jira`); await updatePriority(issueId, target.id); const comment = `Priority changed from **${priority.name}** to **${new_priority}**. **Reasons:** ${reasons}`; await addComment(issueId, comment); } else { //TThe current priority already matches the suggestion. No changes made. } } catch (err) { jira.log.warn("Failed issue priorization: " + err); }


Configuring (JS Listener)


Navigate to:

Administration → Manage apps → JavaScript Runner → JS Listener

image-20250528-221332.png

Click Create Listener and fill out the pop up dialog:

  • Name: issue_created_classifier

  • Event: issue.created

image-20250528-220924.png

Click Accept to save the new JS Listener.

image-20250528-221200.png

Testing

Test it! Create a new issue with this description:

I've seen this error in the system log. It does not seems to much relevant as the system is working perfectly. 2025-05-29T12:47:05.842Z [ERROR] [token-service:production] trace_id=9f1c2a7d3b4e5f6081a2bc3d4e5f6a7b span_id=7a6b5c4d3e2f1a0b host=auth01.prod.internal user_id=98765 client_ip=203.0.113.42 → JWT signature verification fallback triggered (expected=RS256, got=HS256, kid=main-rsa-key) Insecure JWT validation: signature verified with HMAC instead of RSA Note: public key used as HMAC secret – potential token forgery


And set Low priority:

image-20250528-223753.png

Open the new issue and check the priority and comments section:

image-20250528-224019.png


The LangChain script has been triggered and it changed the priority from Low to High.
Furthermore a new comment explaining the reason of the change has been added to the issue!

Improving

Configuration Management

To improve maintainability:

  • Move the OpenAI API key to a dedicated configuration module in Jira.

  • Import the API key into the LangChain script, ensuring that updates to the key require changes in only one location.

  • Encapsulate the OpenAI model name in the same module to handle potential model deprecation by OpenAI, minimizing script modifications.

Jira Comment Attribution

To avoid confusion in production:

  • The current setup attributes comments to the issue reporter (current user), which may mislead them into thinking they created the comment.

  • Create a dedicated Jira “bot” user for automated tasks.

  • Use the bot’s credentials in the create-comment function to ensure that comments and priority changes are attributed to the bot, enhancing transparency for the reporter.