Blacksmith MCP

An MCP server that connects Claude to your Blacksmith CI data. Query workflow runs, analyze test failures, detect flaky tests, and monitor usage—all through natural conversation.

# Why?

Debugging CI failures usually means clicking through dashboards, copying run IDs, and piecing together information across multiple pages. With this MCP, you can just ask:

• "Why did the last CI run fail?"
• "Which tests are flaky this week?"
• "Compare test failures between main and my PR"
• "What's using the most cache storage?"

Claude handles the API calls and gives you actionable insights.

# Quick Start

Zero-config if you're logged into Blacksmith in Chrome:

#  Add to Claude Code
claude mcp add blacksmith - - npx blacksmith-mcp

#  Set your org (run once)
export BLACKSMITH_ORG="your-org-name"

The MCP automatically extracts your session from Chrome cookies. No manual token copying needed.

# Installation

# # Option 1: Claude Code CLI

claude mcp add blacksmith - - npx blacksmith-mcp

# # Option 2: Project Configuration

Add to your .mcp.json:

{
  "mcpServers": {
    "blacksmith": {
      "type": "stdio",
      "command": "npx",
      "args": ["blacksmith-mcp"],
      "env": {
        "BLACKSMITH_ORG": "your-org-name"
      }
    }
  }
}

# # Option 3: Global Install

npm install -g blacksmith-mcp

# Configuration

# # Authentication

Automatic (recommended): Log into app.blacksmith.sh in Chrome. The MCP extracts your session cookie automatically.

Manual: Set BLACKSMITH_SESSION_COOKIE environment variable with your session cookie value.

# # Environment Variables

| Variable | Required | Description |
|- -- -- -- -- -|- -- -- -- -- -|- -- -- -- -- -- --|
| BLACKSMITH_ORG | Yes | Your Blacksmith organization name |
| BLACKSMITH_SESSION_COOKIE | No | Session cookie (auto-extracted from Chrome if not set) |

# Available Tools

# # Workflow Runs

| Tool | Description |
|- -- -- -|- -- -- -- -- -- --|
| list_runs | List workflow runs with filters (status, branch, workflow, actor, PR) |
| get_run | Get run details including all jobs |
| list_jobs | List jobs for a workflow run |
| get_job | Get job details (steps, timing, runner info) |
| get_job_logs | Get raw log output for a job |

# # Test Analytics

| Tool | Description |
|- -- -- -|- -- -- -- -- -- --|
| get_job_tests | Get all test results for a job |
| get_failed_tests | Get failed tests with full error messages |
| get_failures_by_pattern | Group failures by error pattern (e.g., "Cannot read properties") |
| compare_test_runs | Compare failures between two runs (find regressions) |
| get_flaky_tests | Detect tests that fail intermittently |
| get_slow_tests | Find tests exceeding duration threshold |
| get_test_history | Track a specific test's pass/fail history |
| get_trends | Analyze trends: duration, failure rate, test count |

# # Usage & Billing

| Tool | Description |
|- -- -- -|- -- -- -- -- -- --|
| get_current_usage | Current core usage (active vs max) |
| get_invoice_amount | Current billing period amount |
| get_usage_summary | Billable minutes vs free tier |
| get_cache_stats | Cache size, entries by repository |
| get_cache_entries | Detailed cache entries for a repo |

# # Organization

| Tool | Description |
|- -- -- -|- -- -- -- -- -- --|
| list_orgs | List accessible organizations |
| get_org_status | Org status (personal, onboarded, region) |
| search_logs | Search logs across all jobs |

# Example Conversations

# # Debug a Failed Run

You: Why did the last CI run fail?

Claude: Uses list_runs(status="failure", limit=1) then get_failed_tests

The last run failed with 34 test failures. The main issues are:

• Connection timeout (12 tests) - Tests in api/ are timing out connecting to the test database
• Undefined property (8 tests) - user.settings.theme is undefined in the new dark mode code
• Assertion failures (14 tests) - Snapshot mismatches in UI components

# # Find Flaky Tests

You: Which tests are flaky?

Claude: Uses get_flaky_tests(days=7, min_runs=5)

Found 3 flaky tests:
| Test | Flake Rate | Pattern |
|- -- -- -|- -- -- -- -- -- -|- -- -- -- --|
| should handle concurrent requests | 40% (4/10 runs) | Fails on CI, passes locally |
| renders loading state | 20% (2/10 runs) | Race condition with setTimeout |
| validates auth token | 30% (3/10 runs) | Intermittent network timeout |

# # Compare Branches

You: Did my PR introduce new test failures?

Claude: Uses compare_test_runs with PR run vs base branch

Your PR introduced 2 new failures:

• UserProfile.test.tsx: "Cannot read property 'avatar' of undefined"
• api/users.test.ts: Expected 200, got 404

5 previously failing tests now pass (nice!).

# Development

#  Install dependencies
pnpm install

#  Build
pnpm build

#  Development mode (watch)
pnpm dev

#  Test with MCP Inspector
npx @modelcontextprotocol/inspector node dist/index.js

# Troubleshooting

# # Session Expired

If you see SESSION_EXPIRED, your Blacksmith session has expired. Simply log back into app.blacksmith.sh in Chrome and retry.

# # Cookie Extraction Failed

The automatic cookie extraction requires:

• macOS with Chrome installed
• Being logged into Blacksmith in Chrome
• Chrome not running with a locked profile

If it fails, set BLACKSMITH_SESSION_COOKIE manually.

# # No Organization Set

Run list_orgs to see available organizations, then set BLACKSMITH_ORG to your org name.

# API Notes

This MCP uses Blacksmith's internal web API, which is undocumented. The API was reverse-engineered from the Blacksmith web app and may change without notice.

# License

MIT

# Contributing

Contributions welcome! Please open an issue first to discuss proposed changes.