Skip to main content

Getting Started with Ship Guard

Welcome to Ship Guard! This guide will get you from zero to your first automated code review in under 5 minutes and show you how to set up our game-changing Incident Memory feature that prevents your team from shipping the same bugs twice.

What You'll Learn​

By the end of this guide, you'll have:

  • ✅ Installed Ship Guard on your repositories
  • ✅ Created your first configuration file
  • ✅ Triggered your first automated review (deterministic + AI)
  • ✅ (Pro & Scale tier only) Set up Incident Memory to prevent repeat bugs
  • ✅ Understood how to train Ship Guard on your team's docs (RAG)
  • ✅ Know where to go next for advanced features

Estimated time: 10-15 minutes

Prerequisites​

Before you begin, make sure you have:

  • A GitHub account with admin access to at least one repository
  • A repository where you want to enable Ship Guard (public or private)
  • Basic familiarity with Git and pull requests
  • (Optional) Your team's internal documentation (ADRs, style guides, etc.) for RAG setup

🛠️ Part 1: Installation (3 Steps)​

Step 1: Install the GitHub App​

  1. Visit Our Website or go GitHub Marketplace:

  2. Choose Your Plan:

    PlanPriceBest ForKey Features
    Hobby$15/monthSolo devs, side projects10 repos, 500 AI reviews/month, 90-day analytics, 2 rule limit
    Pro$29/monthSmall teamsUnlimited repos, 1,500 AI reviews/month, Incident Memory, Slack/email alerts, 4 rule limit
    Scale$45/monthGrowing teams5,000 AI reviews/month, 6 rule limit, Incident Memory and Incident memory analytics and dashboard

    💡 Hobby plans include a 14-day free trial (no credit card required)

    💡 A first time one month discount for hobby and pro plan

  3. Select Repositories:

    • Choose "Only select repositories"

  4. Authorize the App:

    • Review the permissions (see below)
    • Click "Install & Authorize"

Step 2: Understand Permissions​

Ship Guard needs the following GitHub permissions to operate:

PermissionAccess LevelWhy We Need It
ChecksRead & writeCreate status checks that can block merges
Pull RequestsRead & writeRead PR diffs and post review comments
ContentsRead-onlyAccess your .github/ship-guard.yml config file
MetadataRead-onlyRead basic repository info (name, owner, visibility)
EmailRead-onlyTo get user email address

🔒 Security Guarantee: We never store your source code. We only analyze code diffs in real-time during PR reviews. See our Security & Permissions page for details.

Step 3: Access Your Dashboard​

After installation, you'll automatically have access to the Ship Guard web dashboard:

  1. Go to https://shipguard.dev
  2. Click "Sign in with GitHub"
  3. You'll see your installed repositories and can manage:
    • Subscription/billing
    • Analytics and insights
    • Incident Memory (Pro & Scale tier)
    • Usage quotas

💡 Bookmark the dashboard - You'll use it to track review metrics and manage incidents.

⚙️ Part 2: Your First Configuration​

Ship Guard is configured via a single YAML file: .github/ship-guard.yml

This file lives in your repository and is version-controlled alongside your code.

Creating Your Configuration File​

  1. In your repository, create the .github directory (if it doesn't exist)
  2. Create a file named ship-guard.yml inside .github/
  3. Copy and paste the starter configuration below:
# .github/ship-guard.yml
version: 1

# Default severity for rules that don't explicitly specify one
# Options: 'error' (blocks merge) | 'warning' (advisory only) | 'info' (informational)
default_severity: warning

# (Optional) RAG Configuration - Train Ship Guard on your team's docs
# Uncomment and configure if you want AI to learn from your internal docs
# rag_sources:
#   - path: "docs/adr/**/*.md"      # Architecture Decision Records
#     weight: 0.9                    # High importance (0.0 - 1.0)
#   - path: "docs/style-guide.md"   # Your team's style guide
#     weight: 0.7
#   - path: "README.md"             # Project overview
#     weight: 0.3

# Array of rules to enforce on every pull request
rules:
  # ==========================================
  # DETERMINISTIC RULES (Fast, No AI Costs)
  # ==========================================
  
  - name: "Block TODO comments in production code"
    condition: banned-terms
    enabled: true # Optional
    severity: error  # This will block merges
    tags: ['code-quality', 'deterministic']
    terms:
        - 'TODO'
        - 'FIXME'
        - 'HACK'
        - 'XXX'
      caseSensitive: false # Optional
      filePatterns:
        - '**/*.js'
        - '**/*.ts'
        - '**/*.jsx'
        - '**/*.tsx'
        - '**/*.py'
      excludePaths: # Optional
        - 'test/**'
        - 'tests/**'
        - '*.test.ts'
        - '*.spec.ts'

  - name: "Require changelog entry for significant changes"
    condition: require-changelog
    enabled: true
    severity: warning  # Advisory, won't block merge
    tags: ['documentation']
    parameters:
      changelogPath: 'CHANGELOG.md'
      excludePaths:
        - 'docs/**'
        - '*.md'
        - 'test/**'

  # ==========================================
  # AI-POWERED RULES (Smart, Context-Aware)
  # ==========================================
  
  - name: "AI: Detect security vulnerabilities"
    condition: ai-review
    enabled: true
    severity: error
    tags: ['security', 'ai-review']
    prompt: |
      Analyze the code changes for security vulnerabilities:
      - SQL injection risks (unsanitized user input in queries)
      - Hardcoded secrets, API keys, or credentials
      - Insecure cryptographic functions
      - Missing input validation or sanitization
      - Path traversal vulnerabilities
      
      If you find any security issues, report them with:
      - The exact line number and file
      - Why it's dangerous
      - A concrete fix recommendation

  - name: "AI: Check code complexity"
    condition: ai-review
    enabled: true
    severity: warning
    tags: ['maintainability', 'ai-review']
    prompt: |
      Identify functions with high cyclomatic complexity:
      - Deeply nested conditionals (>3 levels)
      - Functions with >50 lines of code
      - Repeated logic that could be extracted
      
      For each complex function found:
      - Suggest refactoring into smaller functions
      - Recommend using early returns to reduce nesting
      - Propose extracting helper utilities

  # ==========================================
  # POLICY QUESTIONS (AI Yes/No Checks)
  # ==========================================
  
  - name: "Policy: Check for non-approved database usage"
    condition: policy-question
    enabled: true
    severity: error
    tags: ['policy', 'infrastructure']
    question: |
      Does this PR introduce or modify code that connects to a database 
      other than PostgreSQL (e.g., MongoDB, DynamoDB, Firebase)?
      
      According to our ADR-001, PostgreSQL is the standard for production 
      data stores. If this change uses a different database, flag it and 
      ask the author to provide an ADR exception justification.

What This Configuration Does​

Rule TypeRule NameSeverityPurposeCost
DeterministicBlock TODO commentserrorPrevents TODO/FIXME from being mergedFree
DeterministicRequire changelogwarningReminds devs to update CHANGELOG.mdFree
AI ReviewSecurity vulnerabilitieserrorCatches SQL injection, hardcoded secrets, etc.~0.001 AI reviews
AI ReviewCode complexitywarningSuggests refactoring overly complex functions~0.001 AI reviews
Policy QuestionDatabase policy checkerrorEnforces PostgreSQL-only policy per ADR~0.001 AI reviews

💡 Start Simple: Your rule limit is based on the plan you're on.

🧪 Part 3: Triggering Your First Review​

Let's test your setup by creating a PR that triggers multiple rules.

Step 1: Create a Test Branch​

git checkout -b test-ship-guard

Step 2: Add Code That Violates Rules​

Create a new file that will trigger both deterministic and AI rules:

File: src/user-service.js

// src/user-service.js

// TODO: Add proper error handling here
// FIXME: This is a temporary implementation

const API_KEY = "sk_..live_abc123"; // Hardcoded secret - Ship Guard will catch this!

function getUserData(userId) {
  // SQL injection vulnerability - Ship Guard will catch this!
  const query = `SELECT * FROM users WHERE id = '${userId}'`;
  
  // Overly complex nested logic - Ship Guard will suggest refactoring
  if (userId) {
    if (query) {
      if (API_KEY) {
        if (typeof userId === 'string') {
          if (userId.length > 0) {
            return executeQuery(query);
          } else {
            return null;
          }
        } else {
          return null;
        }
      } else {
        return null;
      }
    } else {
      return null;
    }
  } else {
    return null;
  }
}

export { getUserData };

Step 3: Commit and Push​

git add .
git commit -m "feat: add user service (test Ship Guard)"
git push origin test-ship-guard

Step 4: Open a Pull Request​

  1. Go to your repository on GitHub
  2. Click "Compare & pull request"
  3. Title: Test Ship Guard automated review
  4. Description: Testing deterministic + AI rules
  5. Click "Create pull request"

Ship Guard will activate within 5-10 seconds. ⚡

📊 Part 4: Understanding the Results​

Within seconds, you'll see Ship Guard's comprehensive analysis.

Check Run Status​

In your PR's "Checks" section, you'll see:

┌────────────────────────────────────────────────────┐
│ ▼ Ship Guard                               ❌ Failed │
├────────────────────────────────────────────────────┤
│ ❌ 3 Errors Found (Merge Blocked)                  │
│   • Block TODO comments in production code         │
│   • AI: Detect security vulnerabilities           │
│   • Policy: Database policy check                  │
│                                                    │
│ ⚠️ 2 Warnings                                      │
│   • Require changelog entry                        │
│   • AI: Check code complexity                      │
└────────────────────────────────────────────────────┘

Detailed Bot Comment​

Ship Guard will post a comprehensive comment:

## 🤖 Ship Guard Analysis Complete

**Status:** ❌ **3 Errors** • ⚠️ **2 Warnings**

---

### ❌ Errors (Must Fix to Merge)

#### 1. Block TODO comments in production code
**Severity:** Error  
**Tags:** `code-quality`, `deterministic`

- **Line 3** in `src/user-service.js`: Found banned term "TODO"
- **Line 4** in `src/user-service.js`: Found banned term "FIXME"

**Action Required:** Remove or resolve these TODO/FIXME comments before merging.

---

#### 2. AI: Detect security vulnerabilities
**Severity:** Error  
**Tags:** `security`, `ai-review`

**Finding 1: Hardcoded Secret**
- **Line 6** in `src/user-service.js`: API key is hardcoded in source code
- **Risk:** If this repository is ever made public or an attacker gains access, your API key will be compromised
- **Fix:** Move `API_KEY` to environment variables and access via `process.env.API_KEY`

**Finding 2: SQL Injection Vulnerability**
- **Line 9** in `src/user-service.js`: User input (`userId`) is directly interpolated into SQL query
- **Risk:** An attacker could pass `userId = "1' OR '1'='1"` to dump your entire users table
- **Fix:** Use parameterized queries or an ORM:
  ```javascript
  const query = 'SELECT * FROM users WHERE id = ?';
  return executeQuery(query, [userId]);

3. Policy: Database policy check​

Severity: Error
Tags: policy, infrastructure

This PR appears to introduce direct SQL queries, but according to ADR-001, all database access should go through our PostgreSQL ORM wrapper. Please refactor to use db.query() or provide an ADR exception.

⚠️ Warnings (Advisory)​

1. Require changelog entry​

Severity: Warning
Tags: documentation

No changes detected in CHANGELOG.md. Consider documenting this change for future reference.

2. AI: Check code complexity​

Severity: Warning
Tags: maintainability, ai-review

Function: getUserData (Lines 8-30)

  • Issue: This function has 5 levels of nested conditionals, making it hard to read and test
  • Recommendation: Refactor using early returns:
function getUserData(userId) {
  if (!userId || typeof userId !== 'string' || userId.length === 0) {
    return null;
  }
  
  if (!API_KEY || !query) {
    return null;
  }
  
  return executeQuery(query);
}

💡 Need help? Check our troubleshooting guide or contact support.

Severity Levels Explained​

SeverityIconBehaviorWhen to Use
Error❌Blocks merge when Branch Protection is enabledSecurity issues, policy violations, critical bugs
Warning⚠️Shows as advisory, doesn't block mergeBest practices, code quality suggestions
Infoℹ️Informational onlyTips, documentation reminders

🛡️ To enforce blocking: You must enable Branch Protection on your repository.

Part 5: Training Ship Guard on Your Docs (RAG Setup)​

One of Ship Guard's most powerful features is RAG (Retrieval-Augmented Generation) - the ability to learn from your team's internal documentation.

What is RAG?​

RAG allows Ship Guard's AI to reference your:

  • Architecture Decision Records (ADRs)
  • Internal style guides
  • API documentation
  • Security policies
  • Past post-mortems

This makes AI reviews context-aware and team-specific, not generic.

How to Configure RAG​

  1. Identify Your Docs:

    • Find markdown files containing your team's standards
    • Common locations: docs/, adr/, wiki/, README.md

  2. Add to Your Config:

# .github/ship-guard.yml
version: 1

rag_sources:
  # Architecture Decision Records (highest priority)
  - path: "docs/adr/**/*.md"
    weight: 0.9  # Very important (0.9 = 90% relevance weight)
  
  # Team style guide
  - path: "docs/style-guide.md"
    weight: 0.8
  
  # API documentation
  - path: "docs/api/**/*.md"
    weight: 0.6
  
  # General project README
  - path: "README.md"
    weight: 0.3  # Lower weight (general context only)

rules:
  # Your AI rules will now reference these docs automatically!
  - name: "AI: Enforce team architecture patterns"
    condition: ai-review
    severity: error
    prompt: |
      Review this code against our Architecture Decision Records (ADRs).
      Check if the code violates any architectural patterns documented in our ADRs.
      If so, cite the specific ADR number and explain the violation.
  1. Weight System Explained:
WeightMeaningExample Use Case
0.9-1.0Critical documentationADRs, security policies
0.7-0.8Important standardsStyle guides, API contracts
0.4-0.6Helpful contextGeneral documentation
0.1-0.3Background infoREADMEs, onboarding docs

Example: ADR-Enforcing Rule​

Let's say you have an ADR that mandates PostgreSQL only:

File: docs/adr/001-use-postgresql.md

# ADR-001: Use PostgreSQL for All Production Databases

## Status
Accepted

## Context
We need a consistent database strategy.

## Decision
All production services MUST use PostgreSQL. NoSQL databases (MongoDB, DynamoDB) are prohibited without explicit CTO approval.

## Consequences
- Developers cannot introduce MongoDB without an ADR exception
- All new services use PostgreSQL by default

Ship Guard will automatically enforce this:

rag_sources:
  - path: "docs/adr/**/*.md"
    weight: 0.9

rules:
  - name: "AI: Enforce ADR database policy"
    condition: ai-review
    severity: error
    prompt: |
      Check if this PR introduces any database connections.
      Reference our ADR documentation to determine if the database choice is allowed.
      If MongoDB or another NoSQL database is used, flag it as a violation of ADR-001.

Result: Ship Guard will automatically cite ADR-001 when reviewing PRs!

Part 6: Incident Memory Setup (Pro & Scale Tier Only)​

What is Incident Memory?​

Incident Memory allows Ship Guard to remember past production incidents and actively prevent them from happening again.

Example scenario:

  1. March 2024: Your team ships a SQL injection bug (incident INC-042)
  2. You document the incident in Ship Guard
  3. July 2024: A developer opens a PR with similar code
  4. Ship Guard instantly flags it: "⚠️ Similar code caused incident INC-042. See post-mortem."
  5. Bug prevented. Crisis averted.

How to Set Up Incident Memory​

Step 1: Upgrade to PRO or Scale Tier ($29 or 45/dev/month)​

Incident Memory is only available on the Pro & Scale tier. 💡 Scale tier users have access to incident analytics and dashboard also.

To upgrade:

  1. Go to https://shipguard.dev/settings/billing
  2. Click "Upgrade or Manage Plan"
  3. Confirm the upgrade

Step 2: Document Your First Incident​

  1. Go to https://shipguard.dev/incidents
  2. Click "+ Add Incident"
  3. Fill out the form:
Repository: myorg/backend-api (select from the drop down menu)

Title: INC-042: SQL Injection in search endpoint

Summary:
On March 15, 2024, we discovered a SQL injection vulnerability in the 
user search endpoint. The root cause was unsanitized user input being 
directly interpolated into a SQL query string. An attacker could have 
extracted our entire users table by passing a malicious search query.

The fix was to replace string interpolation with parameterized queries 
using prepared statements.

Severity: High

Tags: security, database, sql-injection, user-input

Related URL: https://github.com/myorg/backend-api/issues/456
  1. Click "Add to Memory"

Example PR comment:

## ⚠️ High Alert: Similar to Past Incident

This PR touches user input validation code, which is similar to code that 
caused **incident INC-042** in March 2024.

**Past Incident Summary:**
SQL injection vulnerability in search endpoint. User input was not sanitized, 
allowing potential data exfiltration.

**Recommendation:**
Ensure all user inputs are sanitized using parameterized queries or an ORM.

[View full post-mortem: INC-042](https://github.com/myorg/backend-api/issues/456)

Step 4: Build Your Incident Library​

Document your top 5-10 past incidents:

  • Production outages
  • Security vulnerabilities
  • Data loss events
  • Performance degradations
  • Critical bugs

The more incidents you document, the smarter Ship Guard becomes.

Incident Memory Best Practices​

Do ✅Don't ❌
Be specific about the root causeWrite vague summaries
Include code examples when possibleOmit technical details
Tag incidents properly (security, performance, etc.)Use generic tags
Link to post-mortems or GitHub issuesSkip the reference links
Update incidents if you learn more laterLeave them stale

Part 7: Enforce Rules with Branch Protection​

By default, Ship Guard reports errors but doesn't block merges. To make it a true enforcer, you must enable GitHub Branch Protection.

Quick Setup (30 seconds)​

  1. Go to your repository on GitHub
  2. Settings → Branches → "Add rule"
  3. Branch name pattern: main (or master)
  4. Check "Require status checks to pass before merging"
  5. Search for and select: "Ship Guard"
  6. Check "Require branches to be up to date before merging" (optional but recommended)
  7. Click "Create"

Result: PRs with Ship Guard errors will now be physically blocked from merging.

📖 For detailed instructions, see Branch Protection Setup

🎯 What's Next?​

Congratulations! You've successfully set up Ship Guard with:

  • ✅ Deterministic rules (fast, free)
  • ✅ AI-powered reviews (smart, context-aware)
  • ✅ RAG training (learns from your docs)
  • ✅ Incident Memory (prevents repeat bugs) [Pro & Scale tier]
  • ✅ Branch protection (enforces blocking)

Level Up Your Setup​

Next StepDescriptionLink
⚙️ Advanced ConfigurationLearn about all available rules, conditions, and parametersconfiguring-rules.md
🤖 AI Review Deep DiveMaster AI prompts, policy questions, and RAG tuningai-reviews.md
🔐 Security Best PracticesUnderstand our data model, encryption, and compliancesecurity-and-permissions.md
🛠️ TroubleshootingCommon issues, debugging tips, and support channelstroubleshooting.md
🛠️ Incident MemorySetting up incident and tracking themincident-memory.md

Pro Tips for Success​

  1. Start with 1 rules - Don't overwhelm your team on day one
  2. Use warnings first - Set new rules to warning severity initially to observe behavior
  3. Get team buy-in - Share the config in a team meeting and gather feedback
  4. Monitor false positives - Check the dashboard weekly to refine rule prompts
  5. Document your incidents - The more incidents you add, the smarter Ship Guard becomes

📚 Quick Reference​

Config File Location​

your-repo/
└── .github/
    └── ship-guard.yml  ← This file controls everything

Common Rule Conditions​

ConditionTypePurposeAI Cost
banned-termsDeterministicBlock specific words/phrasesFree
require-changelogDeterministicEnforce changelog updatesFree
ai-reviewAI-poweredDeep code analysis with custom prompts~0.001 reviews
policy-questionAI-poweredYes/no policy enforcement~0.001 reviews

Severity Levels​

  • error - Blocks merge (requires Branch Protection)
  • warning - Advisory only
  • info - Informational

🆘 Need Help?​