Skip to main content

🛠️ Troubleshooting & Known Issues

A practical guide to diagnose and fix common issues with Ship Guard. Use this to quickly restore expected behavior during setup and operation.

📌 Quick Triage Checklist​

  • Is .github/ship-guard.yml present on the default branch?
  • Is the GitHub App installed on this repository with required scopes?
  • Did you open/update a Pull Request (not a direct push to default branch)?
  • Are branch protection rules configured if merges should be blocked?
  • Are your rule paths/excludePaths targeting the right files?
  • For AI rules, is settings.enableAIReviews enabled (if required)?

❗ Common Symptoms → Likely Causes → Fixes​

1) Review doesn’t run on PRs​

  • Likely causes:
    • App not installed on the repo or missing permissions
    • Missing/invalid .github/ship-guard.yml
    • PR opened from a fork with restricted permissions
    • Event not triggered (no new commits after config changes)
  • Diagnostics:
    • GitHub → Repo → Settings → Installed GitHub Apps → Confirm Ship Guard installed
    • Check PR → Checks tab → See if the check is present/queued
    • Validate YAML using an online linter
  • Quick fixes:
    • Install/enable the app for the repository
    • Commit a valid .github/ship-guard.yml to the default branch
    • Push a new commit to retrigger checks

2) Checks show as passed, but rules didn’t run​

  • Likely causes:
    • paths/excludePaths filtering everything out
    • Rules disabled via enabled: false
    • Severity all set to warning (and branch protection isn’t requiring checks)
  • Diagnostics:
    • Inspect .github/ship-guard.yml scoping
    • Temporarily remove paths filters to test
  • Quick fixes:
    • Adjust glob patterns (start simple: src/**)
    • Set severity to error for blocking rules and configure branch protection

3) AI findings never appear​

  • Likely causes:
    • settings.enableAIReviews: false
    • AI safety limits too strict (maxFilesPerReview, maxFileSizeKB, maxTokens)
    • Prompt too vague or out-of-scope
  • Diagnostics:
    • Check settings block and AI rule parameters
    • Narrow the paths for AI rules to the changed files
  • Quick fixes:
    • Enable AI reviews globally
    • Lower strict limits temporarily; verify behavior; then tighten again
    • Improve prompts; ask for structured JSON output

4) Too many comments/noisy output​

  • Likely causes:
    • Excessive rules or broad globs
    • maxFindings or maxFindingsInComment not set
  • Diagnostics:
    • Review rule count and scope
  • Quick fixes:
    • Set settings.maxFindings and settings.maxFindingsInComment
    • Consolidate findings into Check Run only by setting disableComments: true

5) False positives (deterministic rules)​

  • Likely causes:
    • Overbroad terms or missing excludePaths
  • Diagnostics:
    • Check which files triggered findings and why
  • Quick fixes:
    • Add excludePaths for test/docs
    • Set caseSensitive: false or useRegex: false appropriately
    • Downgrade severity to warning while tuning

6) RAG seems irrelevant (AI quotes wrong docs)​

  • Likely causes:
    • Low-quality or outdated docs; poor weighting
    • rag_sources includes noisy paths
  • Diagnostics:
    • Inspect RAG source list and weights
  • Quick fixes:
    • Prioritize ADRs/security docs with higher weights (0.8–1.0)
    • Remove broad patterns; prefer curated paths
    • Keep docs current and concise

7) Merges aren’t blocked despite errors​

  • Likely causes:
    • Branch protection is not configured
  • Diagnostics:
    • Repo → Settings → Branches → Branch protection rules
  • Quick fixes:
    • Enable “Require status checks to pass before merging” and select “Ship Guard”
    • Require branches to be up to date before merging (recommended)

🔎 Diagnostics & Logging​

  • GitHub PR → Checks tab View Ship Guard check details, including summaries and any error traces.

  • GitHub PR → Conversation tab Look for the Ship Guard bot comment with structured findings.

  • Configuration validation Use a YAML validator and compare against samples in configuring-rules.md and advanced-configuration.md.

  • Rule scoping test Temporarily restrict to a single file pattern known to change, e.g.:

    paths:
      - 'src/**/*.ts'
    excludePaths: []

🧩 Self-Check: Minimal Working Config​

version: 1
default_severity: error

rules:
  - name: 'No TODOs'
    condition: 'banned-terms'
    severity: error
    parameters:
      terms: ['TODO', 'FIXME']

  - name: 'AI: Basic error handling'
    condition: 'ai-review'
    severity: warning
    paths: ['src/**']
    parameters:
      prompt: |
        Check changed code for missing try/catch around async calls.
        Return JSON items: { file, line, severity, summary, fix }

settings:
  enableAIReviews: true
  maxFindings: 25
  maxFindingsInComment: 15

Commit this to the default branch and open a PR with a // TODO to confirm both deterministic and AI checks are active.

🧯 Quick Fix Recipes​

  • Silence noisy paths:

    excludePaths:
      - '**/*.md'
      - 'docs/**'
      - '**/*.test.*'

  • Reduce comment volume:

    settings:
      maxFindings: 20
      maxFindingsInComment: 10
      disableComments: false

  • Tighten AI safety:

    parameters:
      maxTokens: 500
      ragContext:
        maxDocs: 3
        similarityThreshold: 0.8
      security:
        allowSensitiveFiles: false
        maxFilesPerReview: 10
        enableContentFiltering: true

🆘 Need Help?​