<div id="readability-page-1" class="page"><div><img src="https://devstarsj.github.io/assets/https://images.unsplash.com/photo-1587620962725-abab19836100?w=1200&amp;q=80">AI code review tools have gone from novelty to ubiquity in two years. Every team has tried one. But the initial enthusiasm often fades when the tool starts flagging every PR with 20 comments about things developers don’t care about. Getting AI-assisted code review to actually improve quality — without drowning developers in noise — requires more thought than just installing a tool.<img src="https://images.unsplash.com/photo-1587620962725-abab19836100?w=900&amp;q=80" alt="Developer reviewing code on screen"> Photo by <a href="https://unsplash.com/@saksham">Saksham Gangwar</a> on Unsplash<h2 id="the-problem-with-naive-ai-code-review">The Problem With Naive AI Code Review</h2>Most teams that “tried AI code review and it didn’t work” made the same mistakes:<ol><li>Applied it to everything — every diff, every file, including auto-generated code and trivial changes</li><li>Didn’t configure severity thresholds — the tool treated a missing docstring the same as a SQL injection</li><li>No feedback loop — comments that developers consistently ignored stayed in the system, eroding trust</li><li>Replaced human review rather than augmenting it — AI caught different things than humans, but teams cut human review time without accounting for what they lost</li></ol>The teams getting value from AI code review treat it as a tool in a workflow, not a silver bullet.<h2 id="what-ai-review-does-well-and-doesnt">What AI Review Does Well (and Doesn’t)</h2>AI excels at:<ul><li>Security patterns: SQL injection, XSS, insecure deserialization, hardcoded secrets</li><li>Common bug patterns: off-by-one errors, null pointer risks, unclosed resources</li><li>Code consistency: naming conventions, error handling patterns, logging standards</li><li>Documentation coverage: missing docstrings, outdated comments</li><li>Dependency issues: known vulnerable packages, unused imports</li><li>Boilerplate suggestions: test coverage gaps, missing error cases</li></ul>AI still struggles with:<ul><li>Business logic correctness — it doesn’t know what your code is supposed to do</li><li>Architectural concerns — is this the right abstraction? Should this be a separate service?</li><li>Performance at scale — “this query is fine” until it’s called 10 million times</li><li>Context-dependent tradeoffs — the right answer often depends on your specific constraints</li><li>Review that requires domain knowledge — medical, financial, legal nuances</li></ul>The mental model: AI is a great first reviewer for the mechanical things. Humans review for intent, architecture, and domain correctness.<h2 id="building-an-effective-pipeline">Building an Effective Pipeline</h2><h3 id="level-1-pre-commit-fastest-lowest-friction">Level 1: Pre-Commit (Fastest, Lowest Friction)</h3>Block obvious issues before they hit CI:<div><pre><code># .pre-commit-config.yaml
repos:
 - repo: https://github.com/astral-sh/ruff-pre-commit
 rev: v0.5.0
 hooks:
 - id: ruff
 args: [--fix]
 - id: ruff-format

 - repo: https://github.com/gitleaks/gitleaks
 rev: v8.18.0
 hooks:
 - id: gitleaks # Block secrets from ever entering git history

 - repo: https://github.com/pre-commit/mirrors-mypy
 rev: v1.10.0
 hooks:
 - id: mypy
 additional_dependencies: [types-requests]
</code></pre></div>This catches secrets, formatting, and type errors locally — before CI even runs.<h3 id="level-2-ci-security-scanning">Level 2: CI Security Scanning</h3>Dedicated security tooling in CI catches what pre-commit misses:<div><pre><code># .github/workflows/security.yml
name: Security Scan

on: [pull_request]

jobs:
 security:
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4
 
 - name: Semgrep SAST
 uses: semgrep/semgrep-action@v1
 with:
 config: |
 p/owasp-top-ten
 p/secrets
 p/python
 severity: WARNING # Only block on WARNING+
 
 - name: Snyk Dependency Check
 uses: snyk/actions/python@master
 with:
 args: --severity-threshold=high # Only fail on HIGH+
 env:
 SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
 
 - name: Trivy Container Scan
 if: hashFiles('Dockerfile') != ''
 uses: aquasecurity/trivy-action@master
 with:
 image-ref: ${{ env.IMAGE_TAG }}
 severity: CRITICAL,HIGH
 exit-code: '1'
</code></pre></div>Key principle: fail the build only on HIGH/CRITICAL severity. Low and medium severity items create noise without preventing real issues.<h3 id="level-3-llm-review-on-pr">Level 3: LLM Review on PR</h3>This is where AI-powered code review shines. The current leaders:<ul><li>CodeRabbit — most configurable, good context window</li><li>GitHub Copilot Code Review — native integration, requires Copilot Enterprise</li><li>Greptile — indexes your entire codebase for context-aware reviews</li><li>Ellipsis — good at auto-generating changelogs and summaries</li></ul><img src="https://images.unsplash.com/photo-1522071820081-009f0129c71c?w=900&amp;q=80" alt="Code review collaboration"> Photo by <a href="https://unsplash.com/@anniespratt">Annie Spratt</a> on UnsplashCodeRabbit configuration that actually works:<div><pre><code># .coderabbit.yaml
language: en-US

reviews:
 high_level_summary: true
 poem: false # Turn this off for professional contexts
 
 path_filters:
 - "!**/*.lock"
 - "!**/migrations/**" # Skip auto-generated migrations
 - "!**/generated/**"
 - "!**/__snapshots__/**"
 
 path_instructions:
 - path: "src/payments/**"
 instructions: |
 This is payment-critical code. Extra scrutiny on:
 - Any floats used for money calculations (require Decimal)
 - Input validation on all amounts
 - Idempotency handling for charge operations
 - PCI-relevant logging (never log card data)
 
 - path: "src/api/**"
 instructions: |
 Focus on: authentication checks, input validation, 
 rate limiting, and proper HTTP status codes.
 Skip style comments.
 
 auto_review:
 enabled: true
 drafts: false # Don't review draft PRs
 
 # Collapse low-priority comments to reduce noise
 collapse_walkthrough: true
</code></pre></div><h3 id="building-a-feedback-loop">Building a Feedback Loop</h3>The most important — and most neglected — part of AI code review:<div><pre><code># Track which AI comments developers act on
# Weekly report to the team:

ai_comment_stats = {
 "total_comments": 247,
 "addressed_by_developer": 89, # 36% — too low
 "dismissed_with_reason": 43,
 "auto_resolved": 31,
 "ignored": 84, # These should trigger config review
}

# Comments ignored &gt;80% of the time → add to suppression list
# Comments addressed &gt;60% of the time → good signal
</code></pre></div>Review your suppression list quarterly. Patterns of ignored comments mean either:<ul><li>The rule is wrong for your codebase</li><li>The severity is miscategorized</li><li>Developers need education on why it matters</li></ul><h2 id="custom-rules-for-your-domain">Custom Rules for Your Domain</h2>Generic rules catch generic problems. The real value comes from encoding your team’s institutional knowledge:<div><pre><code># semgrep custom rules
rules:
 - id: no-raw-sql-in-services
 patterns:
 - pattern: db.execute("...")
 - pattern-not: db.execute($QUERY, ...) # Parameters are OK
 message: |
 Raw SQL strings without parameters. Use parameterized queries
 to prevent SQL injection. See: docs/database-patterns.md
 languages: [python]
 severity: ERROR
 
 - id: require-correlation-id-logging
 pattern: logger.$METHOD(...)
 pattern-not: logger.$METHOD(..., correlation_id=...)
 paths:
 include:
 - src/api/**
 message: |
 API handlers must include correlation_id in all log calls
 for distributed tracing. See: docs/logging-standards.md
 severity: WARNING
</code></pre></div>These custom rules carry your team’s knowledge forward. New engineers get the same guidance senior engineers give in code review — automatically.<h2 id="the-human-review-that-remains">The Human Review That Remains</h2>After AI handles the mechanical review, human reviewers should focus on:<ol><li>Intent verification — does this code do what the ticket describes?</li><li>Architectural decisions — is this the right approach? Any concerns about maintainability?</li><li>Domain correctness — does this business logic make sense?</li><li>Knowledge transfer — is there anything the author should know for future work?</li><li>Risk assessment — what could go wrong in production that tests don’t cover?</li></ol>Set this expectation explicitly with your team. AI handles the mechanical; humans handle the judgment.<h2 id="measuring-success">Measuring Success</h2>Metrics worth tracking monthly:<div><pre><code>Code Quality Health:
- Security vulnerabilities in production (target: trend down)
- Escaped defect rate (bugs found in production vs. in review)
- Time from PR open to first review (AI should reduce this)
- Reviewer time per PR (should drop for mechanical review)
- Developer satisfaction with review process (survey)
</code></pre></div>Don’t measure “AI comments accepted.” That optimizes for compliance, not quality.<h2 id="conclusion">Conclusion</h2>AI-assisted code review works when it’s configured thoughtfully, calibrated to reduce noise, and positioned as a complement to human review rather than a replacement. The teams winning with it have invested in custom rules for their domain, track and act on feedback loops, and kept human reviewers focused on what AI can’t do.Start with security scanning in CI, add an LLM reviewer configured for your codebase, measure what gets actioned vs. ignored, and iterate. The goal isn’t more comments — it’s better code with less review toil.The best code review is fast, focused, and catches what matters. AI can help you get there. <div>이 글이 도움이 되셨다면 공감 및 광고 클릭을 부탁드립니다 :) </div></div></div>

Dev Note

AI code review tools have gone from novelty to ubiquity in two years. Every team has tried one. But the initial enthusiasm often fades when the tool starts flagging every PR with 20 comments about things developers don’t care about. Getting AI-assisted code review to actually improve quality — without drowning developers in noise — requires more thought than just installing a tool.
Photo by Saksham Gangwar on Unsplash
The Problem With Naive AI Code Review
Most teams that “tried AI code review and it didn’t work” made the same mistakes:
Applied it to everything — every diff, every file, including auto-generated code and trivial changes
Didn’t configure severity thresholds — the tool treated a missing docstring the same as a SQL injection
No feedback loop — comments that developers consistently ignored stayed in the system, eroding trust
Replaced human review rather than augmenting it — AI caught different things than humans, but teams cut human review time without accounting for what they lost
The teams getting value from AI code review treat it as a tool in a workflow, not a silver bullet.
What AI Review Does Well (and Doesn’t)
AI excels at:
Security patterns: SQL injection, XSS, insecure deserialization, hardcoded secrets
Common bug patterns: off-by-one errors, null pointer risks, unclosed resources
Code consistency: naming conventions, error handling patterns, logging standards
Documentation coverage: missing docstrings, outdated comments
Dependency issues: known vulnerable packages, unused imports
Boilerplate suggestions: test coverage gaps, missing error cases
AI still struggles with:
Business logic correctness — it doesn’t know what your code is supposed to do
Architectural concerns — is this the right abstraction? Should this be a separate service?
Performance at scale — “this query is fine” until it’s called 10 million times
Context-dependent tradeoffs — the right answer often depends on your specific constraints
Review that requires domain knowledge — medical, financial, legal nuances
The mental model: AI is a great first reviewer for the mechanical things. Humans review for intent, architecture, and domain correctness.
Building an Effective Pipeline
Level 1: Pre-Commit (Fastest, Lowest Friction)
Block obvious issues before they hit CI:
# .pre-commit-config.yaml
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.5.0
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.18.0
    hooks:
      - id: gitleaks    # Block secrets from ever entering git history
  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.10.0
    hooks:
      - id: mypy
        additional_dependencies: [types-requests]
This catches secrets, formatting, and type errors locally — before CI even runs.
Level 2: CI Security Scanning
Dedicated security tooling in CI catches what pre-commit misses:
# .github/workflows/security.yml
name: Security Scan
on: [pull_request]
jobs:
  security:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Semgrep SAST
        uses: semgrep/semgrep-action@v1
        with:
          config: |
            p/owasp-top-ten
            p/secrets
            p/python
          severity: WARNING   # Only block on WARNING+
      
      - name: Snyk Dependency Check
        uses: snyk/actions/python@master
        with:
          args: --severity-threshold=high   # Only fail on HIGH+
        env:
          SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
      
      - name: Trivy Container Scan
        if: hashFiles('Dockerfile') != ''
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: ${{ env.IMAGE_TAG }}
          severity: CRITICAL,HIGH
          exit-code: '1'
Key principle: fail the build only on HIGH/CRITICAL severity. Low and medium severity items create noise without preventing real issues.
Level 3: LLM Review on PR
This is where AI-powered code review shines. The current leaders:
CodeRabbit — most configurable, good context window
GitHub Copilot Code Review — native integration, requires Copilot Enterprise
Greptile — indexes your entire codebase for context-aware reviews
Ellipsis — good at auto-generating changelogs and summaries
Photo by Annie Spratt on Unsplash
CodeRabbit configuration that actually works:
# .coderabbit.yaml
language: en-US
reviews:
  high_level_summary: true
  poem: false   # Turn this off for professional contexts
  
  path_filters:
    - "!**/*.lock"
    - "!**/migrations/**"      # Skip auto-generated migrations
    - "!**/generated/**"
    - "!**/__snapshots__/**"
  
  path_instructions:
    - path: "src/payments/**"
      instructions: |
        This is payment-critical code. Extra scrutiny on:
        - Any floats used for money calculations (require Decimal)
        - Input validation on all amounts
        - Idempotency handling for charge operations
        - PCI-relevant logging (never log card data)
    
    - path: "src/api/**"
      instructions: |
        Focus on: authentication checks, input validation, 
        rate limiting, and proper HTTP status codes.
        Skip style comments.
  
  auto_review:
    enabled: true
    drafts: false   # Don't review draft PRs
    
  # Collapse low-priority comments to reduce noise
  collapse_walkthrough: true
Building a Feedback Loop
The most important — and most neglected — part of AI code review:
# Track which AI comments developers act on
# Weekly report to the team:
ai_comment_stats = {
    "total_comments": 247,
    "addressed_by_developer": 89,   # 36% — too low
    "dismissed_with_reason": 43,
    "auto_resolved": 31,
    "ignored": 84,                   # These should trigger config review
}
# Comments ignored >80% of the time → add to suppression list
# Comments addressed >60% of the time → good signal
Review your suppression list quarterly. Patterns of ignored comments mean either:
The rule is wrong for your codebase
The severity is miscategorized
Developers need education on why it matters
Custom Rules for Your Domain
Generic rules catch generic problems. The real value comes from encoding your team’s institutional knowledge:
# semgrep custom rules
rules:
  - id: no-raw-sql-in-services
    patterns:
      - pattern: db.execute("...")
      - pattern-not: db.execute($QUERY, ...)   # Parameters are OK
    message: |
      Raw SQL strings without parameters. Use parameterized queries
      to prevent SQL injection. See: docs/database-patterns.md
    languages: [python]
    severity: ERROR
    
  - id: require-correlation-id-logging
    pattern: logger.$METHOD(...)
    pattern-not: logger.$METHOD(..., correlation_id=...)
    paths:
      include:
        - src/api/**
    message: |
      API handlers must include correlation_id in all log calls
      for distributed tracing. See: docs/logging-standards.md
    severity: WARNING
These custom rules carry your team’s knowledge forward. New engineers get the same guidance senior engineers give in code review — automatically.
The Human Review That Remains
After AI handles the mechanical review, human reviewers should focus on:
Intent verification — does this code do what the ticket describes?
Architectural decisions — is this the right approach? Any concerns about maintainability?
Domain correctness — does this business logic make sense?
Knowledge transfer — is there anything the author should know for future work?
Risk assessment — what could go wrong in production that tests don’t cover?
Set this expectation explicitly with your team. AI handles the mechanical; humans handle the judgment.
Measuring Success
Metrics worth tracking monthly:
Code Quality Health:
- Security vulnerabilities in production (target: trend down)
- Escaped defect rate (bugs found in production vs. in review)
- Time from PR open to first review (AI should reduce this)
- Reviewer time per PR (should drop for mechanical review)
- Developer satisfaction with review process (survey)
Don’t measure “AI comments accepted.” That optimizes for compliance, not quality.
Conclusion
AI-assisted code review works when it’s configured thoughtfully, calibrated to reduce noise, and positioned as a complement to human review rather than a replacement. The teams winning with it have invested in custom rules for their domain, track and act on feedback loops, and kept human reviewers focused on what AI can’t do.
Start with security scanning in CI, add an LLM reviewer configured for your codebase, measure what gets actioned vs. ignored, and iterate. The goal isn’t more comments — it’s better code with less review toil.
The best code review is fast, focused, and catches what matters. AI can help you get there.

AI-Assisted Code Review: How to Actually Improve Code Quality Without the Noise

댓글