Changelog: Specification v1 to v2

From: Compilation-Focused Specification (Original) To: Review-Focused Specification v2.0 Date: 2025-10-23 Reason: User clarification - "The extracted code doesn't have to work, it's for reviewing"

Summary of Changes

This document details the transformation from a compilation-focused tool to a review-optimized code extraction tool. The change eliminates ~60% of complexity while adding critical missing features for actual code review workflows.

🔴 Major Removals

1. Compilation Requirements ❌

Removed:

• Goal: "smallest compilable code closure"
• Single-file mode with merged, compilable output
• Multi-file mode with directory tree generation
• Import rewriting and alias preservation
• Package name conflict resolution
• Synthetic package creation (package main wrapper)
• go.mod generation and module synthesis
• Require/replace pruning
• Vendor mode support
• Workspace configuration
• Build verification tests

Rationale: Code doesn't need to compile for review purposes. Focusing on readability is more valuable.

2. Topological Sorting and Dependency Resolution ❌

Removed:

• Kahn's algorithm for declaration ordering
• Strongly connected component (SCC) detection
• Cycle resolution logic
• Init function ordering
• Dependency graph topological sort

Rationale: Declaration order doesn't matter for review. Natural source order is more intuitive.

3. Build Infrastructure ❌

Removed:

• Build tag handling (--build-tags flag)
• Platform-specific file inclusion/exclusion
• cgo detection and error handling
• Generated file skipping logic (--skip-generated)
• Vendor directory traversal (--vendor flag)
• File count limits (--max-files flag)

Rationale: Reviewers care about default build context. Complex build variations are edge cases.

4. Cross-Package Unexported Symbol Resolution ❌

Removed:

• Complex logic for including unexported symbols across packages
• Forcing multi-file mode when unexported needed
• Package boundary validation
• Access control enforcement

Rationale: Just note "unexported from package X" and move on. Simple and clear.

5. Two Output Modes ❌

Removed:

• --mode single-file|multi-file
• Single-file assembler (merge + rewrite)
• Multi-file assembler (directory tree)
• Mode selection logic

Rationale: One format (structured document) is clearer than two complex modes.

🟢 Major Additions

1. Depth Control ✅

Added:

• --depth N flag (default: 1)
• Depth 0: Target only
• Depth 1: Target + direct dependencies
• Depth N: Transitive to N levels
• Breadth-first traversal with depth tracking

Rationale: Essential for balancing context vs noise in reviews.

2. Caller Analysis ✅

Added:

• --show-callers flag
• Reverse dependency lookup
• Context snippets around call sites
• Test caller inclusion with --show-tests
• Caller grouping by file

Rationale: "Where is this used?" is the #1 question in code review.

3. Syntax Highlighting and Rich Formatting ✅

Added:

• HTML output with chroma/highlight.js
• Markdown with fenced code blocks
• File:line annotations
• Clickable hyperlinks in HTML
• Section separators
• Collapsible regions
• Copy buttons

Rationale: Readable output is the entire point of a review tool.

4. External Reference Stubs ✅

Added:

• --stub-external flag (default: true)
• Signature extraction for external symbols
• Clear "External References" section
• Type signature display
• Standard library notation

Rationale: Show what's being used without including entire stdlib/third-party.

5. Metrics and Analysis ✅

Added:

• --metrics flag
• Cyclomatic complexity calculation
• Lines of code (physical + logical)
• Dependency count (direct + transitive)
• External package list
• Metrics section in output

Rationale: Quick complexity assessment helps prioritize review effort.

6. Git Integration ✅

Added:

• --git-blame flag
• Last author/date per line
• Recent commit history for symbol
• Uncommitted change highlighting
• Git history section in output

Rationale: Historical context (who wrote this, when, why) is valuable for reviews.

7. JSON Output ✅

Added:

• --format json
• Structured machine-readable output
• Complete metadata
• Tool integration support
• Detailed schema (see spec appendix)

Rationale: Enables CI/CD integration, automated review tools, custom processing.

📊 Detailed Change Matrix

🔧 Architecture Changes

v1 Architecture (Compilation-Focused)

internal/extract/
  loader.go                   // packages.Load
  locator.go                  // position → symbol
  graph.go                    // dependency graph
  walker.go                   // AST walk
  assemble_single.go          // Single-file compiler
  assemble_multi.go           // Multi-file compiler
  manifest.go                 // JSON output
  rewrite.go                  // Import rewriting
  guard.go                    // Limits and validation

Total: 7 packages, ~5500 lines

v2 Architecture (Review-Focused)

internal/extract/
  loader.go                   // packages.Load wrapper
  locator.go                  // position → symbol
  collector.go                // Depth-limited BFS (NEW)
  analysis.go                 // Caller analysis + metrics (NEW)
  git.go                      // Git integration (NEW)
  format/
    markdown.go               // Markdown formatter (NEW)
    html.go                   // HTML formatter (NEW)
    json.go                   // JSON output (RENAMED from manifest.go)

Total: 4 packages (+ 3-file subpackage), ~2300 lines

Changes:

• ❌ Removed: graph.go, walker.go, assemble_single.go, assemble_multi.go, rewrite.go, guard.go
• ✅ Added: collector.go, analysis.go, git.go, format/ subpackage
• ✅ Simplified: loader.go, locator.go (less complexity)
• ✅ Renamed: manifest.go → format/json.go (clearer purpose)

🎯 API Changes

v1 API (Compilation-Focused)

type Target struct {
  Root, File string
  Line, Col  int
  BuildTags  []string    // ❌ Removed
  IncludeTests bool      // ❌ Removed (now a flag, not target property)
}

type Result struct {
  Mode    string         // ❌ Removed (single-file vs multi-file)
  Files   []EmittedFile  // ❌ Removed (compilable .go files)
  Manifest Manifest      // ✅ Renamed to Metadata
}

func Extract(ctx context.Context, t Target, opts Options) (Result, error)

v2 API (Review-Focused)

type Target struct {
  Root   string  // Module root
  File   string  // Source file
  Line   int     // Line number
  Column int     // Column (default: 1)
}

type Options struct {
  Depth          int      // ✅ NEW: Dependency depth
  Format         string   // ✅ NEW: markdown/html/json
  StubExternal   bool     // ✅ NEW: Show signatures
  ShowCallers    bool     // ✅ NEW: Reverse deps
  ShowTests      bool     // ✅ NEW: Include tests
  ContextLines   int      // ✅ NEW: Extra lines
  Annotate       bool     // ✅ NEW: File:line comments
  IncludeMetrics bool     // ✅ NEW: Complexity metrics
  GitBlame       bool     // ✅ NEW: Git history
}

type Symbol struct {        // ✅ NEW: Explicit symbol type
  Package   string
  Name      string
  Kind      string
  Receiver  string
  File      string
  Line      int
  EndLine   int
  Code      string
  Doc       string
  Exported  bool
}

type Reference struct {     // ✅ NEW: Explicit reference type
  Symbol       Symbol
  Reason       string       // ✅ NEW: Why included
  Depth        int          // ✅ NEW: Depth in tree
  External     bool         // ✅ NEW: External vs local
  Stub         bool         // ✅ NEW: Signature only
  Signature    string       // ✅ NEW: For stubs
  ReferencedBy string       // ✅ NEW: Back-reference
}

type Caller struct {        // ✅ NEW: Reverse dependency
  File     string
  Line     int
  Function string
  Context  string
}

type Metrics struct {       // ✅ NEW: Complexity metrics
  LinesOfCode         int
  LogicalLines        int
  CyclomaticComplexity int
  DependencyCount     int
  DirectDeps          int
  TransitiveDeps      int
  ExternalPackages    []string
}

type GitBlame struct {      // ✅ NEW: Git history
  Commit  string
  Author  string
  Date    time.Time
  Message string
}

type Extract struct {       // ✅ NEW: Structured result
  Target     Symbol
  References []Reference
  External   []string
  Callers    []Caller
  Metrics    *Metrics
  GitHistory []GitBlame
  Graph      string
}

type Result struct {
  Extract  Extract          // ✅ NEW: Structured extract
  Rendered string           // ✅ NEW: Formatted output
  Metadata Metadata         // ✅ RENAMED from Manifest
}

func ExtractSymbol(ctx context.Context, target Target, opts Options) (*Result, error)

// ✅ NEW: Convenience functions
func ExtractToFile(ctx context.Context, target Target, opts Options, outputPath string) error
func ExtractToMarkdown(ctx context.Context, target Target, depth int) (string, error)
func ExtractToHTML(ctx context.Context, target Target, depth int) (string, error)
func ExtractToJSON(ctx context.Context, target Target, depth int) (string, error)

Summary:

• More explicit types (Symbol, Reference, Caller, Metrics, GitBlame)
• Clearer separation of concerns
• Richer metadata
• Better ergonomics (convenience functions)

🧪 Testing Changes

v1 Testing (Compilation-Focused)

Focus:

• Does extracted code compile?
• Are imports correct?
• Is topological order valid?
• Does go.mod work?

Scenario: Extract handler that uses pkg/service.UserService
  Given example module "ex2"
  When I run go-scope for "cmd/api/main.go" line 30 mode "multi-file"
  Then emitted files include "pkg/service/service.go"
  And emitted files exclude unrelated "pkg/repo/unused.go"
  And I can run "go build" in output directory  # ❌ No longer relevant

v2 Testing (Review-Focused)

Focus:

• Are correct symbols extracted?
• Is depth limiting accurate?
• Are annotations correct?
• Are callers found?
• Is output valid markdown/HTML/JSON?

Scenario: Extract handler with depth control
  Given example module "ex2"
  When I extract "cmd/api/main.go" line 30 with depth 0
  Then output includes only "handleCreateUser"
  And output shows "UserService.CreateUser" as stub

  When I extract same location with depth 2
  Then output includes "handleCreateUser"
  And output includes "UserService.CreateUser" implementation
  And output includes "validateEmail" helper

New Scenarios (not in v1):

• Caller discovery
• Metric accuracy
• Git blame integration
• Format validation (HTML, JSON)
• Depth limiting
• External stub generation

• Compilation tests
• Build tag validation
• go.mod generation
• Import conflict resolution
• Topological order verification

📝 CLI Changes

Removed Flags

Added Flags

Changed Flags

📈 Impact Analysis

Development Time

Maintenance Burden

User Value

🚀 Migration Path

For users of hypothetical v1 (if it existed):

Command Migration

v1 Command:

go-scope \
  --root . \
  --file pkg/service/user.go \
  --line 128 \
  --mode single-file \
  --output extracted.go

v2 Equivalent:

go-scope \
  --root . \
  --file pkg/service/user.go \
  --line 128 \
  --format markdown \
  --output extracted.md

API Migration

v1 Code:

result, err := extract.Extract(ctx,
    extract.Target{
        Root: ".",
        File: "pkg/service/user.go",
        Line: 128,
        BuildTags: []string{"prod"},
        IncludeTests: false,
    },
    extract.Options{})

v2 Code:

result, err := extract.ExtractSymbol(ctx,
    extract.Target{
        Root: ".",
        File: "pkg/service/user.go",
        Line: 128,
    },
    extract.Options{
        Depth: 1,
        Format: "markdown",
        ShowTests: false,
    })

🎓 Lessons Learned

What Went Wrong in v1

What v2 Gets Right

Key Takeaways

• Always clarify the goal first: "Compilable" vs "reviewable" is a fundamental difference
• Question assumptions: "Does it need to compile?" could have been asked earlier
• Start with MVP: Build the simplest thing that works, then add features
• User feedback early: A quick prototype would have revealed the mismatch

📋 Checklist for Adopting v2

• [ ] Review and approve v2 specification
• [ ] Archive v1 specification (rename to SPEC_v1_COMPILATION_ARCHIVE.md)
• [ ] Update project README to reference v2
• [ ] Create v2 milestones and issues
• [ ] Set up project structure per v2 architecture
• [ ] Implement Phase 1 (Core Extraction)
• [ ] Implement Phase 2 (Review Features)
• [ ] Implement Phase 3 (Advanced Features)
• [ ] Implement Phase 4 (Polish)
• [ ] Release v1.0.0

🔮 Future Considerations

Features Not in Either v1 or v2

Some features weren't in v1 and aren't in v2 scope, but could be considered later:

• IDE integration: VS Code extension, LSP server
• Web service: HTTP API for remote extraction
• Collaboration: Shared extracts, annotations, discussions
• Diff mode: Compare two versions of a symbol
• Advanced analysis: Data flow, type hierarchy, impact analysis

📊 Side-by-Side Comparison

Example: Extract CreateUser Method

v1 Output (Single-File Mode):

// Merged, compilable .go file
package main

import (
    "context"
    "database/sql"  // Full import
    "fmt"
    "time"
)

// User type (from pkg/model)
type User struct {
    ID        int64
    Email     string
    Name      string
    CreatedAt time.Time
}

// validateEmail (from pkg/service)
func validateEmail(email string) error {
    // ... implementation
}

// UserService stub
type UserService struct {
    db interface{} // Simplified
}

// CreateUser (target)
func (s *UserService) CreateUser(ctx context.Context, email, name string) (*User, error) {
    // ... implementation
}

func main() {
    // Stub main (for compilation)
}

Problems:

• Package names lost (everything in main)
• No context (where is this from?)
• No caller information
• No metrics
• Hard to read (no structure)

# Code Extract: CreateUser

**File**: pkg/service/user.go:128
**Package**: github.com/example/app/pkg/service
**Kind**: Method (UserService)

## Metrics
- Cyclomatic Complexity: 3
- Lines of Code: 17
- Dependencies: 4

## Target Symbol

## Direct Dependencies (depth 1)

### validateEmail
**File**: pkg/service/validation.go:45

### User
**File**: pkg/model/user.go:12

## External References
- database/sql: `DB.Create(value interface{}) error`
- fmt: `Errorf(format string, a ...any) error`

## Called By
- cmd/api/handlers.go:67 - `handleCreateUser`
- cmd/cli/user.go:123 - `createUserCommand`
- pkg/service/user_test.go:34 - `TestCreateUser`

## Recent Changes
- 2025-10-20 by alice@example.com - "Add email validation"

Benefits:

• Clear structure with sections
• Full context (package paths, file:line)
• Caller information
• Metrics for quick assessment
• Annotations show dependencies
• Much more readable

✅ Approval Status

• [ ] Analysis reviewed and approved
• [ ] v2 specification reviewed and approved
• [ ] Changelog reviewed and approved
• [ ] Ready to begin implementation

End of Changelog