Migration Plan: Current to Target Architecture

Version: 1.1.0
Status: READY
Date: 2025-01-29
Timeline: 4 weeks

Overview

This document provides a step-by-step migration plan from the current architecture (full array replacements, excessive re-renders) to the target architecture (message streaming, minimal re-renders).

CRITICAL: Phase 2 (Message Streaming) solves 70% of performance issues and MUST be prioritized.

Migration Principles

No Breaking Changes: All changes must be backward compatible
Feature Flags: New behavior behind flags during transition
Incremental Rollout: Test each phase before proceeding
Rollback Ready: Each step must be reversible

Phase 1: Foundation Cleanup (Week 1)

Step 1.1: Clean Up Storage (Day 1)

Actions:

// Remove from localStorage
localStorage.removeItem('sasha_chat_messages_*');

// Remove from sessionStorage
sessionStorage.removeItem('pendingSessionId');
sessionStorage.removeItem('activeSessionId');
sessionStorage.removeItem('sessionState');

Files to modify:

src/App.jsx - Remove localStorage message caching
src/components/ChatInterface.jsx - Remove sessionStorage duplicates

Validation:

No errors in console
Chat still works
Onboarding still works

Step 1.2: Fix useEffect Dependencies (Days 2-3)

Current Problem:

// BAD - Objects as dependencies
useEffect(() => {
  // ...
}, [projects, selectedProject, location]);

Fix:

// GOOD - Primitives as dependencies
useEffect(() => {
  // ...
}, [selectedProject?.id, location.pathname]);

Files to modify:

src/App.jsx (lines 128-209) - Split mega useEffect
src/components/ChatInterface.jsx - Fix all useEffect deps
src/hooks/useProjectWebSocketV2.js - Use primitive checks

Validation:

ESLint exhaustive-deps warnings resolved
Re-render count reduced by 30%

Step 1.3: Remove Unused Flags (Day 4)

Remove from reducer:

// Delete these flags
isProcessingWebSocket: false,  // Never checked
isNavigating: false            // Duplicates router

Files to modify:

src/reducers/projectReducer.js - Remove unused flags
Remove all SET_FLAG calls for these flags

Validation:

Search codebase - no references to removed flags
All features still work

Step 1.4: Implement Operation State Machine (Day 5)

Add to reducer:

// New state
operationState: 'idle' | 'selecting_project' | 'creating_session' | 'loading_files'
operationContext: { projectId?, sessionId?, timestamp }

// New action
case 'SET_OPERATION_STATE':
  return {
    ...state,
    operationState: action.payload.state,
    operationContext: action.payload.context
  };

Files to modify:

src/reducers/projectReducer.js - Add state machine
Update all components to use new state

Validation:

State transitions logged correctly
No invalid state combinations

Phase 2: Message Streaming Implementation (Week 2) - CRITICAL

THIS IS THE MOST CRITICAL PHASE - Solves 70% of performance issues

Step 2.1: Implement JSONL Incremental Reader (Days 6-7)

Create new file: server/services/message-stream.js

class MessageStreamReader {
  constructor() {
    this.lastMessagePosition = new Map(); // Track file positions
  }
  
  async readNewMessages(filePath, fromPosition = 0) {
    const stream = fs.createReadStream(filePath, { 
      start: fromPosition,
      encoding: 'utf8' 
    });
    
    const newMessages = [];
    const rl = readline.createInterface({ input: stream });
    
    for await (const line of rl) {
      if (line.trim()) {
        try {
          const message = JSON.parse(line);
          newMessages.push(message);
        } catch (e) {
          console.error('Failed to parse JSONL line:', e);
        }
      }
    }
    
    // Update position for next read
    const stats = await fs.promises.stat(filePath);
    this.lastMessagePosition.set(filePath, stats.size);
    
    return newMessages;
  }
}

Step 2.2: Update State Structure for Message Streaming (Day 8)

Add to reducer state: src/reducers/projectReducer.js

// Add message streams to state
messageStreams: {
  'session-456': {
    messages: [],  // Append-only array
    lastMessageId: 'msg-123',
    isStreaming: false,
    lastUpdate: Date.now()
  }
}

Step 2.3: Implement Message Streaming Actions (Day 9)

Add to reducer: src/reducers/projectReducer.js

case 'APPEND_MESSAGE':
  // Stream single message - no array replacement
  return {
    ...state,
    messageStreams: {
      ...state.messageStreams,
      [action.sessionId]: {
        ...state.messageStreams[action.sessionId],
        messages: [
          ...(state.messageStreams[action.sessionId]?.messages || []),
          action.message
        ],
        lastMessageId: action.message.id,
        isStreaming: action.message.isStreaming || false,
        lastUpdate: Date.now()
      }
    }
  };

case 'UPDATE_MESSAGE':
  // Update streaming message content in place
  const stream = state.messageStreams[action.sessionId];
  const messageIndex = stream.messages.findIndex(m => m.id === action.messageId);
  if (messageIndex === -1) return state;
  
  const updatedMessages = [...stream.messages];
  updatedMessages[messageIndex] = {
    ...updatedMessages[messageIndex],
    ...action.updates,
    isStreaming: action.isStreaming
  };
  
  return {
    ...state,
    messageStreams: {
      ...state.messageStreams,
      [action.sessionId]: {
        ...stream,
        messages: updatedMessages,
        isStreaming: action.isStreaming
      }
    }
  };

Step 2.4: Create Smart File Watcher with Streaming (Day 10)

Create new file: server/services/smart-watcher.js

class SmartWatcher {
  constructor() {
    this.watchers = new Map();
    this.messageReader = new MessageStreamReader();
    this.debounces = {
      documentation: 1000,  // 1s for docs
      session: 50,          // 50ms for streaming messages (CRITICAL)
      files: 500           // 500ms for code
    };
  }

  async routeFileChange(path, changeType) {
    if (path.includes('/docs/') && path.endsWith('.md')) {
      return { handler: 'documentation', debounce: 1000 };
    }
    if (path.includes('/sessions/') && path.endsWith('.jsonl')) {
      // For session files, read only new lines (streaming)
      const lastPos = this.messageReader.lastMessagePosition.get(path) || 0;
      const newMessages = await this.messageReader.readNewMessages(path, lastPos);
      return { 
        handler: 'message_stream', 
        debounce: 50,  // Very short for real-time feel
        messages: newMessages 
      };
    }
    return { handler: 'files', debounce: 500 };
  }
}

Step 2.5: Implement WebSocket Message Streaming (Day 10)

Update server to send streaming messages: server/index.js

// Instead of sending entire projects array
io.emit('message_streamed', {
  type: 'message_streamed',
  projectId: 'project-123',
  sessionId: 'session-456',
  message: {
    id: 'msg-123',
    type: 'assistant',
    content: '...',
    timestamp: new Date().toISOString(),
    isStreaming: true
  },
  operation: 'append'
});

Update client to handle streaming: src/hooks/useProjectWebSocketV2.js

useEffect(() => {
  if (latestMessage?.type === 'message_streamed') {
    dispatch({
      type: 'APPEND_MESSAGE',
      sessionId: latestMessage.sessionId,
      message: latestMessage.message
    });
  }
}, [messages]);

Performance Targets for This Phase:

WebSocket message size: ~100KB → <500 bytes
Re-renders per message: 10-20 → 1-2
Message display latency: 800ms → <100ms

Validation:

Messages stream individually, not as arrays
No full project array replacements
Chat updates without flashing
Scroll position preserved

Phase 3: Smart Watcher & Targeted Updates (Week 3)

Step 3.1: Handle Targeted Messages (Days 11-12)

Update reducer:

case 'DOCUMENTATION_UPDATED':
  // Show toast only, no state change
  showToast(`Documentation updated: ${action.payload.file}`);
  return state;

case 'SESSION_UPDATED':
  // Update only specific session
  return updateSessionMessages(state, action.payload);

case 'PROJECT_FILES_CHANGED':
  // Update only file cache
  return updateFileCache(state, action.payload);

Files to modify:

src/reducers/projectReducer.js - Add new action handlers
src/hooks/useProjectWebSocketV2.js - Listen for new messages

Validation:

Documentation changes = toast only
Session changes = chat update only
File changes = tree update only

Step 3.2: Implement URL Routing (Days 13-14)

Update routes:

// src/App.jsx
<Routes>
  <Route path="/project/:projectName/chat/:sessionId?" element={<ChatView />} />
  <Route path="/project/:projectName/files/*" element={<FilesView />} />
  <Route path="/project/:projectName/docs/*" element={<DocsView />} />
</Routes>

Add navigation updates:

// When selecting project
navigate(`/project/${projectName}/chat`);

// When selecting session
navigate(`/project/${projectName}/chat/${sessionId}`);

Files to modify:

src/App.jsx - Update route definitions
src/components/Sidebar.jsx - Update navigation calls
src/hooks/useNavigation.js - Create if needed

Validation:

URLs update correctly
Browser back/forward works
Deep linking works
Old URLs redirect (backward compat)

Step 3.3: Optimize Component Rendering (Day 15)

Add React.memo:

export default React.memo(ProjectCard, (prev, next) => {
  return prev.project.id === next.project.id &&
         prev.project.lastModified === next.project.lastModified;
});

Files to modify:

All components in src/components/ that receive projects
Use custom comparison functions

Validation:

React DevTools shows fewer re-renders
Performance profiler shows improvement

Phase 4: Testing & Rollout (Week 4)

Step 4.1: Performance Testing (Days 16-17)

Metrics to measure:

Re-renders per file save
WebSocket message size
Time to UI update
Memory usage over time

Tools:

React DevTools Profiler
Chrome Performance tab
Custom performance logging

Success criteria:

70% reduction in re-renders
90% reduction in WebSocket payload
Sub-100ms UI updates

Step 4.2: Integration Testing (Days 18-19)

Test scenarios:

Onboarding flow start to finish
Multi-project switching
Rapid file saves
Documentation generation
Session creation and updates

Validation:

All scenarios pass
No regressions
Performance targets met

Step 4.3: Gradual Rollout (Day 20)

Rollout plan:

Enable for internal testing (10% users)
Monitor for 24 hours
Enable for beta users (50% users)
Monitor for 48 hours
Full rollout (100% users)

Monitoring:

Error rates
Performance metrics
User feedback

Step 4.4: Cleanup (Day 21)

Remove old code:

Old watcher implementation
Backward compatibility shims
Feature flags

Final validation:

All tests pass
Documentation updated
No dead code remains

Rollback Plan

If issues arise at any phase:

Immediate: Toggle feature flag off
Quick: Revert last commit
Full: Restore from backup branch

Success Metrics

Week 1 Complete

Storage cleaned up
useEffect dependencies fixed
State machine implemented
30% reduction in re-renders

Week 2 Complete (CRITICAL MILESTONE)

Message streaming implemented
JSONL incremental reader working
WebSocket messages <500 bytes
Re-renders reduced to 1-2 per message
Message latency <100ms

Week 3 Complete

Smart watcher with proper debouncing
Documentation updates = toast only
File changes = tree update only
URL routing implemented

Week 4 Complete

All tests passing
Performance targets met (see table below)
Rolled out to all users
Old code removed

Final Performance Targets

Metric	Current	Target	Achieved
Re-renders per message	10-20	1-2	[ ]
WebSocket message size	~100KB	<500 bytes	[ ]
Message display latency	800ms	<100ms	[ ]
Project switches	3-5	1	[ ]
File fetches per project	10-40	1	[ ]
Debounce (messages)	300ms	50ms	[ ]

Risk Registry

Risk	Impact	Probability	Mitigation
Breaking changes	High	Low	Feature flags, testing
Performance regression	Medium	Medium	Monitoring, rollback plan
User confusion	Low	Low	Gradual rollout
Data loss	High	Very Low	Backups, testing

Communication Plan

Week 0: Announce migration plan to team
Week 1: Daily updates on progress
Week 2: Demo smart watcher to stakeholders
Week 3: Beta testing announcement
Week 4: Success announcement

Post-Migration

After successful migration:

Document lessons learned
Update onboarding for new architecture
Plan next optimization phase
Celebrate!

This migration plan provides a safe, incremental path from the current problematic architecture to the optimized target architecture.

Migration Plan: Current to Target Architecture

Related Documents

Overview

Migration Principles

Phase 1: Foundation Cleanup (Week 1)

Step 1.1: Clean Up Storage (Day 1)

Step 1.2: Fix useEffect Dependencies (Days 2-3)

Step 1.3: Remove Unused Flags (Day 4)

Step 1.4: Implement Operation State Machine (Day 5)

Phase 2: Message Streaming Implementation (Week 2) - CRITICAL

Step 2.1: Implement JSONL Incremental Reader (Days 6-7)

Step 2.2: Update State Structure for Message Streaming (Day 8)

Step 2.3: Implement Message Streaming Actions (Day 9)

Step 2.4: Create Smart File Watcher with Streaming (Day 10)

Step 2.5: Implement WebSocket Message Streaming (Day 10)

Phase 3: Smart Watcher & Targeted Updates (Week 3)

Step 3.1: Handle Targeted Messages (Days 11-12)

Step 3.2: Implement URL Routing (Days 13-14)

Step 3.3: Optimize Component Rendering (Day 15)

Phase 4: Testing & Rollout (Week 4)

Step 4.1: Performance Testing (Days 16-17)

Step 4.2: Integration Testing (Days 18-19)

Step 4.3: Gradual Rollout (Day 20)

Step 4.4: Cleanup (Day 21)

Rollback Plan

Success Metrics

Week 1 Complete

Week 2 Complete (CRITICAL MILESTONE)

Week 3 Complete

Week 4 Complete

Final Performance Targets

Risk Registry

Communication Plan

Post-Migration