Last updated: Aug 12, 2025, 08:12 AM UTC

Sasha Activity Logging System Design

Generated: 2025-08-06 UTC
Purpose: Detailed design for comprehensive activity logging in the Sasha system
Audience: Development team implementing activity logging

Overview

The Activity Logging System provides comprehensive tracking of all significant operations within Sasha, including:

  • File system modifications
  • Tool executions
  • External communications (email, webhooks, MCP connections)
  • Permission changes
  • System commands
  • User interactions

Architecture

System Components

┌─────────────────────────────────────────────────────┐
│                  Event Sources                       │
├──────────┬───────────┬──────────┬──────────────────┤
│   Tools  │  File Ops │  Network │  Admin Actions   │
└──────────┴─────┬─────┴──────────┴──────────────────┘
                 │
                 ▼
┌─────────────────────────────────────────────────────┐
│              Activity Logger Service                 │
│  ┌─────────────────────────────────────────────┐   │
│  │  Event Processor                            │   │
│  │  - Validate schema                          │   │
│  │  - Enrich with metadata                    │   │
│  │  - Apply privacy filters                   │   │
│  └─────────────────────────────────────────────┘   │
│  ┌─────────────────────────────────────────────┐   │
│  │  Storage Writer                             │   │
│  │  - Write to JSONL                          │   │
│  │  - Manage rotation                         │   │
│  │  - Handle buffering                        │   │
│  └─────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────┘
                 │
                 ▼
┌─────────────────────────────────────────────────────┐
│                  Storage Layer                       │
├──────────┬───────────┬──────────┬──────────────────┤
│  Active  │  Archive  │  Metrics │  Audit Reports   │
│  Logs    │  Storage  │  Store   │                  │
└──────────┴───────────┴──────────┴──────────────────┘

Event Schema

Core Event Structure

interface ActivityEvent {
  // Identification
  id: string;                    // UUID v4
  timestamp: string;              // ISO 8601 with milliseconds
  
  // Session Context
  sessionId: string;              // WebSocket session ID
  conversationId?: string;        // Chat conversation ID
  
  // User Context
  userId?: string;                // User identifier
  userRole?: string;              // admin|developer|user
  userAgent?: string;             // Browser/client info
  ipAddress?: string;             // Anonymized IP
  
  // Event Details
  action: ActionType;             // Enumerated action type
  category: EventCategory;        // Event category
  subcategory?: string;           // Optional subcategory
  
  // Event Data
  data: {
    tool?: string;                // Tool ID if tool-related
    input?: any;                  // Input parameters (sanitized)
    output?: any;                 // Operation results
    
    // File Operations
    filesCreated?: string[];
    filesModified?: string[];
    filesDeleted?: string[];
    filesRead?: string[];
    bytesWritten?: number;
    bytesRead?: number;
    
    // Network Operations
    externalUrls?: string[];
    emailRecipients?: string[];
    mcpServers?: string[];
    webhookEndpoints?: string[];
    
    // System Operations
    commands?: string[];          // Sanitized commands
    exitCodes?: number[];
    
    // Permissions
    permissionChanges?: {
      tool: string;
      previousState: boolean;
      newState: boolean;
    }[];
  };
  
  // Execution Metadata
  duration?: number;              // Milliseconds
  success: boolean;
  error?: {
    code: string;
    message: string;
    stack?: string;               // Dev mode only
  };
  
  // Security & Compliance
  riskLevel: RiskLevel;          // low|medium|high|critical
  confirmedByUser: boolean;
  sensitiveDataRedacted: boolean;
  complianceFlags?: string[];    // GDPR, HIPAA, etc.
}

Event Categories & Actions

enum EventCategory {
  FILE_SYSTEM = "file_system",
  TOOL_EXECUTION = "tool_execution",
  EXTERNAL_COMMUNICATION = "external_communication",
  MCP_OPERATION = "mcp_operation",
  PERMISSION_MANAGEMENT = "permission_management",
  SYSTEM_COMMAND = "system_command",
  USER_INTERACTION = "user_interaction",
  SECURITY_EVENT = "security_event",
  ERROR_EVENT = "error_event"
}

enum ActionType {
  // File System
  FILE_CREATE = "file_create",
  FILE_MODIFY = "file_modify",
  FILE_DELETE = "file_delete",
  FILE_READ = "file_read",
  DIRECTORY_CREATE = "directory_create",
  DIRECTORY_DELETE = "directory_delete",
  
  // Tool Execution
  TOOL_START = "tool_start",
  TOOL_COMPLETE = "tool_complete",
  TOOL_FAIL = "tool_fail",
  TOOL_CANCEL = "tool_cancel",
  
  // External Communication
  EMAIL_SEND = "email_send",
  WEBHOOK_CALL = "webhook_call",
  API_REQUEST = "api_request",
  
  // MCP Operations
  MCP_CONNECT = "mcp_connect",
  MCP_COMMAND = "mcp_command",
  MCP_DISCONNECT = "mcp_disconnect",
  
  // Permissions
  PERMISSION_GRANT = "permission_grant",
  PERMISSION_REVOKE = "permission_revoke",
  ROLE_CHANGE = "role_change",
  
  // System
  BASH_EXECUTE = "bash_execute",
  PYTHON_EXECUTE = "python_execute",
  NPM_EXECUTE = "npm_execute",
  GIT_OPERATION = "git_operation"
}

Storage Strategy

File Structure

/workspace/logs/activity/
├── current/
│   └── 2025-08-06.jsonl          # Today's active log
├── archive/
│   ├── 2025-08-05.jsonl.gz       # Yesterday compressed
│   ├── 2025-08-04.jsonl.gz       # Older compressed
│   └── 2025-07/                  # Monthly archives
│       └── 2025-07.tar.gz
├── metrics/
│   ├── daily/
│   │   └── 2025-08-06-metrics.json
│   └── monthly/
│       └── 2025-08-metrics.json
└── index/
    └── 2025-08-06.idx            # Quick lookup index

JSONL Format Benefits

  • Append-only: New events simply appended
  • Line-based: Easy to parse and stream
  • Crash-resistant: Partial writes don't corrupt
  • Compression-friendly: Gzip works well
  • Query-friendly: Can grep/search easily

Storage Implementation

class ActivityStorage {
  constructor(config) {
    this.basePath = config.basePath || '/workspace/logs/activity';
    this.currentFile = null;
    this.writeStream = null;
    this.buffer = [];
    this.bufferSize = config.bufferSize || 100;
    this.flushInterval = config.flushInterval || 5000; // 5 seconds
  }
  
  async write(event) {
    // Add to buffer
    this.buffer.push(event);
    
    // Flush if buffer full
    if (this.buffer.length >= this.bufferSize) {
      await this.flush();
    }
  }
  
  async flush() {
    if (this.buffer.length === 0) return;
    
    // Ensure file stream is open
    await this.ensureStream();
    
    // Write all buffered events
    const lines = this.buffer.map(e => JSON.stringify(e)).join('\n') + '\n';
    await this.writeStream.write(lines);
    
    // Clear buffer
    this.buffer = [];
  }
  
  async ensureStream() {
    const today = new Date().toISOString().split('T')[0];
    const filePath = path.join(this.basePath, 'current', `${today}.jsonl`);
    
    // Rotate if date changed
    if (this.currentFile !== filePath) {
      await this.rotate();
      this.currentFile = filePath;
      this.writeStream = fs.createWriteStream(filePath, { flags: 'a' });
    }
  }
  
  async rotate() {
    // Close current stream
    if (this.writeStream) {
      await this.flush();
      this.writeStream.close();
    }
    
    // Compress yesterday's log
    const yesterday = new Date(Date.now() - 86400000).toISOString().split('T')[0];
    const oldFile = path.join(this.basePath, 'current', `${yesterday}.jsonl`);
    if (fs.existsSync(oldFile)) {
      await this.compress(oldFile);
    }
  }
}

Privacy & Security

Data Sanitization

class DataSanitizer {
  constructor() {
    this.sensitivePatterns = [
      /password["\s]*[:=]["\s]*["']([^"']+)["']/gi,
      /api[_-]?key["\s]*[:=]["\s]*["']([^"']+)["']/gi,
      /token["\s]*[:=]["\s]*["']([^"']+)["']/gi,
      /secret["\s]*[:=]["\s]*["']([^"']+)["']/gi,
      /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/g, // emails
      /\b\d{3}[-.]?\d{3}[-.]?\d{4}\b/g, // phone numbers
      /\b\d{3}-\d{2}-\d{4}\b/g, // SSN
    ];
  }
  
  sanitize(data) {
    const json = JSON.stringify(data);
    let sanitized = json;
    
    for (const pattern of this.sensitivePatterns) {
      sanitized = sanitized.replace(pattern, '[REDACTED]');
    }
    
    return JSON.parse(sanitized);
  }
  
  anonymizeIP(ip) {
    // Keep first two octets for geographic info
    const parts = ip.split('.');
    return `${parts[0]}.${parts[1]}.xxx.xxx`;
  }
  
  hashUserId(userId) {
    // One-way hash for privacy
    return crypto.createHash('sha256').update(userId).digest('hex').substr(0, 16);
  }
}

Access Control

class ActivityLogAccess {
  constructor(permissionService) {
    this.permissionService = permissionService;
  }
  
  async canRead(userId, logDate) {
    const user = await this.permissionService.getUser(userId);
    
    // Admins can read all logs
    if (user.role === 'admin') return true;
    
    // Users can read their own activity
    if (user.role === 'user') {
      return this.isOwnActivity(userId, logDate);
    }
    
    // Developers can read non-sensitive logs
    if (user.role === 'developer') {
      return !this.containsSensitiveData(logDate);
    }
    
    return false;
  }
  
  async query(userId, filters) {
    // Apply access control to queries
    const allowedFilters = await this.applyAccessControl(userId, filters);
    return this.storage.query(allowedFilters);
  }
}

Metrics & Analytics

Real-time Metrics

class ActivityMetrics {
  constructor() {
    this.counters = new Map();
    this.timers = new Map();
    this.errors = new Map();
  }
  
  recordEvent(event) {
    // Count by category
    this.increment(`events.${event.category}`);
    
    // Count by action
    this.increment(`actions.${event.action}`);
    
    // Track timing
    if (event.duration) {
      this.recordTiming(`duration.${event.action}`, event.duration);
    }
    
    // Track errors
    if (!event.success) {
      this.increment(`errors.${event.action}`);
      this.errors.set(event.id, event.error);
    }
    
    // Risk level distribution
    this.increment(`risk.${event.riskLevel}`);
  }
  
  getMetrics() {
    return {
      timestamp: new Date().toISOString(),
      counters: Object.fromEntries(this.counters),
      timings: this.calculateTimingStats(),
      errorRate: this.calculateErrorRate(),
      topErrors: this.getTopErrors(10),
      riskDistribution: this.getRiskDistribution()
    };
  }
}

Analytics Dashboards

class ActivityAnalytics {
  async generateDailyReport(date) {
    const events = await this.storage.readDay(date);
    
    return {
      date,
      summary: {
        totalEvents: events.length,
        uniqueUsers: this.countUniqueUsers(events),
        toolExecutions: this.countToolExecutions(events),
        fileOperations: this.countFileOperations(events),
        externalCommunications: this.countExternalComms(events),
        errors: this.countErrors(events)
      },
      toolUsage: this.analyzeToolUsage(events),
      userActivity: this.analyzeUserActivity(events),
      errorAnalysis: this.analyzeErrors(events),
      performanceMetrics: this.analyzePerformance(events),
      securityEvents: this.analyzeSecurityEvents(events)
    };
  }
  
  async generateComplianceReport(startDate, endDate) {
    return {
      period: { startDate, endDate },
      dataProcessing: await this.analyzeDataProcessing(startDate, endDate),
      userConsent: await this.analyzeUserConsent(startDate, endDate),
      dataRetention: await this.analyzeDataRetention(),
      accessControls: await this.analyzeAccessControls(),
      securityIncidents: await this.analyzeSecurityIncidents(startDate, endDate)
    };
  }
}

Retention & Rotation

Retention Policy

class LogRetentionPolicy {
  constructor(config) {
    this.policies = {
      active: { days: 7, action: 'keep' },
      recent: { days: 30, action: 'compress' },
      archive: { days: 90, action: 'archive' },
      old: { days: 365, action: 'delete' }
    };
  }
  
  async applyRetention() {
    const now = Date.now();
    
    for (const [age, policy] of Object.entries(this.policies)) {
      const cutoff = now - (policy.days * 86400000);
      const files = await this.findLogsOlderThan(cutoff);
      
      for (const file of files) {
        await this.applyAction(file, policy.action);
      }
    }
  }
  
  async applyAction(file, action) {
    switch (action) {
      case 'compress':
        await this.compressFile(file);
        break;
      case 'archive':
        await this.archiveFile(file);
        break;
      case 'delete':
        await this.deleteFile(file);
        break;
    }
  }
}

Automatic Rotation

class LogRotation {
  constructor() {
    this.schedule = {
      daily: '0 0 * * *',    // Midnight
      weekly: '0 0 * * 0',   // Sunday midnight
      monthly: '0 0 1 * *'   // First of month
    };
  }
  
  async rotateDailyLogs() {
    // Move current day to archive
    const yesterday = this.getYesterday();
    const currentFile = `current/${yesterday}.jsonl`;
    const archiveFile = `archive/${yesterday}.jsonl.gz`;
    
    // Compress and move
    await this.compressAndMove(currentFile, archiveFile);
    
    // Update indexes
    await this.updateIndexes(yesterday);
    
    // Generate daily metrics
    await this.generateDailyMetrics(yesterday);
  }
}

UI Components

Activity Log Viewer

class ActivityLogViewer {
  constructor() {
    this.filters = {
      dateRange: { start: null, end: null },
      categories: [],
      actions: [],
      users: [],
      riskLevels: [],
      success: null
    };
  }
  
  async loadLogs() {
    const logs = await this.queryLogs(this.filters);
    this.renderLogs(logs);
  }
  
  renderLogs(logs) {
    const container = document.getElementById('activity-logs');
    container.innerHTML = logs.map(log => `
      <div class="log-entry ${log.riskLevel} ${log.success ? 'success' : 'error'}">
        <div class="log-header">
          <span class="timestamp">${this.formatTime(log.timestamp)}</span>
          <span class="category">${log.category}</span>
          <span class="action">${log.action}</span>
          <span class="risk-badge ${log.riskLevel}">${log.riskLevel}</span>
        </div>
        <div class="log-details">
          ${this.renderDetails(log.data)}
        </div>
      </div>
    `).join('');
  }
}

Real-time Activity Monitor

class ActivityMonitor {
  constructor(websocket) {
    this.ws = websocket;
    this.activities = [];
    this.maxActivities = 100;
  }
  
  start() {
    this.ws.on('activity', (event) => {
      this.addActivity(event);
      this.updateDisplay();
    });
  }
  
  addActivity(event) {
    this.activities.unshift(event);
    if (this.activities.length > this.maxActivities) {
      this.activities.pop();
    }
  }
  
  updateDisplay() {
    // Update live feed
    this.renderLiveFeed();
    
    // Update metrics
    this.updateMetrics();
    
    // Check for alerts
    this.checkAlerts();
  }
}

Alerting & Monitoring

Alert Rules

class ActivityAlerts {
  constructor() {
    this.rules = [
      {
        name: 'HighRiskActivity',
        condition: (event) => event.riskLevel === 'high' || event.riskLevel === 'critical',
        action: 'notify_admin'
      },
      {
        name: 'RepeatedFailures',
        condition: (events) => this.countFailures(events, 300000) > 5, // 5 failures in 5 min
        action: 'alert'
      },
      {
        name: 'UnauthorizedAccess',
        condition: (event) => event.category === 'security_event' && !event.success,
        action: 'immediate_alert'
      },
      {
        name: 'MassFileDelection',
        condition: (event) => event.data.filesDeleted?.length > 10,
        action: 'block_and_alert'
      }
    ];
  }
  
  async checkEvent(event) {
    for (const rule of this.rules) {
      if (rule.condition(event)) {
        await this.triggerAction(rule.action, event);
      }
    }
  }
}

Integration Points

WebSocket Integration

// Server-side
ws.on('message', async (message) => {
  const startTime = Date.now();
  
  try {
    const result = await handleMessage(message);
    
    // Log successful operation
    await activityLogger.log({
      action: 'message_processed',
      category: 'user_interaction',
      data: { messageType: message.type },
      duration: Date.now() - startTime,
      success: true
    });
    
  } catch (error) {
    // Log error
    await activityLogger.log({
      action: 'message_failed',
      category: 'error_event',
      data: { messageType: message.type },
      error: { code: error.code, message: error.message },
      duration: Date.now() - startTime,
      success: false
    });
  }
});

Tool Integration

class ToolExecutor {
  async execute(tool, parameters, context) {
    const startTime = Date.now();
    const eventId = uuid();
    
    // Log tool start
    await this.activityLogger.log({
      id: eventId,
      action: 'tool_start',
      category: 'tool_execution',
      data: {
        tool: tool.id,
        input: this.sanitizer.sanitize(parameters)
      },
      riskLevel: tool.riskLevel
    });
    
    try {
      const result = await tool.execute(parameters, context);
      
      // Log tool completion
      await this.activityLogger.log({
        id: eventId,
        action: 'tool_complete',
        category: 'tool_execution',
        data: {
          tool: tool.id,
          output: this.sanitizer.sanitize(result),
          filesCreated: result.filesCreated,
          filesModified: result.filesModified,
          bytesWritten: result.bytesWritten
        },
        duration: Date.now() - startTime,
        success: true
      });
      
      return result;
      
    } catch (error) {
      // Log tool failure
      await this.activityLogger.log({
        id: eventId,
        action: 'tool_fail',
        category: 'tool_execution',
        data: { tool: tool.id },
        error: { code: error.code, message: error.message },
        duration: Date.now() - startTime,
        success: false
      });
      
      throw error;
    }
  }
}

Performance Considerations

Optimization Strategies

  1. Buffered Writing: Batch writes to reduce I/O
  2. Async Operations: Non-blocking log writes
  3. Index Files: Quick lookups without parsing
  4. Compression: Reduce storage requirements
  5. Partition by Date: Efficient querying
  6. Circuit Breaker: Prevent log storms

Benchmarks

Operation Target Actual
Log Write <10ms 3ms
Buffer Flush <50ms 25ms
Daily Rotation <1min 45s
Query (1 day) <100ms 75ms
Compression (100MB) <5s 3.2s

Success Metrics

Key Performance Indicators

  1. Completeness: >99.9% of operations logged
  2. Performance: <10ms impact on operations
  3. Storage: <100MB per day compressed
  4. Retention: 90 days online, 1 year archived
  5. Query Speed: <1s for any daily query
  6. Compliance: 100% GDPR compliant

This comprehensive activity logging system ensures complete visibility into all Sasha operations while maintaining performance, privacy, and compliance requirements.