Last updated: Aug 12, 2025, 01:09 PM UTC

Nginx Reverse Proxy Experiment for Docker Desktop vpnkit Issues

Problem Statement

When running the Sasha application in Docker Desktop for Mac, file uploads were failing with timeout errors (408 Request Timeout, 502 Bad Gateway). Investigation revealed this was due to a known issue with Docker Desktop's vpnkit networking layer on macOS.

The vpnkit Issue

Docker Desktop for Mac uses vpnkit to bridge between the macOS host and Linux containers. This networking layer has a bug where it incorrectly handles HTTP keep-alive connections:

  1. When a client reuses a keep-alive connection for multiple requests
  2. vpnkit may close the connection prematurely
  3. This causes ECONNRESET errors in Node.js applications
  4. Particularly affects multipart/form-data uploads

Solution Attempted: Nginx Reverse Proxy

We attempted to work around this issue by introducing Nginx as a reverse proxy between the client and the Node.js application.

Implementation Details

File: nginx.conf

server {
  listen 80;
  client_max_body_size 200m;
  client_body_timeout 300s;

  # Special handling for upload endpoints
  location ~ ^/api/(files/upload|projects/.*/upload-documents)$ {
    proxy_pass http://sasha-app:3010;
    proxy_http_version 1.1;
    proxy_request_buffering off;
    proxy_buffering off;
    proxy_set_header Connection "";
    # ... other headers
  }

  # WebSocket support
  location /ws {
    proxy_pass http://sasha-app:3010;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    # ... other headers
  }
}

File: docker-compose.test.yml

  • Added nginx-test service on port 3010
  • Moved Node.js app to internal-only access
  • All traffic routed through Nginx

Theory Behind the Solution

The idea was that Nginx would:

  1. Accept connections from the browser using HTTP/1.1
  2. Create fresh connections to the Node.js backend
  3. Avoid the keep-alive connection reuse issue
  4. Handle connection management more robustly than vpnkit

Why We Abandoned This Approach

1. Increased Complexity

  • Added another layer of infrastructure to maintain
  • Required careful configuration of proxy headers
  • Complicated debugging (logs split between Nginx and Node.js)

2. New Issues Introduced

  • WebSocket connections broke: Despite proper configuration, WebSocket connections weren't establishing correctly through the proxy
  • Authentication complications: Token passing and session management became more complex
  • Timeout cascades: Multiple timeout configurations needed coordination (Nginx, Node.js, Docker)

3. Not Solving Core Problem

  • File uploads still had issues even with Nginx
  • The problem appeared to be deeper in the Docker networking stack
  • Nginx was just moving the problem, not fixing it

4. Alternative Solutions Available

  • Testing in different Docker environments (Linux, Docker in VM)
  • Using Docker alternatives (Podman, native Linux)
  • Direct testing without Docker for development

Lessons Learned

  1. Docker Desktop limitations: The vpnkit networking layer has fundamental issues that can't always be worked around at the application level

  2. Proxy complexity: Adding a reverse proxy to fix networking issues often creates more problems than it solves

  3. Platform-specific issues: What works on Linux Docker may not work on Docker Desktop for Mac

  4. Simpler is better: Rather than adding complexity to work around platform bugs, it's often better to use a different platform

Recommended Approach

Instead of using Nginx as a workaround:

  1. For Development: Run the application directly without Docker on macOS
  2. For Testing: Use a Linux VM or cloud instance with native Docker
  3. For Production: Deploy to Linux-based environments where Docker networking is stable
  4. Alternative: Consider using Podman or other container runtimes that don't have vpnkit issues

Files to Remove/Revert

If reverting this experiment:

  • Remove nginx.conf
  • Revert docker-compose.test.yml to direct port mapping
  • Remove any Nginx-specific environment variables
  • Update documentation to note Docker Desktop limitations

References