wallarm/JUMP_SERVER.md

# Jump Server Setup for Wallarm Remote Assistance

This guide explains how to configure a Linux VM as a jump server so engineers
can securely access customer Wallarm nodes through the `wallarm --tunnel` feature.

## Architecture

```
Customer VM (VM-A)              Jump Server (VM-B)             Engineer
┌──────────────────┐   TLS:443  ┌──────────────────┐   SSH    ┌──────────┐
│ wallarm binary   │───────────►│ sshd              │◄─────────│  You     │
│ (reverse tunnel) │◄───────────│                   │          │          │
│                  │            │ localhost:9042    │          │          │
│                  │            │   → VM-A:22       │          │          │
└──────────────────┘            └──────────────────┘          └──────────┘
```

VM-A opens an outbound TLS connection to VM-B on port 443 (looks like HTTPS to
firewalls).  VM-B terminates TLS and proxies to its internal SSH server.  VM-A
then requests a reverse port forward: VM-B:9042 → VM-A:22.  The engineer SSH's
into VM-B, then jumps through localhost:9042 to reach VM-A.

## Prerequisites

- **Linux VM** (any distro) with a public IP or reachable via Zoraxy/Nginx reverse proxy
- **OpenSSH server** 7.6+
- **Port 443** accessible from customer environments (outbound only, no inbound needed)

## Step 1: Install and Configure SSH Server

```bash
# Install SSH server
apt-get install -y openssh-server    # Debian/Ubuntu
yum install -y openssh-server        # RHEL/CentOS

# Edit SSH config
nano /etc/ssh/sshd_config
```

Add or uncomment these lines:

```ini
# Required: allow TCP forwarding for the reverse tunnel
AllowTcpForwarding yes

# Required: keep idle tunnels alive
ClientAliveInterval 60
ClientAliveCountMax 3

# Optional: allow remote binds on non-localhost addresses
# (only needed if you want engineers to connect directly to VM-B:9042
#  instead of SSH'ing into VM-B first)
GatewayPorts yes
```

Restart SSH:

```bash
systemctl restart sshd
```

## Step 2: Create the Tunnel User

This user is used by the `wallarm` binary on customer VMs to authenticate
and open the reverse tunnel.  It should NOT have a shell — it exists only
for port forwarding.

```bash
# Create tunnel-only user (no shell, no home directory needed)
useradd -m -s /bin/false wallarm-tunnel

# Create SSH directory
mkdir -p ~wallarm-tunnel/.ssh
chmod 700 ~wallarm-tunnel/.ssh
```

### Option A: Password Authentication (simpler for customers)

```bash
# Set a password for the tunnel user
passwd wallarm-tunnel
```

The customer enters this password when prompted by `wallarm`.

### Option B: SSH Key Authentication (more secure)

Generate a key pair and share the private key with customers:

```bash
# On the jump server
ssh-keygen -t ed25519 -f ~wallarm-tunnel/.ssh/wallarm_tunnel -N "" -C "wallarm-tunnel"
cat ~wallarm-tunnel/.ssh/wallarm_tunnel.pub >> ~wallarm-tunnel/.ssh/authorized_keys
chmod 600 ~wallarm-tunnel/.ssh/authorized_keys
chown -R wallarm-tunnel:wallarm-tunnel ~wallarm-tunnel/.ssh

# Share the private key securely with customers
cat ~wallarm-tunnel/.ssh/wallarm_tunnel
```

The customer provides the key path when prompted by `wallarm`.

## Step 3: Zoraxy / Reverse Proxy (Optional)

If the jump server sits behind a Zoraxy edge proxy, configure it to forward
TLS:443 → internal SSH:22.  This lets `wallarm` connect over port 443 (which
passes through most corporate firewalls).

### Zoraxy Configuration

1. Add a new proxy rule:
   - **Domain**: `ssh.sechpoint.app`
   - **Target**: `tcp://<jump-server-ip>:22`
   - **TLS**: Enabled (auto-cert or custom)

2. No WebSocket or HTTP mode needed — Zoraxy proxies raw TCP.

### Manual Nginx Stream Proxy (Alternative)

```nginx
stream {
    server {
        listen 443 ssl;
        proxy_pass <jump-server-ip>:22;
        ssl_certificate     /etc/ssl/certs/jump.crt;
        ssl_certificate_key /etc/ssl/private/jump.key;
    }
}
```

## Step 4: Test the Setup

### From the Jump Server Itself

```bash
# Verify SSH is listening
ss -tlnp | grep 22

# Verify tunnel user can authenticate (no shell expected)
ssh wallarm-tunnel@localhost echo test
# Expected: command fails (user has no shell), authentication succeeds
```

### From an External Machine (Customer VM)

```bash
# Test TLS connection (if Zoraxy is configured)
openssl s_client -connect ssh.sechpoint.app:443 </dev/null 2>/dev/null | head -5

# Test SSH over TLS
ssh -p 443 wallarm-tunnel@ssh.sechpoint.app
# Expected: "This account is currently not available" or immediate disconnect
# This confirms authentication works but no shell is allowed — correct.
```

## How wallarm Uses It

On the customer VM, the `wallarm` binary offers two paths:

1. **`wallarm --tunnel`** — starts tunnel with pre-configured credentials (build-time or env var)
2. **`wallarm` → Remote Assistance** — prompts for jumphost URL, username, password/key interactively

The binary does:
1. TLS dial to `ssh.sechpoint.app:443`
2. SSH authenticate as `wallarm-tunnel`
3. Request reverse forward: `0.0.0.0:9042 → localhost:22` (or `localhost:9042` if GatewayPorts is off)
4. Keep tunnel alive with 30s heartbeats

## Engineer Connection (Assistance Flow)

After the customer starts the tunnel, the engineer connects with two hops:

```bash
# Step 1: SSH into the jump server
ssh engineer@ssh.sechpoint.app

# Step 2: Jump through the tunnel to the customer VM
ssh root@localhost -p 9042
```

Or as a single command:

```bash
ssh -o ProxyJump=engineer@ssh.sechpoint.app root@localhost -p 9042
```

The engineer now has a root shell on the customer VM and can run the deployment
wizard or troubleshoot directly.

## Security Notes

- The `wallarm-tunnel` user has no shell (`/bin/false`) — authentication only succeeds for port forwarding
- The tunnel is outbound-only from the customer VM — no inbound ports opened
- TLS encrypts the connection end-to-end (VM-A → Zoraxy → VM-B)
- Add `Match User wallarm-tunnel` blocks in `sshd_config` to further restrict:
  ```ini
  Match User wallarm-tunnel
      PermitTTY no
      PermitTunnel no
      X11Forwarding no
      AllowAgentForwarding no
      ForceCommand /bin/false
  ```

## Troubleshooting

| Symptom | Check |
|---------|-------|
| "Connection refused" from customer VM | Jump server port 443 reachable? Zoraxy running? |
| "Permission denied" | Tunnel user password/key correct? `~wallarm-tunnel/.ssh/authorized_keys` permissions 600? |
| Tunnel opens but engineer can't reach VM-A | `AllowTcpForwarding yes` in sshd_config? SSH server restarted? |
| Tunnel drops after a few minutes | `ClientAliveInterval` set? Check firewall idle timeout |