mockapi/README.md

# Configurable Mock API with Admin Interface and OAuth2 Provider

A lightweight, configurable mock API application in Python that allows dynamic endpoint management via an admin interface. The API serves customizable responses stored in a SQLite database with template variable support. Includes a full OAuth2 provider for securing endpoints with token-based authentication.

## Features

- **Dynamic Endpoint Configuration**: Create, read, update, and delete API endpoints through a web-based admin interface.
- **Template Variable Support**: Response bodies can include Jinja2 template variables (e.g., `{{ user_id }}`, `{{ timestamp }}`) populated from path parameters, query strings, headers, request body, system variables, and endpoint defaults.
- **Dynamic Route Registration**: Endpoints are registered/unregistered at runtime without restarting the server.
- **Admin Interface**: Secure web UI with session-based authentication for managing endpoints.
- **OAuth2 Provider**: Full OAuth2 implementation supporting authorization code, client credentials, and refresh token grant types.
- **Endpoint‑Level OAuth Protection**: Individual endpoints can require OAuth2 tokens with configurable scopes.
- **Admin OAuth2 Management**: Web UI for managing OAuth clients, tokens, and users.
- **OpenID Connect Discovery**: Standards‑compliant discovery endpoint.
- **Production Ready**: Uses Waitress WSGI server, SQLAlchemy async, and FastAPI with proper error handling and security measures.

## Technology Stack

- **Framework**: FastAPI (with automatic OpenAPI documentation)
- **Server**: Waitress (production WSGI server)
- **Database**: SQLite with SQLAlchemy 2.0 async ORM
- **Templating**: Jinja2 with sandboxed environment
- **Authentication**: Session‑based admin authentication with bcrypt password hashing
- **OAuth2**: JWT‑based tokens with configurable scopes, client validation, and token revocation
- **Frontend**: Bootstrap 5 (CDN) for admin UI

## Project Structure

```
mockapi/
├── main.py                # Development entry point (uvicorn with reload)
├── wsgi.py                # Production WSGI entry point (Waitress/Gunicorn)
├── app/                   # Main application package
│   ├── core/              # Core application setup
│   │   ├── app.py         # FastAPI application factory & lifespan
│   │   ├── config.py      # Configuration (Pydantic Settings)
│   │   ├── database.py    # SQLAlchemy async database setup
│   │   ├── dependencies.py # FastAPI dependencies
│   │   ├── middleware/    # Middleware (authentication, etc.)
│   │   └── observers/     # Observer pattern placeholder
│   ├── modules/           # Feature modules
│   │   ├── admin/         # Admin UI controllers & templates
│   │   ├── endpoints/     # Endpoint management (models, repositories, services, schemas)
│   │   └── oauth2/        # OAuth2 provider (controllers, models, repositories, schemas)
│   ├── static/            # Static assets (CSS, etc.)
│   └── templates/         # Jinja2 HTML templates
│       ├── base.html      # Base layout
│       └── admin/         # Admin interface templates
├── requirements.txt       # Python dependencies
├── example.env            # Example environment variables
├── .env                   # Local environment variables (create from example.env)
├── docs/                  # Project documentation
├── examples/              # API testing collections and examples
├── tests/                 # Test suite
│   ├── test_admin.py      # Admin authentication tests
│   ├── test_endpoint_repository.py
│   ├── test_route_manager_fix.py
│   ├── test_oauth2_controller.py
│   └── integration/       # Integration tests
├── LICENSE                # MIT License
└── README.md              # This file
```

## Installation

1. **Navigate to project directory**:
   ```bash
   cd ~/GitLab/customer-engineering/mockapi
   ```

2. **Create a virtual environment** (recommended):
   ```bash
   python3 -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate
   ```

3. **Install dependencies**:
   ```bash
   pip install -r requirements.txt
   ```

4. **Configure environment variables**:
   ```bash
   cp example.env .env
   # Edit .env with your settings (see example.env for all available variables)
   ```

   Minimal `.env` example:
   ```ini
   DATABASE_URL=sqlite+aiosqlite:///./mockapi.db
   ADMIN_USERNAME=admin
   ADMIN_PASSWORD=admin123           # Change this in production!
   SECRET_KEY=your-secret-key-here   # Change this!
   DEBUG=True                        # Set to False in production
   ```

   The `example.env` file includes additional configuration options for OAuth2, logging, and server settings.

5. **Initialize the database** (tables are created automatically on first run).

**For production deployment**, consider using the automated `install.sh` script which handles virtual environment creation, dependency installation, secure credential generation, and systemd service setup. See [Production Deployment with install.sh](#production-deployment-with-installsh) for details.

## Running the Application

### Development (with auto‑reload)

Make sure your virtual environment is activated:

```bash
source venv/bin/activate  # Linux/macOS
# venv\Scripts\activate   # Windows
```

Then run with auto-reload for development:

```bash
# Using main.py (development entry point)
python main.py

# Or directly with uvicorn
uvicorn main:app --reload --host 0.0.0.0 --port 8000
```

### Production (with Waitress)

For production deployment, use Waitress WSGI server with the provided WSGI adapter (a2wsgi):

```bash
waitress-serve --host=0.0.0.0 --port=8000 --threads=4 wsgi:wsgi_app
```

The server will start on `http://localhost:8000` (or your configured host/port).

**Note:** Waitress is a WSGI server, but FastAPI is an ASGI framework. The `wsgi.py` file uses `a2wsgi` to wrap the ASGI application into a WSGI-compatible interface. Routes are automatically refreshed from the database on server startup.

## Quick Start

### 1. Start the Server

```bash
# Development (auto-reload)
python main.py

# Or directly with uvicorn
uvicorn main:app --reload --host 0.0.0.0 --port 8000

# Production (with Waitress)
waitress-serve --host=0.0.0.0 --port=8000 --threads=4 wsgi:wsgi_app
```

The server will start on `http://localhost:8000` (or your configured host/port).

### 2. Test Basic Functionality

```bash
# Health check
curl http://localhost:8000/health

# Access Swagger UI (auto-generated docs)
open http://localhost:8000/docs
```

### 3. Use API Testing Collections

Ready-to-use API collections are available in the `examples/` directory:

| Collection | Format | Description |
|------------|--------|-------------|
| **Bruno** | `mockapi-collection.bru` | Bruno collection with scripting support |
| **Postman** | `mockapi-postman-collection.json` | Postman Collection v2.1 |

Both collections include:
- Global variables (base URL, credentials, tokens)
- Full OAuth2 flow testing (client credentials, authorization code, refresh)
- Mock endpoint CRUD operations
- Admin authentication
- Protected endpoint examples

**Setup:**
```bash
# Create test OAuth client
./examples/setup.sh

# Or manually
python examples/setup-test-client.py
```

**Import:**
- **Bruno**: Drag and drop `.bru` file or use "Import Collection"
- **Postman**: Use "Import" button and select JSON file

See [examples/README.md](examples/README.md) for detailed usage.

### 4. Basic cURL Examples (Quick Reference)

```bash
# Admin login (sets session cookie)
curl -c cookies.txt -X POST http://localhost:8000/admin/login \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "username=admin&password=admin123"

# Create a simple mock endpoint
curl -b cookies.txt -X POST http://localhost:8000/admin/endpoints \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "route=/api/test&method=GET&response_body={\"message\":\"test\"}&response_code=200&content_type=application/json&is_active=true"

# Call the endpoint
curl http://localhost:8000/api/test

# OAuth2 client credentials grant (using test client)
curl -X POST http://localhost:8000/oauth/token \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "grant_type=client_credentials&client_id=test_client&client_secret=test_secret&scope=api:read"
```

For comprehensive testing with all OAuth2 flows and examples, use the provided API collections.



## OAuth2 Authentication

The application includes a full OAuth2 provider implementing RFC 6749 and OpenID Connect Discovery.

### Supported Grant Types

- **Authorization Code**: For web applications with server‑side components.
- **Client Credentials**: For machine‑to‑machine communication.
- **Refresh Token**: To obtain new access tokens without user interaction.

### Endpoints

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/oauth/authorize` | GET, POST | Authorization endpoint (interactive user consent) |
| `/oauth/token` | POST | Token issuance and refresh |
| `/oauth/userinfo` | GET | OpenID Connect user claims |
| `/oauth/introspect` | POST | Token introspection (RFC 7662) |
| `/oauth/revoke` | POST | Token revocation (RFC 7009) |
| `/.well‑known/openid‑configuration` | GET | OpenID Connect discovery document |

### Scope Management

Default scopes include:
- `openid` – OpenID Connect support
- `profile` – Basic profile information
- `email` – Email address claim
- `api:read` – Read access to protected endpoints
- `api:write` – Write access to protected endpoints

### Admin OAuth2 Management

Access OAuth2 management via the admin interface navigation (OAuth dropdown):

- **Clients**: `http://localhost:8000/admin/oauth/clients` – Register and manage OAuth clients
- **Tokens**: `http://localhost:8000/admin/oauth/tokens` – View and revoke issued tokens  
- **Users**: `http://localhost:8000/admin/oauth/users` – Manage OAuth user records

**Admin UI Navigation**: The admin interface now includes an "OAuth" dropdown menu in the top navigation bar with direct access to Clients, Tokens, and Users management pages.

## Production Deployment Considerations

### 1. **Environment Configuration**
- Set `DEBUG=False` in production
- Use strong, unique values for `ADMIN_PASSWORD` and `SECRET_KEY`
- Consider using a more robust database (PostgreSQL) by changing `DATABASE_URL`
- Store sensitive values in environment variables or a secrets manager

### 2. **Process Management**
Use a process manager like systemd (Linux) or Supervisor to keep the application running:

**Example systemd service (`/etc/systemd/system/mockapi.service`)**:
```ini
[Unit]
Description=Mock API Service
After=network.target

[Service]
User=www-data
Group=www-data
WorkingDirectory=/path/to/mockapi
Environment="PATH=/path/to/mockapi/venv/bin"
EnvironmentFile=/path/to/mockapi/.env
# HOST and PORT are read from .env file at runtime
ExecStart=/path/to/mockapi/venv/bin/waitress-serve --host=$HOST --port=$PORT wsgi:wsgi_app
Restart=always
RestartSec=10

# Security hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ReadWritePaths=/path/to/mockapi

[Install]
WantedBy=multi-user.target
```

**Note**: The `install.sh` script automatically creates an optimized systemd service file with dynamic port configuration, security hardening, and proper environment variable loading from `.env`.

### 3. **Production Deployment with install.sh**
For production deployments, use the provided `install.sh` script which handles all setup automatically:

```bash
cd /opt/mockapi
sudo ./install.sh
```

The install script will:
1. Check Python version and handle Python 3.13+ compatibility
2. Create Python virtual environment with required dependencies
3. Generate secure credentials (admin password, secret key)
4. Create `.env` configuration file with all required variables
5. Initialize the database
6. Create and configure systemd service with dynamic port configuration
7. Start and enable the service

**Key Features**:
- **Python 3.13.5+ compatibility**: Automatically handles SQLAlchemy 2.0.27+ requirements
- **Dynamic port configuration**: Service reads HOST/PORT from `.env` file at runtime
- **Secure credential generation**: Random admin password and secret key
- **Missing variable protection**: Automatically adds required environment variables if missing

### 4. **Changing Port Configuration**
Port configuration is now simplified - all settings are read from the `.env` file:

1. **Edit the `.env` file**:
   ```bash
   sudo nano /opt/mockapi/.env
   ```
   Update these lines:
   ```ini
   PORT=8080                # Change to your desired port
   HOST=0.0.0.0             # Or change host if needed
   OAUTH2_ISSUER=http://localhost:8080  # Update to match new port
   ```

2. **Restart the service**:
   ```bash
   sudo systemctl restart mockapi
   ```

3. **Verify the change**:
   ```bash
   curl http://localhost:8080/health
   ```

**Important Notes**:
- The systemd service automatically reads HOST and PORT from `.env` file at runtime
- No need to edit the service file when changing ports
- After changing port, update any reverse proxy configurations
- OAuth2 clients may need reconfiguration if issuer URL changes

### 5. **Reverse Proxy (Recommended)**
Use Nginx or Apache as a reverse proxy for SSL termination, load balancing, and static file serving:

**Example Nginx configuration**:
```nginx
server {
    listen 80;
    server_name api.yourdomain.com;
    
    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
```

### 6. **Database Backups**
For SQLite, regularly backup the `mockapi.db` file. For production, consider migrating to PostgreSQL.

## Usage

### 1. Access the Admin Interface
- Open `http://localhost:8000/admin/login`
- Log in with the credentials set in `.env` (default: `admin` / `admin123`)

### 2. Create a Mock Endpoint
1. Navigate to **Endpoints** → **Create New**.
2. Fill in the form:
   - **Route**: `/api/greeting/{name}` (supports path parameters)
   - **Method**: GET
   - **Response Body**: `{ "message": "Hello, {{ name }}!" }`
   - **Response Code**: 200
   - **Content-Type**: `application/json`
   - **Variables**: `{ "server": "mock-api" }` (optional defaults)
3. Click **Create**.

### 3. Call the Mock Endpoint
```bash
curl http://localhost:8000/api/greeting/World
```
Response:
```json
{ "message": "Hello, World!" }
```

### 4. Template Variables
The following variable sources are available in response templates:

| Source | Example variable | Usage in template |
|--------|------------------|-------------------|
| Path parameters | `{{ name }}` | `/users/{id}` → `{{ id }}` |
| Query parameters | `{{ query.page }}` | `?page=1` → `{{ page }}` |
| Request headers | `{{ header.authorization }}` | `Authorization: Bearer token` |
| Request body | `{{ body.user.email }}` | JSON request body |
| System variables | `{{ timestamp }}`, `{{ request_id }}` | Automatically injected |
| Endpoint defaults | `{{ server }}` | Defined in endpoint variables |

### 5. Admin Functions
- **List endpoints** with pagination and filtering
- **Edit** existing endpoints (changes take effect immediately)
- **Activate/deactivate** endpoints without deletion
- **Delete** endpoints (removes route)
- **Dashboard** with statistics (total endpoints, active routes, etc.)
- **OAuth2 management** – clients, tokens, users

## Security Considerations

- **Admin authentication**: Uses bcrypt password hashing. Store a strong password hash in production.
- **Session management**: Signed cookies with configurable secret key.
- **Template sandboxing**: Jinja2 environment restricted with `SandboxedEnvironment` and `StrictUndefined`.
- **Request size limits**: Maximum body size of 1MB to prevent DoS.
- **Route validation**: Prevents path traversal (`..`) and other unsafe patterns.
- **SQL injection protection**: All queries use SQLAlchemy ORM.
- **OAuth2 security**: Client secret hashing, token revocation, scope validation, secure token storage.

## Configuration Options

See `config.py` for all available settings. Key environment variables:

| Variable | Default | Description |
|----------|---------|-------------|
| `DATABASE_URL` | `sqlite+aiosqlite:///./mockapi.db` | SQLAlchemy database URL |
| `ADMIN_USERNAME` | `admin` | Admin login username |
| `ADMIN_PASSWORD` | `admin123` | Admin login password (plaintext) |
| `SECRET_KEY` | `your‑secret‑key‑here‑change‑me` | Session signing secret |
| `DEBUG` | `False` | Enable debug mode (more logging, relaxed validation) |
| `OAUTH2_ISSUER` | `http://localhost:8000` | OAuth2 issuer URL for discovery |
| `OAUTH2_ACCESS_TOKEN_EXPIRE_MINUTES` | `30` | Access token lifetime |
| `OAUTH2_REFRESH_TOKEN_EXPIRE_DAYS` | `7` | Refresh token lifetime |
| `OAUTH2_SUPPORTED_SCOPES` | `["openid","profile","email","api:read","api:write"]` | Available OAuth2 scopes |

**Warning**: In production (`DEBUG=False`), the default `ADMIN_PASSWORD` and `SECRET_KEY` will cause validation errors. You must set unique values via environment variables.

## API Documentation

FastAPI automatically provides OpenAPI documentation at:
- Swagger UI: `http://localhost:8000/docs`
- ReDoc: `http://localhost:8000/redoc`

The root URL (/) automatically redirects to the Swagger documentation at /docs.

The dynamic mock endpoints are not listed in the OpenAPI schema (they are registered at runtime).

## Development & Testing

## API Testing Collections

Ready-to-use API collections are available in the `examples/` directory:

### Bruno Collection (`mockapi-collection.bru`)
- **Format**: Bruno native format (`.bru`)
- **Features**: Scripting support, environment variables, folder organization
- **Import**: Drag and drop into Bruno or use "Import Collection"

### Postman Collection (`mockapi-postman-collection.json`)
- **Format**: Postman Collection v2.1
- **Features**: Pre-request scripts, tests, environment variables
- **Import**: Import into Postman via "Import" button

**Quick setup:**
```bash
# Create test OAuth client and view instructions
./examples/setup.sh

# Or import directly:
# examples/mockapi-collection.bru (Bruno)
# examples/mockapi-postman-collection.json (Postman)
```

Both collections include:
- Global variables for base URL and credentials
- Pre-configured requests for all endpoints
- OAuth2 flow examples (client credentials, authorization code, refresh)
- Admin authentication
- Mock endpoint creation and testing
- Protected endpoint examples

See [examples/README.md](examples/README.md) for detailed usage instructions.



### Running Tests

Run tests with pytest:
```bash
pytest tests/
```

The test suite includes:
- Unit tests for repository and service layers
- Integration tests for admin authentication
- Template rendering tests
- OAuth2 unit and integration tests (21+ tests)

### Example Integration Test

A ready‑to‑run integration test demonstrates the core functionality:

```bash
# Make the script executable (Linux/macOS)
chmod +x run_example.sh

# Run the example
./run_example.sh
```

Or directly with Python:
```bash
python example_usage.py
```

The example script will:
1. Start the FastAPI app (via TestClient)
2. Log in as admin
3. Create a mock endpoint with template variables
4. Call the endpoint and verify the response
5. Report success or failure

This is a great way to verify that the API is working correctly after installation.

## Troubleshooting

### Common Issues

1. **"no such table: endpoints" error**
   - The database hasn't been initialized
   - Restart the application - tables are created on first startup
   - Or run `python -c "from database import init_db; import asyncio; asyncio.run(init_db())"`

2. **Login fails even with correct credentials**
   - Check that `DEBUG=True` is set in `.env` (or provide unique credentials)
   - The default credentials only work when `DEBUG=True`
   - In production, you must set unique `ADMIN_PASSWORD` and `SECRET_KEY`

3. **Routes not being registered**
   - Check that the endpoint is marked as active (`is_active=True`)
   - Refresh the page - routes are registered immediately after creation
   - Check application logs for errors

4. **Template variables not rendering**
   - Ensure you're using double curly braces: `{{ variable }}`
   - Check variable names match the context (use path_, query_, header_ prefixes as needed)
   - View the rendered template in the admin edit form preview

5. **OAuth2 token validation fails**
   - Verify the token hasn't expired
   - Check that the client is active
   - Confirm the token has required scopes for the endpoint
   - Ensure the token hasn't been revoked

### Logging
Enable debug logging by setting `DEBUG=True` in `.env`. Check the console output for detailed error messages.

## Limitations & Future Enhancements

- **Current limitations**:
  - SQLite only (but can be extended to PostgreSQL via `DATABASE_URL`)
  - Single admin user (no multi‑user support)
  - No request logging/history
  - OAuth2 protection fields (requires_oauth, oauth_scopes) for endpoints not exposed in admin UI (though OAuth client/token/user management is available via navigation dropdown)
  - OAuth2 user authentication uses placeholder user IDs (integration with external identity providers pending)

- **Possible extensions**:
  - Import/export endpoints as JSON/YAML
  - Request logging and analytics
  - WebSocket notifications for admin actions
  - Multiple admin users with roles
  - Rate limiting per endpoint
  - CORS configuration
  - PKCE support for public OAuth2 clients
  - Integration with external identity providers (SAML, LDAP)

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## Acknowledgments

- Built with [FastAPI](https://fastapi.tiangolo.com/), [SQLAlchemy](https://www.sqlalchemy.org/), and [Jinja2](https://jinja.palletsprojects.com/).
- Admin UI uses [Bootstrap 5](https://getbootstrap.com/) via CDN.
- OAuth2 implementation follows RFC 6749, RFC 7662, and OpenID Connect standards.