Production Security Hardening: From 5/100 to Battle-Tested

Consolidated from real security audit and hardening sessions spanning April–May 2026. All secrets, IPs, and internal endpoints replaced with placeholders.

Security Focus 1: Tradeoffs that matter in production for this workload (Production Security Hardening)

This article documents the complete security hardening journey of a production web application — from an initial security header score of 5/100 through SQL injection pentesting, XSS protection, rate limiting, WAF deployment, and OWASP compliance verification.

---

Editorial review note for Production Security Hardening

This section was reviewed by a human editor to keep the recommendations actionable and technically grounded. Reviewed by: Med Amine Mahmoud. Last editorial review: 2026-05-26T16:10:01Z.

Security Focus 3: Runtime checks you should not skip for production readiness (Production Security Hardening)

• Security headers at nginx, backend, and SSR layers
• HTTPS with TLS 1.3 (ACM + ALB HTTPS listener)
• HTTP → HTTPS redirect (301)
• HSTS with preload directive
• CSP tailored to your scripts (avoid unsafe-eval/inline)
• Rate limiting with real client IP (X-Forwarded-For)
• CSRF protection (X-Requested-With or token-based)
• Request body size limits
• Input validation (Pydantic schemas + regex)
• XSS prevention (html.escape + framework auto-escaping)
• ORM parameterized queries (no raw SQL)
• Non-root container users
• Direct ALB access blocked (host-based rules)
• WAF with OWASP CRS (ModSecurity or AWS WAF)
• ALB access logs enabled (S3)
• Auto-scaling for DDoS resilience
• Deployment circuit breaker enabled

Security Focus 4: How this maps to real exam objectives for sustained reliability (Production Security Hardening)

---

Security Focus 5: Failure modes and quick prevention for secure delivery (Production Security Hardening)

Attack 1: Directory Enumeration (Critical Incident)

Two IPs used ffuf and feroxbuster (directory enumeration tools):

• Attacker A (repeat attacker) — Also attempted login brute-force
• Attacker B — ffuf + feroxbuster automated scanning

Impact: Elevated 5XX errors due to resource starvation (all containers share resources in single task)

Response:

• Identified via ALB access logs (S3)
• Created IP blocking automation script
• Documented attack patterns for future detection

Attack 2: WordPress Vulnerability Scanners

Automated scanners probing for WordPress-specific paths (/wp-admin, /wp-login.php, /xmlrpc.php):

• Source: Google Cloud and DigitalOcean IPs
• All returned 404 (paths don't exist)
• Zero legitimate user impact

Attack 3: Traffic Spike (Benign Crawler)

1,187 HEAD requests in 4.5 minutes from a residential IP (Tunisia):

• Empty User-Agent
• 1,186 unique URLs (full sitemap crawl)
• All returned 200
• Pattern: SEO monitoring tool, not malicious
• Alarm self-resolved in 3 minutes

---

Security Focus 6: A cleaner way to operate this pattern for predictable operations (Production Security Hardening)

Architecture

ModSecurity deployed as a sidecar in the ECS task:

ALB → nginx + ModSecurity CRS → frontend/backend containers

Configuration

• OWASP CRS v4 — 853 rules loaded
• Detection-Only mode initially (log but don't block)
• Paranoia Level 1 — balanced false-positive rate
• Blocks: Known vulnerability scanners, bot user agents, common attack patterns

Validation Results

• Bot scanner requests: 403 Forbidden
• Legitimate traffic: 200 OK (no false positives)
• CRS rule count: 853 active rules

---

Security Focus 7: What to automate first for exam and field confidence (Production Security Hardening)

All fixes verified on live production:

---

Security Focus 8: How to keep this maintainable at scale for cleaner ownership (Production Security Hardening)

Setup

A full sqlmap pentest infrastructure was created:

• Docker container with sqlmap (Python 3.12-slim)
• Isolated test stack (db, backend, frontend, nginx, sqlmap)
• 18 automated test scenarios across 6 categories covering all 49+ API endpoints

Results: 0/18 Vulnerabilities

Why It Passed

1. SQLAlchemy ORM — All queries use parameterized statements
2. Pydantic validation — Request schemas reject malformed input
3. Regex validation — Exam codes validated with ^[A-Z0-9-]+$
4. Integer coercion — ID parameters auto-cast (non-integers return 422)
5. Rate limiting — Throttles automated tools

---

Security Focus 9: Pragmatic guardrails for day two ops for measurable outcomes (Production Security Hardening)

B1: Admin Endpoint Protection (CRITICAL)

Before: Admin seed endpoint accessible without authentication After: Protected behind require_admin() decorator + CSRF validation

@router.post("/admin/seed")
async def seed_data(request: Request):
# Now requires: valid session + admin role + X-Requested-With header
admin = require_admin(request)
...

B2: Answer Exposure Prevention (HIGH)

Before: /api/questions/random returned correct and explanation fields After: Response only includes correct_count — actual answers never sent to client during quiz

class QuestionResponse(BaseModel):
text: str
options: List[Option]
# correct and explanation EXCLUDED from response model

B3: Rate Limiting (HIGH)

Before: No rate limiting — unlimited requests After: Global rate limiting with SlowAPI

from slowapi import Limiter
from slowapi.util import get_remote_address

limiter = Limiter(key_func=get_remote_address)

@router.post("/forum/threads")
@limiter.limit("100/minute")
async def create_thread(request: Request, ...):
...

Critical Bug Found: get_remote_address was returning nginx loopback (127.0.0.1) instead of the real client IP. Fixed by reading X-Forwarded-For header:

def get_real_ip(request: Request) -> str:
forwarded = request.headers.get("X-Forwarded-For")
if forwarded:
return forwarded.split(",")[0].strip()
return request.client.host

B5: Request Body Size Limit (HIGH)

@app.middleware("http")
async def limit_body_size(request, call_next):
content_length = request.headers.get("content-length")
if content_length and int(content_length) > 524288: # 512 KB
return JSONResponse(status_code=413, content={"detail": "Request body too large. Maximum size is 512 KB."})
return await call_next(request)

F1: Forum XSS Prevention (HIGH)

Backend: html.escape() on all user-provided text before storage Frontend: Angular's {{ }} interpolation auto-escapes output (no [innerHTML])

F2: CSRF Protection (HIGH)

Global middleware requiring X-Requested-With header on all state-changing requests:

@app.middleware("http")
async def csrf_protection(request, call_next):
if request.method in ("POST", "PUT", "DELETE", "PATCH"):
if not request.headers.get("X-Requested-With"):
return JSONResponse(status_code=403, content={"detail": "CSRF validation failed"})
return await call_next(request)

D3: Non-Root Containers (CRITICAL)

# Both frontend and backend Dockerfiles
RUN adduser --disabled-password --gecos '' appuser
USER appuser

---

Security Focus 10: Risk controls worth enforcing early for fewer incident surprises (Production Security Hardening)

Initial State

A web security scanner rated the application 5/100. The only security measure was basic HTTPS.

Implementation: Defense-in-Depth Headers

Security headers were added at three layers to ensure defense-in-depth:

# nginx.conf
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
add_header Content-Security-Policy "default-src 'self'; script-src 'self' https://pagead2.googlesyndication.com; style-src 'self' 'unsafe-inline'; img-src 'self' data: https:; font-src 'self'; connect-src 'self' https://pagead2.googlesyndication.com; frame-src https://googleads.g.doubleclick.net;" always;
add_header X-Frame-Options "DENY" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
add_header Permissions-Policy "camera=(), microphone=(), geolocation=(), payment=()" always;
add_header X-XSS-Protection "1; mode=block" always;

# FastAPI middleware (backup layer)
@app.middleware("http")
async def security_headers(request, call_next):
response = await call_next(request)
response.headers["X-Content-Type-Options"] = "nosniff"
response.headers["X-Frame-Options"] = "DENY"
response.headers["Referrer-Policy"] = "strict-origin-when-cross-origin"
return response

// Angular SSR server.ts (third layer)
server.use((req, res, next) => {
res.setHeader('X-Content-Type-Options', 'nosniff');
res.setHeader('X-Frame-Options', 'DENY');
next();
});

Critical Lesson: CSP upgrade-insecure-requests

Adding upgrade-insecure-requests to the Content-Security-Policy caused the entire page to go blank when served over HTTP. The directive forces browsers to upgrade all subresource loads to HTTPS. Since the ALB initially had no TLS, all JS/CSS failed to load silently.

Rule: Never add upgrade-insecure-requests until HTTPS is confirmed working end-to-end.

---