Tuesday, 5 May 2026

Implementation of Autonomy Intelligence Features

v2.0 — Self-Learning Pipeline

These features transform the tool from a reactive code generator into a self-improving autonomous system. Each capability adds a new dimension of machine learning to the pipeline, reducing human intervention and improving output quality over time.

1. Closed-Loop Outcome Scoring
Foundational Layer
QA Pipeline Score + Findings Outcome Card Generator Outcome DB Tech Stack Score Trajectory Lessons Learned Stage 2 Analysis "What worked before?" Continuous Learning Loop Live Metrics Cards: 0 Success: 0%
Technical Architecture
  • Entity: OutcomeCard — JPA entity in H2/PostgreSQL
  • Service: OutcomeScoringService — generates structured outcome cards after QA
  • Trigger: PRCreatedEvent listener auto-generates cards
  • Storage: H2 table outcome_cards with indexed lookups by tech stack
  • AI Integration: Bedrock generates "lessons learned" via meta-analysis prompt
  • Retrieval: During Stage 2, top-N matching cards injected into analysis prompt
Business Impact
  • Reduces analysis time — past successes/failures guide decisions instantly
  • Avoids repeated failures — if an approach failed before, system warns proactively
  • Quantifies ROI — tracks time-to-complete and fix iterations across all requirements
  • Enables trend analysis — success rates by tech stack reveal strengths/weaknesses
  • Transforms KB from "here's similar code" to "here's what worked and what didn't"
2. Prompt Evolution via Meta-Learning
Adaptive Intelligence
21 Prompt Templates system-codegen.txt... QA Outcomes Scores + Findings Meta Analysis Amendments ALWAYS/NEVER rules Staged for Review Human Review Approved amendments feed back into prompts Blind Spot Detection "CSRF errors in 70% of runs despite prompt rule"
Technical Architecture
  • Service: PromptEvolutionService — tracks prompt→outcome correlations
  • Blind Spot Detection: Categories appearing in >50% of runs flagged as ineffective rules
  • Amendment Generation: Bedrock meta-prompt generates ALWAYS/NEVER rules
  • Safety Gate: All amendments are STAGED for human review before activation
  • Metrics: Per-template avg QA score, avg iterations, total findings tracked
  • Trigger: Auto-runs after every N requirements (configurable threshold)
Business Impact
  • Self-improving prompts — system gets smarter with every requirement processed
  • Addresses "stale rules" — detects when prompt rules aren't preventing errors
  • Reduces QA iterations — better prompts → fewer errors → less fix loop work
  • Human-in-the-loop safety — amendments never go live without approval
  • Measurable improvement — before/after metrics prove prompt evolution value
4. Proactive Defect Prediction
Pre-Generation Intelligence
Risk Scanner Risk Assessment ๐Ÿ”ด CSRF → HIGH (80%) ๐ŸŸก API_TYPO → MEDIUM ๐ŸŸก MISSING_DEP → MEDIUM ๐ŸŸข TEMPLATE → LOW Code Generation + Weighted Pitfalls + Guard Code + High-Risk Flags → Fewer Defects! QA Score
Technical Architecture
  • Service: DefectPredictionService — pre-generation risk analysis
  • AI Prompt: Bedrock predicts error categories by requirement + tech stack
  • Pattern Matching: Cross-references full Error Pattern Library for weighted prevention
  • Guard Code: Auto-generates defensive code snippets for HIGH-risk areas
  • File Risk Scoring: Identifies historically problematic files for extra scrutiny
  • Integration Point: Runs in CodeGenerationService before story loop starts
Business Impact
  • Shift-left quality — catches likely defects BEFORE code is generated
  • Reduces fix iterations — predicted problems are prevented at source
  • Smarter resource allocation — high-risk requirements get more defensive code
  • Quantifiable ROI — track defect prediction accuracy vs actual QA findings
  • Cost savings — fewer LLM calls for fix loops = lower Bedrock costs
5. Autonomous Self-Resolution (Stage 2)
Human Touchpoint Reduction
Analysis Engine Generates 5 Q's ๐Ÿค– Self-Resolution Engine 1. Query KB 2. Past Q&A 3. Repo Context 4. AI Reasoning Confidence ≥ 0.7? ✓ Resolved Auto-Answered 3/5 Questions ✗ Escalate User Review Only 2 Questions Impact Comparison Before: 5 questions → user ~2 day wait After: 2 questions → user 60% faster ↓60%
Technical Architecture
  • Service: SelfResolutionService — multi-strategy question resolver
  • 4 Resolution Strategies: KB search, past Q&A matching, repo context inference, AI reasoning
  • Confidence Scoring: Each resolution gets a 0.0–1.0 confidence; threshold = 0.7
  • Integration: Hooks into ProposalService.runAnalysis() before user escalation
  • Outcome Cards: Uses historical outcomes for additional reasoning context
  • Safety: All auto-resolutions are marked with source + confidence for audit trail
Business Impact
  • 60% fewer user interruptions — most questions answered autonomously
  • Faster pipeline throughput — no wait for human Q&A responses
  • Better user experience — users confirm assumptions vs answering from scratch
  • Knowledge compounds — each answered Q becomes future resolution context
  • Audit trail preserved — all auto-resolutions logged with confidence + source
6. Fix Strategy Memory (QA Intelligence)
Fix Loop Acceleration
QA Finding "Missing CSRF" protection ๐Ÿง  Strategy Memory CSRF → "Add @EnableCSRF + th:action on forms" ✓ 8/10 success rate Fix Prompt + Proven Strategy + Context → Converges faster! Fixed! 1 iteration vs 3 Score: 8.5 → PASS Success recorded → strategy strengthened Live Metrics Strategies: 0 Success: 0%
Technical Architecture
  • Entity: FixStrategy — JPA entity tracking fix approaches
  • Service: FixStrategyService — records successes/failures, retrieves best strategies
  • Integration: QaFixLoopService queries strategy memory before each fix attempt
  • Matching: By finding category + framework, with fallback to category-only
  • Scoring: Tracks success rate + avg score improvement per strategy
  • Prompt Injection: "FIX STRATEGY MEMORY" section with proven patterns
Business Impact
  • Faster convergence — fix loop resolves issues in 1 iteration vs 2-3
  • Reduced Bedrock costs — fewer fix iterations = fewer API calls
  • Knowledge retention — proven fixes persist even as prompts evolve
  • Compounding intelligence — each fix makes future fixes faster
  • Measurable: Track avg iterations before vs after strategy memory activation
7. Cross-Requirement Dependency Graph
Conflict Prevention
REQ-A (Auth) Modifying AuthService REQ-B (Roles) Modifying AuthService AuthService.java ⚠️ CONFLICT Modified by 2 reqs ๐Ÿ›ก️ Dependency Graph File Registry tracks all in-flight modifications Conflict detected! Resolution Strategy Option 1: Sequence work Option 2: Inject pending changes as context → Compatible code gen! Live Reqs: 0 Files: 0 ⚠ 0
Technical Architecture
  • Entity: FileRegistryEntry — tracks file→requirement→branch mappings
  • Service: DependencyGraphService — conflict detection + resolution
  • Registration: Auto-registers files at code generation start
  • Detection: Before generating, queries for overlapping in-flight modifications
  • Resolution: Injects conflict context into generation prompt for compatible output
  • Lifecycle: Files marked COMPLETED after PR merge
Business Impact
  • Zero merge conflicts — detected at generation time, not PR time
  • Parallel work enabled — multiple requirements can modify overlapping areas safely
  • DevOps efficiency — no manual conflict resolution, no blocked PRs
  • Visibility — dashboard shows all in-flight modifications at a glance
  • Scales with team size — more concurrent requirements handled safely
Autonomy Maturity Model
L1: Reactive L2: Proactive L3: Predictive L4: Self-Learning ← XXXXX v2.0 (with these features) XXXXX v1.x
Feature Autonomy Dimension Human Reduction Learning Type Status
Outcome Scoring Decision Intelligence Reduces analysis guesswork by 40% Historical Pattern Active
Prompt Evolution Self-Improvement Auto-discovers blind spots Meta-Learning Active
Defect Prediction Preventive Quality 30% fewer QA fix iterations Predictive Risk Active
Self-Resolution Autonomous Analysis 60% fewer clarification questions Multi-Strategy Reasoning Active
Fix Strategy Memory QA Intelligence Fix loop converges 2x faster Strategy Reinforcement Active
Dependency Graph Coordination Intelligence Zero merge conflicts Graph Awareness Active

Friday, 1 May 2026

Autonomous QA — Architecture Diagram

⚡ Stage 6: Autonomous QA — Architecture

5-Layer Quality Gate Pipeline · Bedrock-Powered · Fully Autonomous

In my last two blogs we build the autonomus SDLC AI powered system , then we build the Auto-healing , and In this blow we will go through the desgin part of Autonomous QA — Architecture

STAGE 6 — AUTONOMOUS QA PIPELINE ๐Ÿ”” TRIGGER: PRCreatedEvent QaOrchestrationService listens → polls GitHub Pages LAYER 1 Structure Check JAVA ๐Ÿ“„ index.html exists ๐Ÿ”— No broken refs (JS/CSS/img) ๐Ÿ“ Valid file structure ๐Ÿ“ HTML5 valid ⚡ HTTP 200 OK Engine: StructureCheckService.java — Pure Java HttpClient + regex scanning — No AI — Deterministic rules PASS ✓ LAYER 2 Security Audit HYBRID ๐Ÿ”’ OWASP A01–A10 scan ๐Ÿ›ก️ XSS detection (inline JS) ๐Ÿ”‘ Credential exposure ๐Ÿšซ Open redirect ๐Ÿ“ฆ CSP headers Engine: SecurityAuditService.java — Static regex rules (fast) + Bedrock deep analysis (qa-security-review.txt) — OWASP Top 10 mapping ๐Ÿ›ก️ PASS ✓ LAYER 3 Functional E2E Tests BEDROCK ๐Ÿงช User flow simulation ๐Ÿ“‹ Form submit validation ๐Ÿ”€ Navigation paths ๐Ÿ” Auth flow check ⚠️ Error handling Engine: FunctionalTestService.java — Bedrock-simulated (qa-functional-test.txt) — AI reads HTML+JS, traces user journeys, reports failures ๐Ÿง  PASS ✓ LAYER 4 Accessibility Audit HYBRID ♿ WCAG 2.1 AA compliance ๐Ÿท️ ARIA labels & roles ๐ŸŽจ Contrast ratio ≥ 4.5:1 ⌨️ Keyboard nav ๐Ÿ“ฑ Responsive Engine: AccessibilityAuditService.java — Java rules (contrast calc, ARIA check) + Bedrock deep review (qa-accessibility-review.txt) ๐Ÿ‘️ PASS ✓ LAYER 5 Performance Audit BEDROCK ⚡ Asset size analysis ๐Ÿ–ผ️ Image optimization ๐Ÿ“ฆ Render-blocking resources ๐Ÿ’พ Caching headers ๐Ÿš€ Load strategy Engine: PerformanceAuditService.java — Bedrock-simulated (qa-performance-review.txt) — AI analyzes asset graph + render path ๐Ÿš€ ๐Ÿ“Š QA Report (HTML + JSON) QaReportBuilder → DB + /api/qa/{reqId} ๐Ÿ“ PR Description Updated GitHub API PATCH → QA badge + findings ๐Ÿ“ก SSE: QA_COMPLETE Real-time dashboard update via SSE SEQUENTIAL EXECUTION → Score P/F 0–10 0–10 0–10 0–10

๐Ÿ”” Trigger: PRCreatedEvent

The entire QA pipeline is event-driven. When CodeGenerationService creates a pull request on GitHub, Spring publishes a PRCreatedEvent. The QaOrchestrationService listens for this event via @EventListener and kicks off the QA pipeline asynchronously (@Async).

Activation Sequence

StepActionDetail
1Event receivedPRCreatedEvent(reqId, prUrl, pagesUrl) captured by listener
2Poll GitHub PagesHTTP GET to pagesUrl every 15s, up to 3 min timeout, waiting for HTTP 200
3Fetch all pagesHttpClient crawls all HTML/JS/CSS from the deployed Pages site
4Execute 5 layersSequential execution — each layer receives the fetched content + previous layer results
5Aggregate & reportQaReportBuilder compiles findings → DB save → PR update → SSE broadcast

Why Event-Driven?

  • Decoupled — Code generation doesn't wait for QA; QA runs independently
  • Non-blocking — User sees PR created immediately; QA results stream in via SSE
  • Retry-safe — If Pages isn't ready, polling handles the delay gracefully

Layer 1: Structure Check Pure Java

The fastest, cheapest gate. Pure deterministic Java rules — no AI, no network calls to Bedrock. Catches deployment-breaking issues in milliseconds.

What It Checks

CheckRuleSeverity on Fail
Entry point existsindex.html must exist at repo rootCRITICAL
Broken referencesEvery <script src>, <link href>, <img src> must resolve to existing fileCRITICAL
File structureAll HTML files reference-able from root; no orphaned pagesHIGH
HTML5 validity<!DOCTYPE html>, <html lang>, <meta charset> presentMEDIUM
HTTP 200GitHub Pages URL returns 200 statusCRITICAL

Engine Details

  • Service: StructureCheckService.java
  • Technique: Java HttpClient for live URL checks; regex-based HTML parsing for reference extraction
  • Scoring: Pass/Fail (binary) — any CRITICAL finding = layer fails, pipeline short-circuits with report
  • Performance: Completes in <2 seconds typically
  • Why first? If the site doesn't load or has broken refs, deeper analysis is pointless

Layer 2: Security Audit Hybrid

Two-pass security analysis modeled on OWASP Top 10. First pass: fast static regex rules catch known patterns. Second pass: Bedrock deep analysis for nuanced vulnerabilities that pattern matching misses.

Pass 1: Static Rules (Java)

RulePatternMaps to OWASP
Inline JavaScript detectiononclick=, javascript:, eval(A03: Injection / XSS
Credential exposurepassword in URL params, hardcoded tokens, localStorage for secretsA07: Auth Failures
Form action validationForms with method="GET" containing password fieldsA04: Insecure Design
Open redirectUnvalidated window.location assignments from URL paramsA01: Broken Access
Missing security headersNo CSP meta tag, no X-Frame-OptionsA05: Security Misconfig

Pass 2: Bedrock Deep Analysis

  • Prompt: qa-security-review.txt — sends full HTML+JS source to Bedrock
  • AI analyzes: Authentication flow logic, session management, data sanitization patterns, DOM manipulation safety, third-party script risks
  • Output: JSON array of findings with severity, owaspCategory, location, remediation

Scoring

  • Security Score: 0–10 scale (10 = no findings)
  • Each CRITICAL finding: −3 points. HIGH: −2. MEDIUM: −1. LOW: −0.5
  • Gate threshold: Advisory only (no blocking) — but CRITICAL findings highlighted in PR

Layer 3: Functional E2E Tests Bedrock AI

Since the generated apps are static GitHub Pages sites (HTML/CSS/JS only), traditional browser automation (Selenium/Playwright) is overkill. Instead, Bedrock AI reads the complete source code and mentally simulates user journeys — tracing event handlers, form submissions, navigation flows, and state management.

What Bedrock Simulates

JourneyWhat AI TracesExpected Behavior
Login flowForm submit handler → validation → redirect → session storageInvalid creds show error; valid creds redirect to home
NavigationAnchor hrefs, window.location, back/forward logicAll links navigate to existing pages; no dead ends
CRUD operationsDOM manipulation, localStorage read/write, event chainsAdd/edit/delete reflect in UI; data persists across page loads
Auth guardssessionStorage/localStorage checks on page loadUnauthenticated users redirected to login
Error handlingTry/catch blocks, error display elements, edge casesGraceful degradation; user-visible messages

Why Bedrock-Simulated vs. Real Browser?

  • No infrastructure: No Selenium grid, no headless Chrome, no Docker containers
  • Deeper analysis: AI understands intent, not just DOM state — catches logic errors a click-test would miss
  • Cost-effective: One Bedrock invocation covers dozens of simulated journeys
  • Trade-off: Cannot catch rendering bugs or CSS layout issues (Layer 4 partially covers this)

Scoring

  • Score: 0–10 (10 = all journeys pass)
  • AI returns structured JSON: { journey, steps[], result: "pass"|"fail", issue?, remediation? }

Layer 4: Accessibility Audit Hybrid

Ensures WCAG 2.1 Level AA compliance through a combination of deterministic Java checks (machine-verifiable criteria) and Bedrock analysis (human-judgment criteria that require understanding context).

Pass 1: Java Rules (Deterministic)

CheckImplementationWCAG Criterion
Image alt textRegex: every <img> must have non-empty alt1.1.1 Non-text Content
Form labelsEvery <input> has associated <label> or aria-label1.3.1 Info and Relationships
Color contrastParse CSS color/background-color; compute luminance ratio ≥ 4.5:11.4.3 Contrast (Minimum)
Heading hierarchyVerify h1h2h3 sequence; no skips1.3.1 Info and Relationships
Language attribute<html lang="..."> present3.1.1 Language of Page
Focus stylesCSS includes :focus rules; no outline: none without replacement2.4.7 Focus Visible

Pass 2: Bedrock Deep Review

  • Prompt: qa-accessibility-review.txt
  • AI evaluates: Semantic HTML usage, ARIA roles/states correctness, keyboard navigation completeness, screen reader experience, touch target sizing, cognitive load assessment
  • Key insight: Many WCAG criteria (e.g., "meaningful sequence", "consistent navigation") require human-level understanding that pure regex cannot provide

Scoring

  • Accessibility Score: 0–10 (weighted: Java checks 40%, Bedrock analysis 60%)
  • Maps each finding to specific WCAG Success Criterion with conformance level (A, AA, AAA)

Layer 5: Performance Audit Bedrock AI

Analyzes the asset graph and render path of the deployed site. Since these are static sites without server-side rendering, performance analysis focuses on client-side loading strategy, asset optimization, and perceived performance.

What Bedrock Analyzes

CategoryAnalysisCommon Findings
Asset sizeTotal page weight, individual file sizes, unminified detectionUnminified JS >50KB, oversized images
Render blocking<script> without defer/async, CSS in <head> load orderRender-blocking scripts in <head>
Image optimizationFormat analysis (PNG vs WebP), dimensions, lazy loadingMissing loading="lazy", no width/height
CachingAsset fingerprinting, cache-control headers, CDN usageNo cache busting on CSS/JS filenames
Critical render pathFirst paint blocking resources, inline critical CSS presenceAll CSS loaded before any content renders

Why Bedrock Instead of Lighthouse?

  • No headless Chrome needed: Lighthouse requires a browser runtime; Bedrock works from source alone
  • Context-aware: AI understands that a login page's performance profile differs from a dashboard
  • Actionable output: AI provides specific remediation steps, not just scores
  • Trade-off: Cannot measure actual FCP/LCP/CLS metrics — these require real rendering

Scoring

  • Performance Score: 0–10
  • Deductions: unminified assets (−2), render-blocking scripts (−1.5), no lazy loading (−1), missing cache strategy (−1)

๐Ÿ“Š Output: Report, PR Update & SSE Broadcast

After all 5 layers complete, QaReportBuilder aggregates findings into a unified report. Three outputs are generated simultaneously:

1. QA Report (Database + API)

  • DB entities: QaReport (one per run) + QaFinding (one per issue) stored via JPA
  • API endpoint: GET /api/qa/{reqId} returns JSON; GET /requirements/{reqId}/qa renders HTML view
  • Schema: Flyway V18__qa_tables.sqlqa_report (id, req_id, overall_score, security_score, accessibility_score, performance_score, functional_score, structure_pass, created_at) + qa_finding (id, report_id, layer, severity, category, description, location, remediation)

2. PR Description Patch

  • Mechanism: GitHub API PATCH /repos/{owner}/{repo}/pulls/{number}
  • Content: Appends QA badge (overall score with color), summary table of findings per layer, and critical findings with remediation steps
  • Advisory only: Does not block merge — provides visibility for human reviewer

3. SSE Broadcast

  • Event: QA_COMPLETE sent via PipelineStreamService
  • Payload: Overall score, per-layer scores, critical finding count
  • Dashboard: Real-time update on requirement detail page — QA section appears with expandable layer results

Composite Scoring

ComponentWeightRange
StructureGate (must pass)Pass / Fail
Security30%0–10
Functional30%0–10
Accessibility25%0–10
Performance15%0–10
Overall100%0–10