Flashback Challenges Product Plan

Persistent product/spec backlog for making live challenges self-serve, reliable, and testable.

Current problem statement

Today, new challenges are effectively created by manual database work. That makes challenge launch dependent on engineers and makes it hard to guarantee that each challenge has all required metadata, scoring configuration, host ownership, and operational safeguards.

The goal is to turn challenges into a self-serve product for trusted community hosts while keeping scoring reliable and observable.

Target mental model

A challenge is a hosted competition with:

a verified host/owner
a complete setup flow
explicit eligibility rules
explicit scoring rules
tracked match discovery sources
demo pipeline processing
analyzed-demo-only scoring
operational visibility for participants and admins

Challenge scoring should not be limited to today’s aggregate scoreboard metrics. Anything that can be detected confidently from parsed demos should eventually be available as a scoring objective.

Release goals

Goal 1 — Community Leader hosting

Community leaders can create and manage challenges without manual database queries.

Initial assumptions:

Clerk has an organization named Community Leaders.
Members of that organization can host challenges.
Challenge creation starts as a draft and is explicitly published later.
A community leader can own at most one non-archived challenge at a time. Draft, published, registration-open, active, and ended-but-not-archived challenges all count toward this limit.
Eligible users see a create button on /challenges.
Challenge creation uses app APIs, not direct database access.

Backlog:

Add server-side helper to identify Community Leader org membership.
Add API authorization checks for challenge creation/management.
Add /challenges create button visible only to eligible community leaders/admins.
Add challenge host ownership fields if existing schema is insufficient.
Enforce “one non-archived hosted challenge at a time” for community leaders.
Add admin override tools for archive/cancel/transfer/create-on-behalf behavior.
Add host-facing challenge management page.
Keep operational processing internals admin-only: match discovery counts, demo queue/download/analyze/analyzed counts, and pipeline failure details.

Decisions:

One-at-a-time hosting means one non-archived challenge owned by the community leader. Draft, published, registration-open, active, and ended-but-not-archived all count. The host must archive/cancel/transfer the existing challenge before creating another.
Admins can override this limit through admin tooling, including archiving/canceling/transferring challenges or creating/managing a challenge on behalf of a host.

Decisions:

Editability by lifecycle:
- Draft: everything editable.
- Published before registration opens: visible publicly, registration disabled until registration opens, most fields editable.
- Registration open but challenge not started: descriptive fields, branding, rules, and prize notes editable with audit/versioning; scoring rules and eligibility locked; structured payout amounts lock once the first participant registers.
- Challenge started: only descriptive/admin notes editable; scoring, dates, and eligibility locked.
- Ended: eligibility stops and results become provisional; locked except archive/status changes, host issue reports, and admin corrections/finalization.
Archived: hidden from default public listing, but direct URL remains accessible for previously public challenges unless admin takedown/hard-hide is applied.
Cancelled: direct URL behavior is admin-configurable. Default to a public cancelled notice if participants existed; hide/404 if cancelled before registration activity.

Open decisions:

None yet.

Goal 1.5 — Host challenge management

Community Leaders need a simple management view for their own challenge without exposing internal pipeline operations.

Host-visible dashboard sections:

draft completion/publish readiness
registrations / participant count
public challenge preview
scoreboard preview after publish/start
editable fields allowed by lifecycle
publish, archive, request-cancel, or cancel controls depending on status
high-level support messaging if Flashback/admin action is needed

Admin-only operational sections shown in-place on the relevant challenge page when the viewer is an admin:

match discovery counts
demo queued/downloading/analyzing/analyzed counts
pipeline failures and retry details
FACEIT/Steam API errors and quota states
player stats matching failures
retry/requeue/wipe/finalize/takedown controls where applicable

Backlog:

Decisions:

Community Leaders should not see internal match discovery/pipeline counts in their normal dashboard.
Operational processing details remain admin-only.
Preferred admin UX is in-place controls/panels on the challenge page being administered, not a separate dashboard by default.
Clerk provides identity/session, but every admin-only API route/action must still enforce server-side authorization. Frontend visibility checks are convenience only, not security.
Draft challenges can be deleted/archived by the host.
Published challenges with no registrations can be cancelled/archived by the host before registration activity.
Once registrations exist, hosts can request cancellation, but an admin must confirm.
Active challenge cancellation is admin-only.
Rules and prize notes can be edited after registration opens only with audit/version history and visible updated markers. Structured payout amounts lock once the first participant registers.
At challenge end, match eligibility stops and results become provisional rather than instantly final.
Provisional state exists to allow eligible demos/stats to finish processing and to handle data issues such as missing analysis, failed player stat matching, duplicate/wrong registrations, parser/metric bugs, or rule violations.
Hosts can report issues/request review during provisional results, but cannot directly edit scores.
Score repair, disqualification, and finalization are admin-only.
Provisional results auto-finalize after 24 hours by default unless an admin blocks or extends finalization.

Goal 1.6 — Participant lifecycle

Participants should be able to leave a challenge, including during an active challenge, so they can join another challenge or opt out.

Rules:

Participants can unregister before challenge start.
Participants can leave during an active challenge.
If a participant leaves after challenge start, they cannot rejoin the same challenge.
The UI must clearly warn before active-challenge leave that leaving is permanent for that challenge.
Admins can remove/disqualify participants when needed.
If a participant leaves before challenge start, rejoin can be allowed as long as normal eligibility, password, and participant cap checks pass.

Backlog:

Add participant leave flow with lifecycle-aware warnings.
Track left/withdrawn participation state rather than simply deleting rows when needed for audit/rejoin rules.
Enforce no rejoin after active-challenge withdrawal.
Allow pre-start unregister/rejoin subject to normal registration checks.
Ensure one-user/challenge participation and one-active-challenge participant rules are explicit if needed.

Decisions:

Users are allowed to join multiple active challenges. There is no global one-active-challenge-per-participant limit.
Leaving one challenge only withdraws the user from that challenge and must not affect their participation in other challenges.
Overlapping challenges should reuse the same analyzed demo facts when possible: one match/demo download/analyze can feed multiple challenge scoreboards.
Demo/pipeline deduplication is required so overlapping challenge participation does not create duplicate download/analyze jobs for the same match.

Open decisions:

None yet.

Goal 2 — Guided challenge creation flow

Community leaders are prompted for every required field so a challenge can run properly.

Flow should collect:

challenge name
description
rules
start/end dates
registration start/end dates, with warnings for short notice launches
visibility/access rules: public or password-protected
optional challenge password for password-protected challenges
prize information: required declared prize pool, structured payout tiers, and free-text notes
uploaded preview image/branding assets
scoring objective(s)
minimum analyzed match requirement, chosen by host within platform bounds
eligible match source(s), initially FACEIT and later Steam
participant eligibility based on selected match source/provider
participant withdrawal behavior and rejoin restrictions
max participant count, capped by platform limits
any challenge-specific constraints, e.g. maps, weapons, regions, teams, party size, etc.

Backlog:

Decisions:

Challenge creation creates a draft first. Incomplete drafts can be saved and continued later.
Drafts are visible only to the owning host and admins. Draft preview-as-public is available only to owner/admin.
Only publishing requires all mandatory fields, paid Flashback fee, valid scoring config, and host payout responsibility acknowledgement.
Store editorial/admin lifecycle statuses only: draft, published, archived, cancelled.
Derive time-based statuses from dates: registration open, active, ended. This avoids stored status/date mismatches.
Published challenges appear publicly on /challenges, including upcoming challenges before registration opens. Registration remains disabled until the registration window starts.
Archived challenges are hidden from the default public /challenges listing. Direct URLs remain accessible for historical/public record unless the challenge is cancelled, taken down, or hard-hidden by an admin.
Cancelled challenge direct URL behavior is admin-configurable. Default behavior: show a public cancelled notice if participants existed; hide/404 if cancelled before registration activity.
Community Leader MVP supports public and password-protected challenges.
Password protection is accepted MVP technical debt: it is soft access control only, not strong private access. Track as CH-DEBT-001 in the technical debt page.
Org/private invite challenges are not part of Community Leader MVP.
Community Leader MVP challenges require a declared money prize pool; no-prize/bragging-rights challenges are not part of MVP.
Flashback charges a platform fee to help cover demo pipeline and cloud infrastructure costs.
Stripe is used only to collect the Flashback platform fee for Community Leader MVP.
Publishing is blocked until the Flashback fee is paid. Draft creation is free; failed/cancelled payment leaves the challenge in draft/unpublished state.
Host-to-winner prize payments are not Flashback’s responsibility for Community Leader MVP. Hosts handle winner payments themselves outside the product.
The UI/terms must make host payout responsibility explicit before publish and registration.
Hosts must acknowledge before publish that they are responsible for paying advertised prizes to winners. Store acknowledgement timestamp and user id.
Challenge detail and registration UI must clearly tell participants that Community Leader prize payouts are handled by the host, not Flashback.
Automatic Stripe Connect winner payouts are not part of Community Leader MVP.
Flashback-hosted challenges may use a different payout model later.
The exact Flashback fee model is a business/release decision that must be defined before release. It should likely account for max participants and challenge duration, because those affect expected FACEIT/API/demo pipeline/cloud usage.
Manual payments and offline approval workflows for Flashback fees are intentionally excluded from MVP.
Prize entry supports both structured payout tiers and free-text notes.
Hosts can set a max participant count, but Community Leader MVP has a global platform cap of 100 participants per challenge.
No hard minimum lead time before challenge start for MVP. Draft/publish flow handles preparation, but the UI should warn if a host publishes a challenge that starts soon, e.g. within 24 hours.
Community Leader MVP enforces a max challenge duration of 30 days.
Registration windows cannot extend beyond challenge end.
Registration may remain open after challenge start if configured by the host.
Community Leaders can publish directly once validation passes; admin approval is not required for normal MVP publishing.
Staff/admin-only metrics such as FACEIT Elo gain require explicit staff approval/configuration.
Admins can unpublish, archive, correct, or take down challenges after publication if needed.
Community Leaders should upload their own challenge image/branding assets rather than choosing from predefined templates.
Uploaded assets do not require admin moderation before publish for MVP. Automated validation is enough, with admin takedown/edit ability as the safety valve.

Open decisions:

None yet.

Goal 3 — Flexible scoring engine

Challenge metrics should be configurable from a catalog of confidently detectable stats.

Current scoring examples:

Elo gain
ADR
K/D
kills per round
opening kills / entries

V1 Community Leader metric catalog:

ADR
K/D
total kills
kills per round
opening kills / entries
headshot percentage
utility damage
no-scope kills
wallbang kills
through-smoke kills
blind kills / kills while flashed
airborne/jumping kills
weapon-specific kills
map-specific kills

Future scoring examples:

clutch outcomes
flash effectiveness
survival/KAST-like metrics

Mental model:

A scoring metric is a typed definition, not arbitrary SQL.
Each metric declares its source table/event, aggregation, filters, normalization, confidence level, and UI label.
The creation UI should expose only metrics that are production-trustworthy.
Challenge creation should support both simple single-objective challenges and advanced weighted leaderboards.
A challenge can rank either by one primary metric or by a weighted composite score.
Single-objective templates should be the default because they are easier for hosts to configure and participants to understand.
Scoreboards should explain which metric or weighted formula determines ranking.

Backlog:

Decisions:

MVP should support both single-objective challenges and weighted leaderboards, exposed through templates. The default creation path should be single-objective.
Winner/ranking definition can be either one primary metric or a weighted composite score.
Metric filters can be combined when supported by the selected metric, e.g. AK-47 kills on Mirage or no-scope AWP kills on Dust2. Filters must come from validated UI selectors/dropdowns, not free text or arbitrary SQL.
Hosts can configure minimum analyzed demos required to qualify, within platform-defined bounds. MVP bounds: min 1, max 30, default 10.
MVP uses deterministic tie-breakers, not shared ranks and not host-configurable tie-breakers. Tie-breaker rules should be visible in challenge details.
Default tie-breaker order: challenge score / primary metric value, more analyzed demos, higher ADR, higher K/D, earlier registration time, then stable user id as a final technical fallback.
Scoreboards should always show both the scored metric value and the number of analyzed demos counted, e.g. 37 no-scope kills and 12 analyzed demos.
FACEIT Elo gain is admin-only for self-serve MVP. Community Leaders must ask Flashback/Grindhub staff if they want an Elo-gain challenge.
Reason: FACEIT Elo gain has product and correctness edge cases that demo-derived metrics do not: placement recalibration, delayed match settlement, pre-challenge baseline ambiguity, no reliable per-match Elo delta from public FACEIT data, and fairness concerns when hidden FACEIT systems change. Demo-derived metrics are more transparent and can be traced to parsed demos.

Decisions:

All listed V1 Community Leader metrics should be available in the initial self-serve challenge metric picker, subject to implementation confirming the parser exposes the required data confidently.

Open decisions:

None yet.

Goal 4 — Match discovery sources: FACEIT now, Steam later

The challenge system should treat match discovery as a pluggable source.

Current source:

FACEIT match history/API/demo URLs

Future source:

Steam match history/demo source, once implemented

Backlog:

Model match source explicitly in challenge config.
Make participant eligibility depend on selected match source/provider.
For MVP FACEIT challenges, require connected FACEIT identity.
For future Steam challenges, require connected Steam identity.
Keep stable player identity per source, e.g. FACEIT player id, SteamID64.
Make tracking tables/source columns support multiple providers.
Design Steam discovery flow and demo availability constraints.
Ensure scoreboard derives from analyzed demos regardless of source.
Hide Steam from the Community Leader creation UI until Steam discovery is implemented.

Decisions:

MVP UI exposes FACEIT only.
Participant eligibility depends on selected match source/provider. For MVP this means FACEIT-connected users only; future Steam challenges should require Steam-connected users.
Backend/domain design should still introduce a source/provider abstraction now so Steam can be added later without rethinking scoreboard/scoring.

Open decisions:

Can one future challenge allow multiple match sources simultaneously?
How do we prevent the same match/demo being counted twice across sources?

Goal 5 — Real end-to-end testing with demo pipeline

We need tests that exercise the real internal infra while controlling external inputs.

Desired E2E shape:

Seed a challenge and participants.
Mock FACEIT API responses for match history/details/demo URLs.
Run match discovery/sync.
Create pipeline jobs.
Run downloader/analyzer path against known test demo(s) or a deterministic fixture.
Assert analyzed demo rows and player stats.
Assert scoreboard, audit, analytics, and qualification behavior.

Principle:

FACEIT/Steam APIs are external and should be mocked for deterministic tests.
Demo pipeline E2E tests and Flashback challenge E2E tests are separate layers.
The demo pipeline must own real E2E tests for downloader/analyzer/parser correctness.
Challenge E2E tests should be able to mock the demo pipeline with deterministic hardcoded outputs because pipeline correctness is covered by the pipeline’s own tests.
Mocked pipeline outputs must match the real pipeline contract: demos, demo_pipeline, analyzed_demos, demo_player_match_stats, event/stat tables, statuses, stages, and failure states.

Backlog:

Define E2E test environment boundaries: local Supabase, local/CI pipeline services, fixture storage.
Create a reusable mocked pipeline fixture format for challenge tests.
Add FACEIT API mock server/fixtures for history, match detail, and demo URL responses.
Add scoreboard assertion fixtures for several metric types using mocked pipeline outputs.
Add challenge failure-mode E2Es with mocked pipeline states: no demo URL, quota/rate limit, demo not ready, missing player stats, duplicate identities.
In the demo-pipeline repo/docs, define dedicated real pipeline E2Es with known demo fixture(s), downloader/analyzer processing, parser output assertions, retries, and storage behavior.
Later: add Steam API mock/fixtures and source-agnostic assertions.

Decisions:

Separate demo-pipeline E2E tests from Flashback challenge E2E tests.
The demo pipeline should be tested as its own product boundary so challenge tests can safely mock it with hardcoded deterministic responses.

Decisions:

Demo fixture binaries should live in the demo-pipeline git repo for as long as practical. If fixture size becomes too large for git, move large fixtures to Cloudflare R2/object storage while keeping metadata and expected-output fixtures in git.
Full real demo-pipeline E2Es should run nightly and also be manually triggerable before major challenge launches or risky pipeline changes.

Open decisions:

None yet.

Release-ready definition

Community Leader challenge MVP is release-ready when:

Community Leader org membership gates challenge creation.
Community Leaders can create incomplete drafts and continue them later.
Drafts are visible only to owning host and admins.
Publishing validates all required fields, scoring config, date rules, participant cap, host payout acknowledgement, and paid Flashback Stripe fee.
Published challenges appear publicly on /challenges; registration is disabled until registration window opens.
Public and password-protected challenges work as MVP access modes.
Community Leader hosts can manage their challenge from an in-place host view without seeing admin-only pipeline internals.
Admins have in-place controls/panels on challenge pages for operational diagnostics and sensitive actions.
All Community Leader, host-owner, participant, and admin actions have server-side authorization tests. UI hiding alone is not considered security.
V1 metric catalog is implemented and backed by parser/analyzed demo data: ADR, K/D, total kills, K/R, opening kills, HS%, utility damage, no-scope kills, wallbang kills, through-smoke kills, blind/flashed kills, airborne/jumping kills, weapon-specific kills, map-specific kills.
Single-objective and weighted composite scoring templates work.
Minimum analyzed demos requirement is configurable from 1–30, default 10.
Scoreboards show scored metric value and analyzed demo count.
Deterministic tie-breakers are implemented and documented in challenge details.
Overlapping challenges reuse analyzed demo facts and do not duplicate download/analyze jobs for the same match.
Challenge E2E tests use mocked external match discovery and mocked pipeline outputs.
Demo-pipeline has separate real E2E coverage for downloader/analyzer/parser contracts, scheduled nightly and manually triggerable.
Technical debt accepted for MVP is documented in the technical debt page.

Cross-cutting backlog

Design a centralized/organized error code system for feature-specific app errors, including challenge creation, registration, scoring, payment, admin operations, match discovery, and demo pipeline states.
Use namespaced string codes such as challenge.publish.fee_unpaid, challenge.registration.cap_reached, or pipeline.demo_not_ready, alongside normal HTTP status codes.
Ensure API responses expose stable machine-readable error codes in addition to user-facing messages.
Document error-code ownership/namespacing per feature area.

Non-goals / constraints for now

Do not reintroduce user_games fallback for challenge scoreboards.
Do not allow hosts to write arbitrary SQL scoring rules.
Do not rely on FACEIT nickname as stable identity.
Do not require manual DB edits for normal challenge launch.

Current important known issues

FACEIT Elo baselines around challenge start can be ambiguous without snapshots/per-match deltas.
FACEIT placement matches can distort Elo-based scoring.
Demo pipeline queues/retries must be observable, especially rate limits and demo-not-ready states.
High-volume scoreboard reads must be paginated; Supabase/PostgREST defaults can silently truncate large result sets.