PRD 07: Integrations, Media, Uploads, Webhooks, Submissions, and Statistics
Problem Statement
The legacy platform supports third-party integrations, media upload, file processing, form submissions, webhooks, email templates, statistics, and analytics. These are key to preserving real customer workflows. The old code has usable domain patterns but cloud lock-in, plaintext secret storage, weak public upload protections, and limited webhook reliability.
Solution
Build integration and media platform services that preserve old capabilities while adding:
- encrypted secret references;
- storage adapters;
- safe upload validation;
- public submission session scoping;
- webhook signing and retry;
- email templates;
- statistics and insights;
- tenant-aware access control.
Legacy Source References
studio-api-reference/src/modules/studio/modules/integration/studio-api-reference/src/modules/studio/modules/media/studio-api-reference/src/modules/studio/modules/webhook/studio-api-reference/src/modules/studio/modules/statistics/studio-api-reference/src/modules/houston/modules/form-submission/studio-api-reference/src/views/mail/zweistein-reference/admin/src/common/FileUpload/picasso-fe-reference/packages/common/src/entities/houston/ui/
User Stories
- As an admin, I want to connect integrations, so that flows can send data to external systems.
- As an admin, I want integration credentials protected, so that secrets are not exposed.
- As an admin, I want webhooks, so that external systems can receive flow/submission events.
- As a developer, I want webhook logs, so that failed deliveries can be debugged.
- As a developer, I want webhook retries, so that temporary failures do not lose events.
- As a creator, I want media upload, so that flows can include images, video, audio, and files.
- As an end user, I want upload progress, so that large file submissions feel reliable.
- As an end user, I want iPhone photos and common media formats to work, so that uploads are not frustrating.
- As a security owner, I want file uploads scanned and validated, so that public endpoints are safer.
- As a creator, I want form submissions collected and exported, so that runtime data is useful.
- As an admin, I want email templates, so that invites, verification, and notifications work.
- As a creator, I want statistics, so that I can understand views, completions, time-to-complete, and submissions.
- As an org admin, I want integration and media access tenant-scoped, so that customers cannot see each other's assets.
Functional Requirements
Integrations
- Support tenant-scoped integration records.
- Support integration types from legacy platform such as Google Sheets, Intercom, Canvas, and future additions.
- Store secret references, not plaintext secrets.
- Filter sensitive fields in API responses.
- Track who created, updated, or used an integration.
Media and Uploads
- Support image, video, audio, document, and general file uploads.
- Validate MIME, size, extension, and file signature server-side.
- Preserve useful old processing: HEIC conversion, image resize, video/audio limits, preview metadata.
- Use storage adapter for Hetzner/S3-compatible/object storage.
- Support signed upload/download URLs where appropriate.
- Support upload progress in frontend.
Public Form Submission
- Support public form submission for published companions.
- Scope upload sessions to a flow/form/session.
- Rate-limit per IP/session.
- Prevent hardcoded tenant/default storage paths.
- Support submission attachments.
- Trigger integration, webhook, and email actions after durable save.
Webhooks
- Support outbound webhook URL, secret, event types, and active status.
- Sign requests with HMAC-SHA256.
- Include event ID and idempotency key.
- Retry with exponential backoff.
- Store delivery logs with redacted payloads.
- Support dead-letter state and manual retry.
Statistics
- Preserve flow views, unique visitors, completions, average completion time, step-level analytics, and date-range filtering.
- Support insights UI for creators and admins.
- Index statistics by tenant, flow, and date.
Non-Functional Requirements
- No plaintext integration secrets in JSONB.
- No public upload endpoint without session validation and rate limit.
- No cloud-specific storage SDK in domain service logic.
- No webhook secret sent as plaintext header.
- PII in logs must be minimized or redacted.
Implementation Decisions
- Build
SecretsServicebefore porting integrations. - Build
StorageAdapterbefore porting media. - Build
WebhookDeliveryServiceas a separate durable service. - Save submissions before side effects.
- Use statistics service as a mostly reusable reporting pattern.
Testing Decisions
- API-test integration CRUD and secret redaction.
- Security-test secret retrieval boundaries.
- API-test media upload validation.
- Abuse-test public upload rate limits.
- Integration-test webhook retries and HMAC verification.
- Browser-test upload UX and statistics views.
Out of Scope
- Choosing every final integration provider before the integration registry exists.
- Keeping Azure or GCP as hard dependency.