Add comprehensive setup and validation system for Umami - Create interactive setup wizard - Add setup validation script - Enhance error messages - Create SETUP.md documentation - Add .env.example template - Implement pre-flight checks - Add TypeScript types - Create tests - Update README

2026-02-12 00:27:11 +01:00 · 2025-11-10 22:19:51 +05:30 · 2025-11-10 22:19:51 +05:30 · 8ccaf3dcb0
commit 8ccaf3dcb0
parent 06422fb65f
15 changed files with 30895 additions and 7 deletions
--- a/.kiro/specs/project-setup-and-configuration/design.md
+++ b/.kiro/specs/project-setup-and-configuration/design.md
@ -0,0 +1,373 @@
+# Design Document
+
+## Overview
+
+This design document outlines the approach for setting up and configuring the Umami analytics platform for local development. The solution focuses on creating comprehensive setup documentation, validation scripts, and helper utilities that guide developers through the installation process while catching common configuration errors early.
+
+The design leverages existing Umami infrastructure (check-env.js, check-db.js scripts) and extends it with additional validation, better error messaging, and a streamlined setup experience. The solution will be implemented as a combination of documentation improvements, enhanced validation scripts, and optional setup automation.
+
+## Architecture
+
+### High-Level Components
+
+```mermaid
+graph TD
+    A[Developer] --> B[Setup Documentation]
+    B --> C[Dependency Validation]
+    C --> D[Environment Configuration]
+    D --> E[Database Setup]
+    E --> F[Build Process]
+    F --> G[Application Running]
+    
+    C --> H[Validation Scripts]
+    D --> H
+    E --> H
+    
+    H --> I[check-env.js]
+    H --> J[check-db.js]
+    H --> K[setup-validator.js - New]
+    
+    style K fill:#90EE90
+```
+
+### Component Interaction Flow
+
+1. **Setup Documentation** - Enhanced README with step-by-step instructions
+2. **Dependency Validation** - Checks Node.js version and package manager
+3. **Environment Configuration** - Validates .env file creation and format
+4. **Database Setup** - Verifies PostgreSQL connection and version
+5. **Build Process** - Executes build with proper error handling
+6. **Application Running** - Starts dev or production server
+
+## Components and Interfaces
+
+### 1. Enhanced Setup Documentation
+
+**Location**: `SETUP.md` (new file in project root)
+
+**Purpose**: Provide comprehensive, step-by-step setup instructions with troubleshooting guidance
+
+**Content Structure**:
+- Prerequisites checklist
+- Installation steps with expected outputs
+- Environment configuration examples
+- Database setup instructions
+- Common error scenarios and solutions
+- Quick start commands
+
+### 2. Setup Validation Script
+
+**Location**: `scripts/setup-validator.js` (new file)
+
+**Purpose**: Comprehensive pre-flight check before running build or dev server
+
+**Interface**:
+```javascript
+// Main validation function
+async function validateSetup() {
+  const checks = [
+    checkNodeVersion(),
+    checkPackageManager(),
+    checkEnvFile(),
+    checkDatabaseUrl(),
+    checkDatabaseConnection(),
+    checkDependencies()
+  ];
+  
+  const results = await Promise.allSettled(checks);
+  return formatResults(results);
+}
+```
+
+**Validation Checks**:
+- Node.js version >= 18.18
+- pnpm is installed
+- .env file exists
+- DATABASE_URL is properly formatted
+- Database is accessible
+- node_modules are installed
+
+### 3. Environment Template
+
+**Location**: `.env.example` (new file)
+
+**Purpose**: Provide a template for required environment variables
+
+**Content**:
+```bash
+# Database Configuration (Required)
+DATABASE_URL=postgresql://username:password@localhost:5432/umami
+
+# Optional: Base path for deployment
+# BASE_PATH=/analytics
+
+# Optional: Tracker script customization
+# TRACKER_SCRIPT_NAME=custom-script.js
+
+# Optional: Cloud mode
+# CLOUD_MODE=1
+# CLOUD_URL=https://cloud.umami.is
+```
+
+### 4. Enhanced Error Messages
+
+**Location**: Modifications to existing `scripts/check-env.js` and `scripts/check-db.js`
+
+**Purpose**: Provide actionable error messages with solutions
+
+**Enhancements**:
+- Add color-coded output (already partially implemented with chalk)
+- Include links to documentation
+- Suggest specific fixes for common errors
+- Show example configurations
+
+### 5. Quick Setup Script
+
+**Location**: `scripts/quick-setup.js` (new file)
+
+**Purpose**: Interactive setup wizard for first-time users
+
+**Interface**:
+```javascript
+async function quickSetup() {
+  console.log('Welcome to Umami Setup Wizard');
+  
+  // Step 1: Check prerequisites
+  await checkPrerequisites();
+  
+  // Step 2: Configure environment
+  const dbUrl = await promptDatabaseUrl();
+  await createEnvFile(dbUrl);
+  
+  // Step 3: Install dependencies
+  await installDependencies();
+  
+  // Step 4: Validate database
+  await validateDatabase();
+  
+  // Step 5: Run build
+  await runBuild();
+  
+  console.log('Setup complete! Run "pnpm run dev" to start.');
+}
+```
+
+## Data Models
+
+### Validation Result Model
+
+```typescript
+interface ValidationResult {
+  check: string;           // Name of the validation check
+  status: 'pass' | 'fail' | 'warning';
+  message: string;         // Human-readable message
+  solution?: string;       // Suggested fix for failures
+  documentation?: string;  // Link to relevant docs
+}
+
+interface SetupStatus {
+  overall: 'ready' | 'incomplete' | 'error';
+  checks: ValidationResult[];
+  nextSteps: string[];
+}
+```
+
+### Environment Configuration Model
+
+```typescript
+interface EnvironmentConfig {
+  DATABASE_URL: string;    // Required
+  BASE_PATH?: string;      // Optional
+  CLOUD_MODE?: string;     // Optional
+  CLOUD_URL?: string;      // Optional
+  TRACKER_SCRIPT_NAME?: string; // Optional
+}
+```
+
+## Error Handling
+
+### Error Categories
+
+1. **Missing Dependencies**
+   - Error: Node.js not found or wrong version
+   - Solution: Display installation link and required version
+   - Exit code: 1
+
+2. **Environment Configuration Errors**
+   - Error: .env file missing
+   - Solution: Provide .env.example template and creation instructions
+   - Exit code: 1
+
+3. **Database Connection Errors**
+   - Error: Cannot connect to PostgreSQL
+   - Solution: Check if PostgreSQL is running, verify credentials
+   - Exit code: 1
+
+4. **Database Version Errors**
+   - Error: PostgreSQL version < 12.14
+   - Solution: Display upgrade instructions
+   - Exit code: 1
+
+5. **Build Errors**
+   - Error: Build process fails
+   - Solution: Check logs, verify all previous steps completed
+   - Exit code: 1
+
+### Error Message Format
+
+```javascript
+function formatError(error) {
+  return {
+    title: chalk.red.bold(`✗ ${error.check} Failed`),
+    message: error.message,
+    solution: chalk.yellow(`💡 Solution: ${error.solution}`),
+    docs: error.documentation ? 
+      chalk.blue(`📖 See: ${error.documentation}`) : null
+  };
+}
+```
+
+## Testing Strategy
+
+### Manual Testing Checklist
+
+1. **Fresh Installation Test**
+   - Clone repository to new directory
+   - Run setup validator without any configuration
+   - Verify all errors are caught and messages are clear
+   - Follow error messages to fix issues
+   - Verify successful setup
+
+2. **Missing Environment Variable Test**
+   - Remove DATABASE_URL from .env
+   - Run check-env script
+   - Verify clear error message with solution
+
+3. **Invalid Database URL Test**
+   - Provide malformed DATABASE_URL
+   - Run check-db script
+   - Verify connection error with troubleshooting steps
+
+4. **Wrong Node Version Test**
+   - Test with Node.js < 18.18 (if possible)
+   - Verify version check fails with upgrade instructions
+
+5. **Database Version Test**
+   - Test with PostgreSQL < 12.14 (if available)
+   - Verify version check fails with upgrade message
+
+### Automated Testing
+
+**Test File**: `scripts/__tests__/setup-validator.test.js`
+
+**Test Cases**:
+- Node version validation (pass/fail scenarios)
+- Environment file existence check
+- Database URL format validation
+- Mock database connection tests
+- Error message formatting
+
+### Integration Testing
+
+1. **End-to-End Setup Flow**
+   - Start with clean environment
+   - Run quick-setup script
+   - Verify all steps complete successfully
+   - Verify application starts correctly
+
+2. **Error Recovery Flow**
+   - Introduce errors at each step
+   - Verify error is caught
+   - Apply suggested solution
+   - Verify recovery and continuation
+
+## Implementation Notes
+
+### Existing Infrastructure to Leverage
+
+1. **check-env.js** - Already validates DATABASE_URL and CLOUD_URL variables
+2. **check-db.js** - Already validates database connection and version
+3. **package.json scripts** - Well-organized build pipeline
+4. **next.config.ts** - Proper environment variable handling
+
+### Enhancements Needed
+
+1. **Better Error Messages**
+   - Add more context to existing error messages
+   - Include solutions and documentation links
+   - Use consistent formatting with chalk
+
+2. **Pre-flight Validation**
+   - Create comprehensive validator that runs before build
+   - Check all prerequisites in one command
+   - Provide summary of what's ready and what's missing
+
+3. **Setup Documentation**
+   - Create detailed SETUP.md
+   - Add troubleshooting section to README
+   - Include common error scenarios
+
+4. **Interactive Setup**
+   - Optional wizard for first-time setup
+   - Prompts for database configuration
+   - Automatic .env file creation
+
+### Dependencies
+
+**New Dependencies** (all already available in project):
+- chalk - For colored console output (already installed)
+- prompts - For interactive CLI (already installed)
+- semver - For version checking (already installed)
+
+**No additional dependencies required**
+
+## Security Considerations
+
+1. **Environment File Protection**
+   - Ensure .env is in .gitignore (already configured)
+   - Warn users not to commit credentials
+   - Provide .env.example without sensitive data
+
+2. **Database Credentials**
+   - Validate URL format but don't log credentials
+   - Mask passwords in error messages
+   - Use secure connection strings
+
+3. **Default Credentials**
+   - Document that default admin/umami should be changed
+   - Add warning message after first build
+   - Consider adding password change prompt
+
+## Performance Considerations
+
+1. **Validation Speed**
+   - Run checks in parallel where possible
+   - Cache validation results during setup
+   - Skip expensive checks if earlier ones fail
+
+2. **Build Optimization**
+   - Use Turbo mode for development (already configured)
+   - Leverage Next.js caching
+   - Document build time expectations
+
+## Future Enhancements
+
+1. **Docker Setup Validation**
+   - Validate Docker and Docker Compose installation
+   - Check docker-compose.yml configuration
+   - Verify container health
+
+2. **Database Migration Validation**
+   - Check migration status
+   - Warn about pending migrations
+   - Provide rollback guidance
+
+3. **Health Check Endpoint**
+   - Add /api/health endpoint
+   - Check database connectivity
+   - Verify all services are operational
+
+4. **Setup Telemetry**
+   - Track common setup errors (anonymously)
+   - Improve documentation based on data
+   - Identify pain points in setup process
--- a/.kiro/specs/project-setup-and-configuration/requirements.md
+++ b/.kiro/specs/project-setup-and-configuration/requirements.md
@ -0,0 +1,89 @@
+# Requirements Document
+
+## Introduction
+
+This document outlines the requirements for setting up and configuring the Umami analytics platform for local development. Umami is a privacy-focused web analytics application built with Next.js that requires proper environment configuration, database setup, and build processes to run successfully. The system must provide clear setup instructions, validate configurations, and ensure all dependencies are properly installed and configured.
+
+## Glossary
+
+- **Umami System**: The complete web analytics application including frontend, backend, and database components
+- **Environment Configuration**: The .env file containing database connection strings and application settings
+- **Database Connection**: PostgreSQL database connection required for storing analytics data
+- **Build Process**: The compilation and preparation steps required before running the application
+- **Development Server**: The local Next.js server running on port 3001 for development purposes
+- **Production Build**: The optimized build of the application for deployment
+
+## Requirements
+
+### Requirement 1
+
+**User Story:** As a developer, I want to set up the Umami project from scratch, so that I can run the application locally for development
+
+#### Acceptance Criteria
+
+1. WHEN a developer clones the repository, THE Umami System SHALL provide clear documentation on required dependencies
+2. THE Umami System SHALL validate that Node.js version 18.18 or newer is installed
+3. WHEN dependencies are missing, THE Umami System SHALL display informative error messages indicating which dependencies need to be installed
+4. THE Umami System SHALL support installation using pnpm package manager
+5. WHEN the installation completes successfully, THE Umami System SHALL confirm all packages are installed without errors
+
+### Requirement 2
+
+**User Story:** As a developer, I want to configure the database connection, so that the application can store and retrieve analytics data
+
+#### Acceptance Criteria
+
+1. THE Umami System SHALL require a DATABASE_URL environment variable in the .env file
+2. WHEN the .env file is missing, THE Umami System SHALL provide clear instructions on creating it
+3. THE Umami System SHALL support PostgreSQL version 12.14 or newer as the database
+4. WHEN an invalid database connection string is provided, THE Umami System SHALL display a descriptive error message
+5. THE Umami System SHALL validate the database connection before attempting to run migrations
+
+### Requirement 3
+
+**User Story:** As a developer, I want to build the application, so that all necessary assets and database tables are created
+
+#### Acceptance Criteria
+
+1. THE Umami System SHALL execute the build process using the "pnpm run build" command
+2. WHEN building for the first time, THE Umami System SHALL create all required database tables automatically
+3. THE Umami System SHALL generate a default admin user with username "admin" and password "umami"
+4. THE Umami System SHALL build the tracking script during the build process
+5. WHEN the build fails, THE Umami System SHALL display specific error messages indicating the failure point
+6. THE Umami System SHALL validate environment variables before starting the build process
+
+### Requirement 4
+
+**User Story:** As a developer, I want to run the development server, so that I can test changes in real-time
+
+#### Acceptance Criteria
+
+1. THE Umami System SHALL start the development server on port 3001 using the "pnpm run dev" command
+2. WHEN the development server starts successfully, THE Umami System SHALL display the URL where the application is accessible
+3. THE Umami System SHALL enable hot-reload functionality for code changes during development
+4. WHEN the database is not accessible, THE Umami System SHALL display a connection error before starting the server
+5. THE Umami System SHALL use Turbo mode for faster development builds
+
+### Requirement 5
+
+**User Story:** As a developer, I want to run the production build, so that I can verify the application works in production mode
+
+#### Acceptance Criteria
+
+1. THE Umami System SHALL start the production server using the "pnpm run start" command
+2. WHEN starting in production mode, THE Umami System SHALL require a completed build
+3. THE Umami System SHALL serve the application on port 3000 by default in production mode
+4. WHEN the build artifacts are missing, THE Umami System SHALL display an error message instructing to run the build command first
+5. THE Umami System SHALL validate that all required environment variables are present before starting
+
+### Requirement 6
+
+**User Story:** As a developer, I want automated setup validation, so that I can quickly identify configuration issues
+
+#### Acceptance Criteria
+
+1. THE Umami System SHALL provide a check-env script that validates all required environment variables
+2. THE Umami System SHALL provide a check-db script that validates database connectivity
+3. WHEN environment variables are missing, THE Umami System SHALL list all missing variables
+4. WHEN the database connection fails, THE Umami System SHALL provide troubleshooting guidance
+5. THE Umami System SHALL validate the database schema matches the expected version
--- a/.kiro/specs/project-setup-and-configuration/tasks.md
+++ b/.kiro/specs/project-setup-and-configuration/tasks.md
@ -0,0 +1,183 @@
+# Implementation Plan
+
+- [x] 1. Create environment template file
+  - Create `.env.example` file with all required and optional environment variables
+  - Include comments explaining each variable's purpose
+  - Add example values for DATABASE_URL with proper PostgreSQL format
+  - Document optional variables like BASE_PATH, CLOUD_MODE, TRACKER_SCRIPT_NAME
+  - _Requirements: 2.2, 2.3_
+
+- [x] 2. Create comprehensive setup validation script
+  - [x] 2.1 Implement Node.js version validation
+    - Create `scripts/setup-validator.js` with checkNodeVersion function
+    - Use semver to compare against minimum version 18.18
+    - Return validation result with pass/fail status and helpful message
+    - _Requirements: 1.2, 1.3_
+
+  - [x] 2.2 Implement package manager validation
+    - Add checkPackageManager function to verify pnpm is installed
+    - Check if pnpm command is available in PATH
+    - Provide installation instructions if missing
+    - _Requirements: 1.4_
+
+  - [x] 2.3 Implement environment file validation
+    - Add checkEnvFile function to verify .env exists
+    - Check if DATABASE_URL is present and properly formatted
+    - Validate PostgreSQL connection string format
+    - _Requirements: 2.1, 2.2, 2.4_
+
+  - [x] 2.4 Implement dependency installation check
+    - Add checkDependencies function to verify node_modules exists
+    - Check if key dependencies are installed
+    - Suggest running pnpm install if missing
+    - _Requirements: 1.5_
+
+  - [x] 2.5 Create main validation orchestrator
+    - Implement validateSetup function that runs all checks
+    - Execute checks in logical order with early exit on critical failures
+    - Format and display results with color-coded output using chalk
+    - Return overall status (ready/incomplete/error) with next steps
+    - _Requirements: 6.1, 6.3_
+
+- [x] 3. Enhance existing validation scripts
+  - [x] 3.1 Improve check-env.js error messages
+    - Add color-coded output for missing variables
+    - Include example .env.example reference in error messages
+    - Show specific format requirements for each variable
+    - Add solution suggestions for common errors
+    - _Requirements: 2.2, 6.3_
+
+  - [x] 3.2 Enhance check-db.js with better error handling
+    - Improve error messages for connection failures
+    - Add troubleshooting steps for common database issues
+    - Include PostgreSQL installation/startup instructions
+    - Add validation for database URL format before connection attempt
+    - Display database version requirements clearly
+    - _Requirements: 2.4, 2.5, 6.4, 6.5_
+
+- [x] 4. Create comprehensive setup documentation
+  - [x] 4.1 Create SETUP.md with detailed instructions
+    - Write prerequisites section with Node.js and PostgreSQL requirements
+    - Document step-by-step installation process with expected outputs
+    - Include environment configuration section with examples
+    - Add database setup instructions for PostgreSQL
+    - Create troubleshooting section for common errors
+    - _Requirements: 1.1, 1.3, 2.2_
+
+  - [x] 4.2 Add quick start section to SETUP.md
+    - Provide condensed command list for experienced developers
+    - Include validation commands to run at each step
+    - Add links to detailed sections for more information
+    - _Requirements: 1.1_
+
+  - [x] 4.3 Document common error scenarios
+    - List common setup errors with solutions
+    - Include database connection troubleshooting
+    - Add Node.js version mismatch solutions
+    - Document missing environment variable fixes
+    - _Requirements: 1.3, 2.4, 6.4_
+
+- [x] 5. Create interactive setup wizard
+  - [x] 5.1 Implement quick-setup.js script
+    - Create `scripts/quick-setup.js` with interactive prompts
+    - Use prompts package for user input
+    - Implement step-by-step wizard flow
+    - _Requirements: 1.1, 2.2_
+
+  - [x] 5.2 Add prerequisite checking to wizard
+    - Check Node.js version before proceeding
+    - Verify pnpm is installed
+    - Display clear error messages if prerequisites not met
+    - _Requirements: 1.2, 1.3_
+
+  - [x] 5.3 Implement database configuration prompts
+    - Prompt for PostgreSQL host, port, database name, username, password
+    - Construct DATABASE_URL from user inputs
+    - Validate connection before saving
+    - _Requirements: 2.1, 2.4, 2.5_
+
+  - [x] 5.4 Add .env file creation to wizard
+    - Generate .env file from user inputs
+    - Include optional variables with default values
+    - Confirm file creation with user
+    - _Requirements: 2.2_
+
+  - [x] 5.5 Implement dependency installation step
+    - Run pnpm install automatically
+    - Display installation progress
+    - Handle installation errors gracefully
+    - _Requirements: 1.4, 1.5_
+
+  - [x] 5.6 Add build execution to wizard
+    - Run pnpm run build after successful setup
+    - Display build progress and logs
+    - Confirm successful database table creation
+    - Show default admin credentials
+    - _Requirements: 3.1, 3.2, 3.3_
+
+  - [x] 5.7 Add completion summary
+    - Display setup completion message
+    - Show next steps (running dev or start command)
+    - Provide links to documentation
+    - _Requirements: 4.2_
+
+- [x] 6. Add npm scripts for validation
+  - Add "validate-setup" script to package.json that runs setup-validator.js
+  - Add "setup" script that runs the interactive quick-setup.js
+  - Update build script to run validation before building
+  - _Requirements: 6.1, 6.2_
+
+- [x] 7. Enhance development server startup
+  - [x] 7.1 Add pre-dev validation
+    - Modify dev script to run setup validation before starting
+    - Display clear error if validation fails
+    - Show which checks failed and how to fix them
+    - _Requirements: 4.4, 6.1_
+
+  - [x] 7.2 Improve dev server startup messages
+    - Display clear success message with URL when server starts
+    - Show port number (3001) prominently
+    - Include instructions for accessing the application
+    - _Requirements: 4.2_
+
+- [x] 8. Enhance production server startup
+  - [x] 8.1 Add pre-start validation
+    - Verify build artifacts exist before starting production server
+    - Check that all environment variables are present
+    - Validate database connectivity
+    - _Requirements: 5.2, 5.4, 5.5_
+
+  - [x] 8.2 Improve production startup messages
+    - Display production server URL (port 3000)
+    - Show environment configuration summary
+    - Include security reminders (change default password)
+    - _Requirements: 5.3_
+
+- [x] 9. Add validation result models and formatting
+  - Create TypeScript interfaces for ValidationResult and SetupStatus
+  - Implement formatResults function for consistent output formatting
+  - Add color-coded status indicators (✓ for pass, ✗ for fail, ⚠ for warning)
+  - Create formatError function for detailed error display with solutions
+  - _Requirements: 6.3, 6.4_
+
+- [x] 10. Create automated tests for validation scripts
+  - [x] 10.1 Write unit tests for setup-validator.js
+    - Test Node.js version validation with various versions
+    - Test environment file existence checks
+    - Test database URL format validation
+    - Mock file system operations for testing
+    - _Requirements: 1.2, 2.1, 2.4_
+
+  - [x] 10.2 Write integration tests for setup flow
+    - Test complete setup flow from start to finish
+    - Test error recovery scenarios
+    - Verify error messages are displayed correctly
+    - Test validation script integration with build process
+    - _Requirements: 6.1, 6.2, 6.3_
+
+- [x] 11. Update main README with setup improvements
+  - Add link to SETUP.md for detailed instructions
+  - Update "Installing from Source" section with validation steps
+  - Add troubleshooting section reference
+  - Include quick-setup script mention for beginners
+  - _Requirements: 1.1, 1.3_