first commit

2026-01-25 23:05:41 +02:00
commit dec7844b49
48 changed files with 10815 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,182 @@
+# OSINT Automation Platform
+
+A private, containerized OSINT (Open Source Intelligence) automation platform for gathering social media intelligence using authenticated browser sessions.
+
+⚠️ **DISCLAIMER**: This tool is intended for authorized intelligence gathering and research purposes only. Ensure you comply with all applicable laws and platform terms of service.
+
+## Features
+
+- 🔐 **Master Password Protection** - Single secure entry point
+- 🎭 **Session Vault** - Store and reuse authenticated social media sessions (encrypted)
+- 🤖 **Automated Scraping** - Playwright-based stealth scraping engine
+- 📊 **Target Management** - Organize and track multiple investigation targets
+- 📱 **Multi-Platform Support** - X/Twitter, Instagram, LinkedIn, Facebook
+- 🔒 **Encrypted Storage** - AES-256-GCM encrypted session data
+- 📈 **Real-time Progress** - WebSocket-based live job updates
+- 🐳 **Docker Ready** - Optimized for Coolify deployment
+
+## Tech Stack
+
+- **Backend**: Node.js, Express, TypeScript
+- **Frontend**: React, Vite, TypeScript, Tailwind CSS
+- **Database**: MongoDB
+- **Browser Automation**: Playwright with stealth plugins
+- **Real-time**: Socket.IO
+- **State Management**: Zustand
+
+## Quick Start
+
+### Development
+
+1. Clone the repository:
+```bash
+git clone <repo-url>
+cd osint-platform
+```
+
+2. Install dependencies:
+```bash
+npm install
+cd backend && npm install
+cd ../frontend && npm install
+cd ..
+```
+
+3. Create environment files:
+```bash
+# Backend
+cp backend/.env.example backend/.env
+# Edit backend/.env with your configuration (especially MONGODB_URI)
+```
+
+4. Start development servers:
+```bash
+npm run dev
+```
+
+This will start:
+- Backend on `http://localhost:3001`
+- Frontend on `http://localhost:5173`
+
+### Production (Docker)
+
+1. Create your `.env` file:
+```bash
+cp .env.example .env
+# Edit .env with secure values
+```
+
+2. Build and run with Docker Compose:
+```bash
+docker-compose up -d
+```
+
+The application will be available at `http://localhost:3001`
+
+## Coolify Deployment
+
+1. Create a new service in Coolify
+2. Point to your Git repository
+3. Set Build Pack to "Dockerfile"
+4. Configure environment variables:
+   - `MONGODB_URI` - Your MongoDB connection string (e.g., MongoDB Atlas)
+   - `MASTER_PASSWORD` - Your secure master password
+   - `JWT_SECRET` - Generate with `openssl rand -hex 32`
+   - `VAULT_ENCRYPTION_KEY` - Generate with `openssl rand -hex 32`
+   - `FRONTEND_URL` - Your domain (e.g., `https://osint.yourdomain.com`)
+
+## Configuration
+
+### Environment Variables
+
+| Variable | Description | Required |
+|----------|-------------|----------|
+| `MONGODB_URI` | MongoDB connection string | Yes |
+| `MASTER_PASSWORD` | Password to access the platform | Yes |
+| `JWT_SECRET` | Secret for JWT token signing | Yes |
+| `VAULT_ENCRYPTION_KEY` | 64-char hex key for session encryption | Yes |
+| `PORT` | Server port (default: 3001) | No |
+| `SESSION_EXPIRY` | JWT expiry time (default: 24h) | No |
+| `FRONTEND_URL` | Frontend URL for CORS | No |
+
+### Generating Secrets
+
+```bash
+# Generate JWT Secret
+openssl rand -hex 32
+
+# Generate Vault Encryption Key
+openssl rand -hex 32
+```
+
+### MongoDB Setup
+
+You can use:
+- **MongoDB Atlas** (recommended for production): Create a free cluster at [mongodb.com/atlas](https://mongodb.com/atlas)
+- **Local MongoDB**: `mongodb://localhost:27017/osint_platform`
+- **Docker MongoDB**: Uncomment the mongodb service in docker-compose.yml
+
+## Usage
+
+### 1. Login
+Access the platform and enter your master password.
+
+### 2. Add Sessions
+Navigate to "Add Session" and provide:
+- Platform (X, Instagram, LinkedIn, Facebook)
+- Session name
+- Cookies JSON (export from your browser)
+
+**Getting cookies:**
+1. Log into the platform in your browser
+2. Open DevTools → Application → Cookies
+3. Export as JSON using the Cookie Editor extension
+
+### 3. Create Targets
+Add investigation targets with optional notes.
+
+### 4. Add Profiles
+Link social media profiles to targets:
+- Platform
+- Username
+- Profile URL
+
+### 5. Run Scrapers
+Click the play button on any profile to start scraping. Monitor progress in the right panel.
+
+## Architecture
+
+```
+osint-platform/
+├── backend/
+│   ├── src/
+│   │   ├── database/      # MongoDB connection
+│   │   ├── middleware/    # Auth middleware
+│   │   ├── models/        # Mongoose models
+│   │   ├── routes/        # API routes
+│   │   ├── scraper/       # Playwright scraper engine
+│   │   └── utils/         # Encryption, logging
+│   └── package.json
+├── frontend/
+│   ├── src/
+│   │   ├── components/    # React components
+│   │   ├── pages/         # Page components
+│   │   └── stores/        # Zustand stores
+│   └── package.json
+├── Dockerfile
+├── docker-compose.yml
+└── package.json
+```
+
+## Security Considerations
+
+1. **Use strong passwords** - The master password is your only line of defense
+2. **Secure your secrets** - Never commit `.env` files
+3. **Use HTTPS** - Always deploy behind HTTPS in production
+4. **Rate limiting** - Login attempts are rate-limited (5 per 15 minutes)
+5. **Session encryption** - All stored cookies are AES-256-GCM encrypted
+6. **MongoDB Security** - Use authentication and TLS for your MongoDB connection
+
+## License
+
+Private - All rights reserved.