279 lines
6.3 KiB
Markdown
279 lines
6.3 KiB
Markdown
# ServiceManagerWeb Workers
|
|
|
|
Workers asíncronos con Celery para el sistema de Mesa de Ayuda B2B.
|
|
|
|
## Estructura
|
|
|
|
```
|
|
workers/
|
|
├── app/
|
|
│ ├── celery.py # Configuración principal de Celery
|
|
│ ├── core/ # Configuración compartida
|
|
│ │ ├── config.py # Settings para workers
|
|
│ │ └── logging.py # Logging estructurado
|
|
│ └── tasks/ # Tareas por dominio
|
|
│ ├── email_tasks.py # Envío de emails
|
|
│ ├── sla_tasks.py # Monitoreo de SLAs
|
|
│ ├── maintenance_tasks.py # Mantenimiento del sistema
|
|
│ └── notification_tasks.py # Notificaciones y digests
|
|
├── requirements.txt # Dependencias Python
|
|
└── README.md # Esta documentación
|
|
```
|
|
|
|
## Tareas Implementadas
|
|
|
|
### Email Tasks (`email_tasks.py`)
|
|
- [x] `send_email_task`: Envío básico de emails SMTP
|
|
- [x] `send_templated_email_task`: Emails con plantillas Jinja2
|
|
- [x] `send_bulk_email_task`: Envío masivo con progreso
|
|
|
|
### SLA Tasks (`sla_tasks.py`)
|
|
- [x] `check_sla_violations`: Monitoreo de violaciones SLA
|
|
- [x] `calculate_sla_metrics`: Cálculo de métricas SLA
|
|
- [x] `send_sla_warnings`: Alertas de SLAs próximos a vencer
|
|
|
|
### Maintenance Tasks (`maintenance_tasks.py`)
|
|
- [x] `health_check`: Health check de workers
|
|
- [x] `cleanup_old_logs`: Limpieza de logs antiguos
|
|
- [x] `generate_weekly_reports`: Reportes semanales
|
|
- [x] `cleanup_temp_files`: Limpieza de archivos temporales
|
|
- [x] `database_maintenance`: Mantenimiento de BD
|
|
|
|
### Notification Tasks (`notification_tasks.py`)
|
|
- [x] `send_daily_digest`: Digest diario para agentes
|
|
- [x] `send_ticket_notifications`: Notificaciones de tickets
|
|
- [x] `send_system_alert`: Alertas del sistema
|
|
|
|
## Programación Automática (Celery Beat)
|
|
|
|
### Tareas Periódicas Configuradas
|
|
|
|
```python
|
|
# Cada 5 minutos
|
|
"check-sla-violations": check_sla_violations
|
|
|
|
# Diario a las 8:00 AM
|
|
"send-daily-digest": send_daily_digest
|
|
|
|
# Semanal los domingos a las 2:00 AM
|
|
"cleanup-old-logs": cleanup_old_logs
|
|
|
|
# Semanal los lunes a las 9:00 AM
|
|
"generate-weekly-reports": generate_weekly_reports
|
|
|
|
# Cada minuto (health check)
|
|
"worker-health-check": health_check
|
|
```
|
|
|
|
## Quick Start
|
|
|
|
### Desarrollo Local
|
|
|
|
```bash
|
|
# Instalar dependencias
|
|
pip install -r requirements.txt
|
|
|
|
# Variables de entorno (usar las del proyecto principal)
|
|
cp ../.env.example .env
|
|
|
|
# Ejecutar worker
|
|
celery -A app.celery worker --loglevel=info
|
|
|
|
# Ejecutar beat scheduler (en otra terminal)
|
|
celery -A app.celery beat --loglevel=info
|
|
|
|
# Monitoreo con Flower (opcional)
|
|
celery -A app.celery flower
|
|
```
|
|
|
|
### Con Docker
|
|
|
|
```bash
|
|
# Worker y Beat se ejecutan automáticamente con docker-compose
|
|
docker-compose up worker beat
|
|
|
|
# Ver logs
|
|
docker-compose logs -f worker
|
|
docker-compose logs -f beat
|
|
```
|
|
|
|
## Configuración
|
|
|
|
### Variables de Entorno Importantes
|
|
|
|
```bash
|
|
# Celery
|
|
CELERY_BROKER_URL=redis://redis:6379/0
|
|
CELERY_RESULT_BACKEND=redis://redis:6379/0
|
|
|
|
# Email
|
|
SMTP_HOST=mailhog
|
|
SMTP_PORT=1025
|
|
DEFAULT_FROM_EMAIL=noreply@servicemanager.local
|
|
|
|
# SLA
|
|
SLA_CHECK_ENABLED=true
|
|
SLA_WARNING_THRESHOLD=0.8
|
|
|
|
# Mantenimiento
|
|
LOG_RETENTION_DAYS=30
|
|
DIGEST_ENABLED=true
|
|
```
|
|
|
|
### Colas de Trabajo
|
|
|
|
- **default**: Tareas generales
|
|
- **email**: Envío de emails
|
|
- **sla**: Monitoreo SLA
|
|
- **maintenance**: Mantenimiento
|
|
- **notifications**: Notificaciones
|
|
|
|
## Monitoreo
|
|
|
|
### Logs Estructurados
|
|
|
|
Todos los workers utilizan structured logging con:
|
|
- Task ID único
|
|
- Correlation ID para tracking
|
|
- Contexto de tenant
|
|
- Métricas de performance
|
|
|
|
### Health Checks
|
|
|
|
```bash
|
|
# Health check manual
|
|
celery -A app.celery inspect ping
|
|
|
|
# Estadísticas de workers
|
|
celery -A app.celery inspect stats
|
|
|
|
# Tareas activas
|
|
celery -A app.celery inspect active
|
|
```
|
|
|
|
### Métricas
|
|
|
|
- Task execution times
|
|
- Success/failure rates
|
|
- Queue lengths
|
|
- Worker load
|
|
|
|
## Desarrollo
|
|
|
|
### Agregar Nueva Tarea
|
|
|
|
1. Crear función en módulo apropiado:
|
|
```python
|
|
@celery_app.task(bind=True, time_limit=300)
|
|
def new_task(self, param1: str, param2: int):
|
|
logger = get_logger(__name__)
|
|
# Implementation
|
|
return result
|
|
```
|
|
|
|
2. Registrar en `celery.py` si es periódica:
|
|
```python
|
|
beat_schedule = {
|
|
"new-periodic-task": {
|
|
"task": "app.tasks.module.new_task",
|
|
"schedule": crontab(minute=0, hour=9),
|
|
}
|
|
}
|
|
```
|
|
|
|
3. Agregar tests en `tests/`
|
|
|
|
### Retry y Error Handling
|
|
|
|
```python
|
|
@celery_app.task(
|
|
bind=True,
|
|
autoretry_for=(ConnectionError, TimeoutError),
|
|
retry_kwargs={'max_retries': 3, 'countdown': 60}
|
|
)
|
|
def reliable_task(self):
|
|
# Task que se reintenta automáticamente
|
|
pass
|
|
```
|
|
|
|
### Templates de Email
|
|
|
|
Los templates están definidos en código por ahora. En el futuro se moverán a base de datos para ser editables por tenants.
|
|
|
|
Templates disponibles:
|
|
- `ticket_created`
|
|
- `ticket_assigned`
|
|
- `ticket_resolved`
|
|
- `sla_warning`
|
|
- `sla_violation`
|
|
- `daily_digest`
|
|
- `system_alert`
|
|
|
|
## Testing
|
|
|
|
```bash
|
|
# Ejecutar tests
|
|
pytest
|
|
|
|
# Tests específicos de workers
|
|
pytest tests/test_tasks/
|
|
|
|
# Ejecutar tarea manualmente para testing
|
|
celery -A app.celery call app.tasks.email_tasks.send_email_task --args='["test@example.com", "Test Subject", "<h1>Test</h1>"]'
|
|
```
|
|
|
|
## Producción
|
|
|
|
### Configuración Recomendada
|
|
|
|
```bash
|
|
# Múltiples workers por queue
|
|
celery -A app.celery worker --loglevel=info --concurrency=4 --queues=email
|
|
celery -A app.celery worker --loglevel=info --concurrency=2 --queues=sla,maintenance
|
|
celery -A app.celery worker --loglevel=info --concurrency=1 --queues=default
|
|
|
|
# Beat scheduler (solo una instancia)
|
|
celery -A app.celery beat --loglevel=info
|
|
|
|
# Con systemd o supervisor para auto-restart
|
|
```
|
|
|
|
### Optimizaciones
|
|
|
|
- Pool de conexiones Redis
|
|
- Compresión de mensajes grandes
|
|
- Rate limiting por tarea
|
|
- Monitoring con Prometheus/Grafana
|
|
|
|
## Troubleshooting
|
|
|
|
### Problemas Comunes
|
|
|
|
1. **Tasks stuck in queue**:
|
|
- Verificar workers activos
|
|
- Revisar configuración de routing
|
|
|
|
2. **Memory leaks**:
|
|
- Configurar `worker_max_tasks_per_child`
|
|
- Monitorear uso de memoria
|
|
|
|
3. **Email delivery failures**:
|
|
- Verificar configuración SMTP
|
|
- Revisar logs de tareas email
|
|
|
|
4. **SLA false positives**:
|
|
- Verificar timezones
|
|
- Validar lógica de cálculo
|
|
|
|
### Debug
|
|
|
|
```bash
|
|
# Ejecutar worker en modo debug
|
|
celery -A app.celery worker --loglevel=debug
|
|
|
|
# Inspeccionar tareas fallidas
|
|
celery -A app.celery inspect failed
|
|
|
|
# Purgar queue
|
|
celery -A app.celery purge -Q queue_name
|
|
``` |