Skip to main content

Worker Engine

Architecture: Hybrid Hangfire + SQS Strategy​

The background worker operates in a highly-concurrent hybrid capacity targeting reliability and idempotency:

LayerEnginePurpose
Scheduled JobsHangfire (PostgreSQL Backed)Recurring cron-based polling routines (Price Sync, Metrics Refresh, News Batching).
Event HandlersAmazon SQS + RedisReactive tasks triggered by API events (price alerts, low holdings, news on-demand).
Real-time PushSignalR + Redis BackplaneInstant delivery of detected alerts from Worker to UI.

Scheduled Job Catalog​

Running inside InventoryAlert.Worker, driven by Hangfire cron schedules.

Schedules are configurable via WorkerSettings.Schedules.* (with sensible defaults in code and environment overrides via appsettings*.json).

InventoryAlert uses a total of 6 background jobs (5 recurring via Hangfire + 1 continuous SQS listener), optimized for active ticker scoping and Finnhub 60 req/min rate limits.

JobSchedule settingFinnhub EndpointKey duty
SyncPricesJob*/15 * * * */quoteActive symbol fetch → insert PriceHistory → evaluate AlertRule → insert Notification → SignalR push
SyncStockFundamentalsJob10 6 * * */metric, /earnings, /recommendation, /insider-transactionsConsolidated daily sync for basic financials, quarterly earnings, analyst recommendations, and SEC insider trades for active symbols
NewsSyncJob5 */2 * * */news & /company-newsConsolidated market + active company news batch sync to DynamoDB read models
CleanupPriceHistoryJob20 2 * * *—Deletes PriceHistory rows older than 1 year to keep PostgreSQL lean
ProcessQueueJobContinuous Poller—Native SQS poller + router + Redis idempotency
KeepAliveJob*/10 * * * *—Self-pings http://127.0.0.1:8080/healthz to guarantee 24/7 Render free-tier container uptime

The Optimized Evaluation Pipeline: SyncPricesJob​

The current architecture uses high-concurrency quote fetching and batch rule evaluation:

1. Collect active TickerSymbols from StockListing.
2. PART 1 (Parallel Sync):
- Fetch Finnhub /quote in parallel (MaxDegreeOfParallelism=5).
- Batch insert into PriceHistory using AddRangeAsync.
3. PART 2 (Batch Alert Check):
- Fetch all active AlertRules for processed symbols in ONE query.
- Evaluate breach conditions (direct comparison or cost-basis math).
4. PART 3 (Real-time Notification):
- Batch insert Notification records.
- For each breach: Push via `IAlertNotifier` (SignalR via Redis backplane).
5. COMMIT: Single SaveChangesAsync call for all history, notifications, and rule updates.

Key Performance Highlights​

  • Parallel I/O: Reduces sync time by fetching multiple quotes simultaneously.
  • N+1 Avoidance: Repository-level batch fetching for alert rules.
  • Backplane Delivery: Alert detection in Worker triggers Hub delivery in Api instantly via Redis.

Event Handlers (SQS Topology)​

Located in IntegrationEvents/Handlers, invoked by IntegrationMessageRouter after messages are pulled from SQS by ProcessQueueJob.

HandlerSQS Event TypeRole
MarketPriceAlertHandlerinventoryalert.pricing.price-drop.v1Evaluate rules for a specific symbol + price payload and push notifications
LowHoldingsHandlerinventoryalert.inventory.stock-low.v1Persist + push a holdings notification for a specific user + symbol
NewsSyncJob (via Router)inventoryalert.news.sync-requested.v1Enqueue the consolidated news sync job
DefaultHandler* (unmatched)Log + acknowledge (prevents poison-message blockage)

Notes:

  • inventoryalert.news.company-sync-requested.v1 is defined in EventTypes, but is not currently routed by IntegrationMessageRouter.

Health Monitoring​

The worker is equipped with health check endpoints exposed via HTTP (port 8080 internally, 8081 in Docker Compose):

  • Liveness & Readiness: GET /health
  • Dependency Checks: Verified connectivity to PostgreSQL on startup.
  • Docker Integration: Configured with interval: 10s and retries: 5 to ensure background services stay responsive.