Skip to content

Integration Inventory

All external systems integrated with the Someli API.

🔐 Security cross-references. Several integrations have security findings flagged in security.md — hardcoded keys / tokens, no rotation procedure, secrets in source. Look for the 🔐 marker in the detailed profiles below.

Summary Table

# Service Package(s) Primary Purpose
1 AWS S3 aws-sdk, @aws-sdk/client-s3 File/image storage
2 AWS Secrets Manager aws-sdk Credential management
3 AWS Bedrock @aws-sdk/client-bedrock-runtime, @aws-sdk/client-bedrock-agent-runtime AI inference (Llama, Claude, Titan, Nova)
4 OpenAI openai AI text generation (GPT-4o-mini)
5 Google Gemini @google/generative-ai, @google/genai AI text/image generation
6 Google Vertex AI @google-cloud/vertexai RAG chatbot
7 Google Cloud Storage @google-cloud/storage Cloud file storage (knowledge base)
8 Paddle @paddle/paddle-node-sdk Payment / subscriptions (primary)
9 Stripe stripe Payment / subscriptions (still active despite "legacy" label)
10 SendGrid @sendgrid/mail Transactional email
11 Slack 🔐 slack, @slack/webhook Internal notifications
12 Facebook/Instagram Graph API passport-facebook, axios Social media publishing / analytics
13 LinkedIn API passport-linkedin-oauth2, axios Social media publishing / analytics
14 Twitter/X API @superfaceai/passport-twitter-oauth2, twitter, oauth-1.0a Social media feed pulling
15 TikTok API passport-tiktok-auth, axios Social media publishing / analytics
16 Polotno 🔐 polotno-node Design template rendering
17 Leonardo AI axios (REST) AI image generation (fallback)
18 Pexels axios (REST) Stock photos
19 Pixabay axios (REST) Stock photos
20 Unsplash unsplash-js Stock photos (verify if active)
21 Puppeteer/Chromium puppeteer, puppeteer-core Web scraping
22 FFmpeg fluent-ffmpeg, ffmpeg-static Video processing
23 Sharp sharp Image processing
24 RSS Feeds rss-parser, feedsub Content aggregation
25 Cheerio cheerio HTML parsing
26 Chaskiq crypto (built-in) Customer chat widget
27 Socket.IO socket.io Real-time communication
28 MySQL / MariaDB 🔐 mysql, mysql2, sync-mysql Primary database
29 Google OAuth passport-google-oauth20 User authentication
30 GitHub OAuth passport-github2 User authentication
31 Natural NLP natural Text vectorization for RAG
32 Vertex AI REST axios Direct Gemini API calls
33 Google Cloud RAG Python scripts Knowledge base management
34 Impact.com axios (REST) Affiliate / partner conversion tracking
35 GoHighLevel (GHL) axios (inferred — REST) CRM appointment integration
36 Microsoft Teams (runtime) request (teamsPost) Operational notifications from background jobs
37 Microsoft Teams (CI/CD) office365ConnectorSend (Jenkins plugin), curl (GitHub Actions) Deploy-pipeline notifications

Detailed Integration Profiles

1. AWS S3

Configuration: AWS_ACCESS_KEY, AWS_SECRET_ACCESS_KEY, S3_BUCKET, S3_Region, S3_BUCKET2, S3_Region2

Key Files: - helper/helper.js -- S3 client initialization and utilities - helper/files_upload.js -- s3UploadFunction - 70+ job_*.js files for media uploads/downloads - routes/auth.js, routes/routes.js -- S3 file operations

Operations: Upload images, upload JSON template files, download files, list objects, put objects, head objects (existence checks), store generated images, store stock photos, store RAG documents

2. AWS Secrets Manager

Configuration: AWS_ACCESS_KEY, AWS_SECRET_ACCESS_KEY, S3_Region, GCS_SECRET_NAME

Key Files: - helper/files_upload.js - helper/functionsForAi/cloudRag.js

Operations: getSecretValue -- fetches Google Cloud service account credentials for GCS and Vertex AI authentication

3. AWS Bedrock Runtime (LLM Inference)

Configuration: AWS_BEDROCK_KEY, AWS_BEDROCK_SECRET, AWS_BEDROCK_MODEL, AWS_BEDROCK_REGION

Key Files: - helper/aiLogics.js -- Llama3, Claude, Nova model invocation - helper/helper.js -- Titan Image Generation - helper/catContentResults.js -- content categorization - helper/ragProcess.js, helper/getRagData.js, helper/getRagDataTopics.js -- RAG embeddings/generation - helper/webScraping.js, helper/webscrapeHomePage.js, helper/webscrapeBni.js -- content summarization - routes/auth.js -- Bedrock Agent Runtime for chatbot - job_first_comment.js -- first comment generation

Operations: - InvokeModelCommand -- invoke Llama 3, Claude (Anthropic), Amazon Titan Image Generator, Amazon Nova Pro - ConverseCommand -- multi-turn conversation with Nova models - InvokeAgentCommand -- invoke Bedrock Agents (agent ID PJGBZ749AC)

Models Used: Llama 3, Claude (Anthropic via Bedrock), Amazon Titan Image Generator, Amazon Nova Pro (amazon.nova-pro-v1:0)

4. OpenAI

Configuration: OPENAI_API_KEY

Key Files: - helper/aiLogics.js -- openAiLogic (chat completions), fetchOpenAIResponse - job_dynamic_post.js, job_post_validation.js, job_pending_post.js, job_text_color_change.js, job_generation_complete.js, job_color_check.js

Operations: chat.completions.create (GPT-4o-mini and other models), completions.create (legacy)

5. Google Gemini / Generative AI

Configuration: GOOGLE_API_KEY

Key Files: - helper/functionsForAi/gemini.js -- getGeminiResult (with Google Search tool), getGeminiWithoutGoogleSearch - helper/aiLogics.js -- geminiLogic (Vertex AI API with OAuth) - agents/conversationAgent.js, agents/inputParserAgent.js, agents/researchAgent.js, agents/profileAgent.js

Operations: generateContent, generateContentStream (streaming), image analysis with inlineData, Google Search tool integration

6. Google Vertex AI

Configuration: CLOUD_PROJECT_ID, CLOUD_LOCATION, GCS_SECRET_NAME

Key Files: - helper/functionsForAi/cloudRag.js -- cloudRagResult - routes/routes.js -- /chat-bot endpoint

Operations: generateContent with vertexRagStore retrieval tool, startChat for multi-turn conversations, model gemini-2.5-flash

7. Google Cloud Storage (GCS)

Configuration: CLOUD_BUCKET_NAME, GCS_PATH, DISPLAY_NAME, GCS_SECRET_NAME

Key Files: - helper/files_upload.js -- gcsUploadFunction

Operations: bucket.file().save() -- upload JSON files to GCS for knowledge base storage

8. Paddle (Payment Processing - Primary)

Configuration: PADDLE_API_KEY, PADDLE_WEBHOOK_KEY, PADDLE_WEBHOOK_KEY1, PADDLE_API_DOMAIN, PADDLE_ENV, plus test equivalents

Key Files: - routes/paddle.js -- subscription and customer list APIs - routes/routes.js -- Paddle SDK initialization, webhook routes - helper/helper.js -- Paddle SDK initialization - helper/constants.js -- PADDLE_DATA configuration - job_paddle_webhook.js -- webhook processing - job_paddle_Admin_webhook_mail.js -- admin webhook notifications - job_invoice_paid.js -- invoice processing

Operations: List subscriptions, list customers, webhook handling for subscription events (customer.created, subscription.updated, transaction.completed, payment_method.saved), invoice processing

9. Stripe (Payment Processing - Legacy)

Configuration: Stripe key loaded dynamically (from database or env)

Key Files: - routes/auth.js -- extensive Stripe integration for subscriptions - routes/routes.js -- Stripe initialization, /stripe_webhooks - get_sub_status.js -- subscription status check

Operations: customers.create, customers.search, customers.createSource, subscriptions.create, subscriptions.retrieve, subscriptions.update, subscriptions.list, webhook handling

10. SendGrid (Email)

Configuration: SENDGRID_API_KEY, FROM (sender email)

Key Files: - helper/helper.js -- mail service setup - job_send_mail.js -- email sending queue processor - job_generation_complete.js, job_paddle_Admin_webhook_mail.js, job_revoke_token_mail.js, job_new_auto_disconnect.js, need_to_reconnect_sm.js

Operations: mailService.setApiKey(), mailService.send() -- sending emails with dynamic templates (template IDs like d-47c22927...)

11. Slack (Notifications) 🔐

Configuration: Hardcoded Slack bot token and channel ID in source files. Not in .env.

🔐 Security findings (cross-referenced from security.md §Hardcoded secrets and enterprise-readiness §1.5 Hard Truth #2):

  • Bot token xoxb-3144030948916-... appears verbatim in multiple job_*.js files (e.g., job_color_check.js, job_auto_disconnect.js).
  • Channel ID C05TS9AHBH6 is hardcoded.
  • Both are in git history. Treat as compromised; rotate immediately per Phase 0a — This Week.
  • Rotation requires editing every file that references the token (no shared helper). Recommended fix: move to SLACK_BOT_TOKEN and SLACK_CHANNEL_ID in .env, add a single helper/slack.js wrapper.

Key Files: - slack_notifications.js -- scheduled reports using slack package - slack_notifications_post_disconnect.js -- @slack/webhook IncomingWebhook for disconnect reports - Many job_*.js files for operational alerts

Operations: bot.chat.postMessage, webhook.send -- posting messages to Slack channels for system monitoring (feed status, content planner status, post scheduling issues, job failures)

Reliability: No retries, no error handling beyond local try/catch / console.log. A failed Slack post is silently dropped.

12. Facebook / Instagram Graph API

Configuration: FACEBOOK_APP_ID, FACEBOOK_APP_SECRET

Key Files: - middlewares/passport.js -- Facebook OAuth strategy - routes/social.js -- OAuth flow - job_facebook_publish.js, job_facebook_video_publish.js, job_facebook_carousal_publish.js, job_facebook_link_publish.js -- publishing - job_instagram_publish.js, job_instagram_video_publish.js, job_instagram_carousal_publish.js -- publishing - job_ig_permalink.js -- permalink retrieval - job_refresh_fb_token.js, job_weekly_refresh_fb_token.js -- token refresh - dashboard/services/job_account_insights.js, dashboard/services/job_post_insights.js -- analytics

Operations: Publish posts (images, videos, carousels, links), upload media, fetch page insights, fetch post insights, refresh long-lived tokens, retrieve permalinks

13. LinkedIn API

Configuration: LINKEDIN_CLIENT_ID, LINKEDIN_CLIENT_SECRET

Key Files: - middlewares/passport.js -- LinkedIn OAuth strategy - job_linkedin_publish.js, job_linkedin_video_publish.js, job_linkedin_carousal_publish.js, job_linkedIn_link_publish.js, job_linkedin_repost_publish.js -- publishing - job_refresh_In_token.js -- token refresh - dashboard/services/job_account_insights.js, dashboard/services/job_post_insights.js -- analytics

Operations: Publish posts (text, images, videos, carousels, links, reposts), upload media, fetch organization insights, token refresh

14. Twitter / X API

Configuration: TWITTER_CLIENT_ID, TWITTER_CLIENT_SECRET

Key Files: - middlewares/passport.js -- Twitter OAuth2 strategy - routes/social.js -- OAuth flow - job_cp_feed.js, job_cp_feed_video.js -- feed pulling (v1 API with OAuth 1.0a) - job_auto_disconnect.js, job_revoke_token_mail.js -- disconnect handling

Operations: OAuth authentication, fetch user tweets, handle disconnections

15. TikTok API

Configuration: TIKTOK_CLIENT_ID, TIKTOK_CLIENT_SECRET

Key Files: - middlewares/passport.js -- TikTok OAuth strategy - routes/auth.js -- TikTok API interactions - helper/social.js -- tiktokRefresh token refresh - job_tiktok_content_publish.js -- content publishing - job_tiktok_refresh.js -- token refresh - dashboard/services/job_account_insights.js, dashboard/services/job_post_insights.js -- insights

Operations: OAuth token refresh, video upload, fetch user info, fetch video insights

16. Polotno (Design/Image Generation) 🔐

Configuration: Hardcoded API key 'FXZvloSJvAe09-bdR9iC' in source. Not in .env.

🔐 Security findings (cross-referenced from security.md and media-processing.md §Polotno):

  • The render key is hardcoded in 49 source files.
  • Rotation = sweeping search-and-replace. Recommended fix: move to POLOTNO_KEY in .env.
  • Treat as compromised; rotate per Phase 0a — This Week.

Operational findings:

  • No instance pooling. 97 call sites do createInstance() (each spawns a fresh Chromium ~100 MB RSS). Only 75 of those call instance.close() afterward — 22 leak sites at the call-site level, ~31 files with createInstance and no close in the same function. This is the single biggest source of memory pressure on the worker fleet.
  • helper/postValidation.js:checkVisibity is a confirmed leak site. This function is called from many of the validation jobs and never closes the Polotno instance it spawns — the leak compounds across calls. Adding await instance.close() at the end of the function would address it. See media-processing.md.
  • 4 callers use the newer createBrowser() pattern that pools browsers; migrating the other 93 is on the readiness roadmap.
  • See media-processing.md §Polotno for the rendering pipeline detail.

Key Files: - routes/auth.js, routes/routes.js -- createInstance, createBrowser - helper/postValidation.js -- post image validation - 50+ job_*.js files for design rendering

Operations: createInstance (spawn headless Chromium with Polotno preloaded), jsonToImageBase64(json) (render to PNG base64), imageToPdf (compose multi-page carousels), createBrowser (pooled-browser variant), instance.close (release browser).

17. Leonardo AI (Image Generation)

Configuration: LEONARDO_KEY

Key Files: - helper/helper.js -- leonardoImageProcess - job_media_image_correction.js, job_post_image_correction.js, job_user_media_correction.js, job_user_media_correction_error_image.js, job_validate_errorMedia.js

Operations: POST to https://cloud.leonardo.ai/api/rest/v1/generations -- generate images (fallback when Amazon Titan fails), GET to retrieve generated images

18. Pexels (Stock Photos)

Configuration: PEXELS_API_KEY

Key Files: - helper/stockImage.js -- getStockImage - routes/routes.js -- /getpexelsimages, /searchpexelsimages - job_dynamic_post.js

Operations: GET https://api.pexels.com/v1/search -- search photos, GET https://api.pexels.com/v1/curated -- curated photos

19. Pixabay (Stock Photos)

Configuration: PIXABAY_API_KEY

Key Files: - helper/stockImage.js -- getStockImage - pixabay_s3upload.js -- bulk upload utility

Operations: GET https://pixabay.com/api/ -- search photos with category/keyword filtering

20. Unsplash (Stock Photos - Disabled)

Configuration: API key from tApiKeys table

Key Files: - helper/stockImage.js -- code exists but is commented out

Status: Currently disabled in production

21. Puppeteer / Chromium

Key Files: - helper/webscrapeHomePage.js -- Puppeteer-based web scraping - helper/webscrapeBni.js -- BNI website scraping

Operations: Headless browser for scraping JavaScript-rendered pages to extract company information

22. FFmpeg (Video Processing)

Key Files: - routes/routes.js -- video processing and thumbnail generation

Operations: Video thumbnail extraction, format conversion, metadata probing

23. Sharp (Image Processing)

Key Files: 35+ files across routes, helpers, and jobs

Operations: Resize, crop, format conversion (PNG/JPEG), composite images, metadata extraction, buffer transformations

24. RSS Feed Parsing

Key Files: - job_rss.js -- FeedSub for real-time monitoring - job_check_daily_account_news.js, job_check_daily_rss_feed.js, job_check_rss_feed_available.js -- RssParser

Operations: Subscribe to feeds, parse feed items, validate feed URLs

25. Cheerio (HTML Parsing)

Key Files: - helper/webScraping.js -- content extraction - job_check_daily_account_news.js, job_check_daily_rss_feed.js

Operations: Load HTML, extract text content, parse DOM elements (for non-JavaScript pages)

26. Chaskiq (Customer Chat Widget)

Configuration: CHASKIQ_SECRET

Key Files: - routes/routes.js -- HMAC signing for identity verification

Operations: crypto.createHmac('sha256', CHASKIQ_SECRET) -- sign user email for chat widget authentication

27. Socket.IO (Real-time Communication)

Key Files: - server.js -- initialization and event handling

Operations: io.on('connection'), io.emit('update') -- broadcasting real-time updates to connected clients

28. MySQL / MariaDB 🔐

Configuration: host, user, password, database, dbPort, connectionLimit (all lowercase env-var names — see findings below)

🔐 Security / hygiene findings (cross-referenced from security.md §Secrets Management and data-model.md):

  • Lowercase env-var name user collides with the shell $USER variable. If .env fails to load, process.env.user silently falls back to the OS user (ubuntu on Lightsail). Recommended: rename to DB_USER / DB_PASSWORD / DB_HOST / DB_NAME / DB_PORT.
  • No SSL/TLS in transit. Default driver setting ssl: false. Cross-region traffic between us-west-2 (app) and us-west-1 (RDS) is unencrypted at the protocol level. Required for SOC 2 / ISO 27001.
  • No fail-fast validation. Missing env vars produce undefined connections and surface as ER_ACCESS_DENIED_ERROR at first query. Add a startup assertion.
  • Driver defaults mask config errors. modules/dbDriver/lib/mysql.js falls back to localhost / root / '' (empty password) if any field is missing. On a dev laptop with MySQL running this can succeed by accident.
  • tMember.auth_pass defaults to literal 'Welcome!1' (DB-level default). Investigate any code path that inserts a member without explicitly setting this column. See data-model.md §1 audit findings.

Operational findings:

  • Three drivers in use (mysql, mysql2, sync-mysql). Consolidation to mysql2/promise is on the readiness roadmap — see Enterprise Readiness §5.4.
  • No connection pool. Each of ~892 call sites constructs its own connection; with 100+ workers, RDS connection limits are a real risk during restart storms.
  • Zero FOREIGN KEY constraints in the schema. All 111 indexes are plain KEY/PRIMARY KEY declarations. Relationships are application-level only — orphan rows are possible.

Key Files: Virtually every file (120+). Core driver at modules/dbDriver/lib/mysql.js

Access Patterns: - mysql package (async callbacks) via App.db and actions/actions.js - sync-mysql (synchronous blocking) via con in routes and jobs — blocks the event loop - mysql2/promise (async/await) in newer job files and dashboard services — the recommended path going forward

29. Google OAuth

Configuration: GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET

Key Files: - middlewares/passport.js -- strategy - routes/social.js -- callback

Operations: OAuth2 authorization flow for user login

30. GitHub OAuth

Configuration: GITHUB_CLIENT_ID, GITHUB_CLIENT_SECRET

Key Files: - middlewares/passport.js -- strategy - routes/social.js -- callback

Operations: OAuth2 authorization flow for user login

31. Natural NLP

Key Files: - helper/ragProcess.js -- TF-IDF vectorization

Operations: TF-IDF term frequency vectorization, cosine similarity for local RAG document retrieval

32. Vertex AI REST (Direct API Calls)

Configuration: OAuth token stored in tApiKeys table (id=6), refreshed by job_vertex_token.js

Key Files: - helper/aiLogics.js -- geminiLogic - job_vertex_token.js -- token refresh

Operations: POST to https://us-central1-aiplatform.googleapis.com/v1/projects/someli-412206/locations/us-central1/publishers/google/models/{model}:streamGenerateContent

33. Google Cloud Knowledge Base / RAG Corpus (via Python)

Key Files: - update_cloud_knowledge_base.js -- calls update_cloud_knowledge_base.py - user_specific_content.js -- calls gemini_content.py

Operations: Corpus creation, file importing into RAG corpus, AI content generation via Python Google AI SDK (uses child_process.execSync)

34. Impact.com (Affiliate Conversion Tracking)

Configuration: - IMPACT_ACCOUNT_SID — Impact account / advertiser SID - IMPACT_AUTH_TOKEN — Impact API auth token - IMPACT_BASE_URL — defaults to https://api.impact.com (hardcoded in source) - IMPACT_CAMPAIGN_ID — campaign identifier sent on every conversion

Key Files: - routes/auth.js (~line 4297) -- IMPACT_CONFIG block, conversion-posting logic

Operations: - POST to https://api.impact.com/Advertisers/{ACCOUNT_SID}/Conversions with HTTP Basic auth (base64-encoded ACCOUNT_SID:AUTH_TOKEN). - Records the conversion locally in tImpactConversion (columns: click_id, memberId, event_date, order_id). - Optional item-level details written to tImpactItemDetails.

Use case: Affiliate / partner attribution. When a referred user converts (subscribes / pays), someli notifies Impact.com so the referring partner can be credited.

Notes: - The "internal" affiliate tables (tAffiliatePartner, tAffiliatePricing) are unreferenced in code — see data-model.md §7. Impact.com is the active affiliate path. - No retry / backoff on conversion-post failure. A network blip = a lost conversion. - Auth token in .env (correctly handled — not hardcoded), but the env-var names lack the DB_* style namespace.

35. GoHighLevel — GHL (CRM Appointments)

Configuration: No env vars visible — webhook-based ingestion. The integration appears to receive webhooks from GHL, not actively call out.

Key Files: - routes/auth.js (~line 120) — query joining tGhl_Appointments with tMember - routes/routes.js (~line 17829, ~17843) — appointment lookup by email and INSERT on incoming webhook

Schema: - tGhl_Appointments(Email, contact_source, fullName, phone, appointment_time, appointment_status, status, isDeleted, ...)

Operations: - Receive appointment webhooks from GHL → INSERT into tGhl_Appointments. - On user lookup, JOIN tGhl_Appointments to tMember via g.Email = m.username to surface appointment history per user.

Use case: GoHighLevel is a CRM platform. Sales / support appointments booked there are mirrored into someli so the API can show appointment history alongside other user data.

Notes: - The webhook endpoint receiving GHL data needs verification — find the route handler that does the INSERT. - No signature validation visible (GHL webhooks support HMAC verification — confirm whether it's used here). - LEFT JOIN tMember ON g.Email = m.username confirms the previous data-model finding that tMember.username is the email-shaped login column, not a separate email column.

36. Microsoft Teams — Runtime Notifications (Background Jobs)

Configuration: Webhook URL passed in code (verify whether hardcoded or env-driven — likely env-driven via request options).

Key Files: - job_auto_post.js -- teamsPost = request(...) - job_schedule_pending.js -- same pattern - job_content_schedule.js -- same pattern

Operations: POST a JSON payload to a Teams Incoming Webhook URL using the request package (deprecated; this is one of the Tier 4 dep-consolidation candidates).

Use case: Operational notifications from background jobs — content-pipeline status updates, scheduling failures, etc. — surfaced to a Teams channel.

Reliability: - Wrapped in a single try/catch that logs "Teams notify error" and continues. Failed notifications are silently dropped. - No retries. - No structured payload — plain JSON body, no Adaptive Cards (in contrast to the deploy notifications below).

Note: Verify whether this overlaps with the Slack notification path (#11) or whether they target different audiences. Same operational events going to two different channels is a maintenance burden.

37. Microsoft Teams — Deploy Notifications (CI/CD)

Configuration: - Jenkins: TEAMS_WEBHOOK credential ID (commented out in current Jenkinsfile); uses office365ConnectorSend plugin - GitHub Actions: DEV_API_WEBHOOK_URL repository secret (used in .github/workflows/dev-api-deploy.yml)

Key Files: - Jenkinsfile (lines 173, 197) -- office365ConnectorSend for success / failure messages - .github/workflows/dev-api-deploy.yml (lines 220, 283) -- curl POST with Adaptive Card JSON

Operations: Send richly-formatted Adaptive Card messages to Teams on each deploy: - Success: author, email, PR link, commit SHA, message, time, files changed, additions / deletions, region, "View Workflow Run" / "View Pull Request" buttons. - Failure: author, message, SHA, status, last ~20 lines of build log, "View Workflow Run" button.

Use case: Real-time deploy visibility for the dev_api branch. Both pipelines target the same notification surface.

Notes: - This integration is more sophisticated than the runtime notifications (#36) — Adaptive Cards with structured fact-sets and buttons. - See deployment.md §6.1 Two CI systems on the same branch for the related concern that both pipelines fire on every push.

Cross-Doc Notes

The findings in this doc align with broader audits documented elsewhere:

When adding a new third-party integration, the recommended hygiene checklist is:

  1. Secret in .env (never hardcoded in source).
  2. Wrapped in a single helper module (no copy-paste of the integration setup across files).
  3. Has retry / backoff for transient failures.
  4. Failures logged with enough context to debug (correlation ID, request payload identifiers).
  5. Documented here with all four sections (Configuration / Key Files / Operations / Notes).