System architecture

How the frontend, backend, database, schedulers, and providers fit together.

ReachConvert is two apps. The Next.js workspace runs in the operator's browser. The NestJS backend owns everything else: sign-in, encrypted provider keys, MongoDB, sending, scheduled jobs, signal polling and live phone calls. Almost all traffic is REST under /api. Live calls are the exception: the phone provider streams audio into the backend over a WebSocket, and the backend bridges it to Gemini Live.

The whole system

Product pages reach the backend through one client, src/lib/api.ts, which attaches the access token. On the backend, main.ts sets the /api prefix and hands the phone provider's WebSocket upgrades to the realtime gateway. Every other request passes the global AuthGuard into a feature module, and every module stores data through MongoService.

BROWSER · AGENTREACH-FRONTEND (NEXT.JS)Product pagesapp/(dashboard)/*AuthGuard + AppShellwrap every routeapi.tsAdds the access token;on a 401, refreshes itand retries oncelocalAuth.tsAccess and refresh tokens,profile and theme inlocalStorageKEYBrowserBackendGeminiOutside serviceCron jobDatabaseRequestAudio / socketWebhookREST /api · Bearer tokenAGENTREACH-BACKEND · NESTJS 11main.tscreateApp()/api prefix · CORS_ORIGINS allow-list · 50 MB request bodiesSwagger at /docs, off in production unless ENABLE_SWAGGERUpgrades /twilio/stream and /plivo/stream for live calls; drops any other socketRESTupgradeGlobal AuthGuardChecks the Bearer tokenand loads the user;@Public() routes skip itSchedulers@CronEvery minute: launchcampaigns that are dueEvery 6 h: poll watchesRealtime gatewayPhone audio ↔ Gemini LiveRuns tools, writes thetranscript, enforces limitsrequest + userlaunch · pollbot knowledgeFeature modulessrc/<module>/authsign-in, refresh, profilesettingsencrypted keys, testscontactscontacts, directories, CSVtemplatesemail copy, AI draftsemail-campaignsSES sendingai-calling-botspersonas, knowledgecalling-campaignsdial, webhooks, recordingssignalswatches, playbooks, reviewhistorysent email, past callsanalyticsdashboard metricsread / writeMongoServicemongo.service.tsPrisma-style delegates over 16 Mongoose models, shared by every module and the gatewayMongooseMongoDBDATABASE_URLOne database for the workspace. Every account reads and writes the same data.Gemini Livegemini-3.1-flash-live-previewOne voice session per call,plus a preflight before dialingPCM audioTwilio or PlivoPlaces calls, sends signedwebhooks, streams call audiocallProvider in Settingsμ-law 8 kHzdialwebhooksAWS SESSendEmail · SendRawEmailsendGemini textgemini-flash-lite-latestgeneratePublic sourcesNews RSS · SEC EDGAR · job boardspoll
Schedulers call the same services the REST routes use, so a scheduled launch and a button click run the same code. The phone provider meets the backend three ways: the backend dials it over REST, it calls back on signed webhooks, and it streams the call's audio into the gateway. Any other WebSocket upgrade is dropped, so those media streams are the only sockets the backend accepts.
Where long work runs: nothing slow happens inside the request that starts it. Launching an email campaign marks it RUNNING and sends in the background, one recipient every 250 ms. Launching a calling campaign returns once each call is placed, and the conversations happen later on the gateway as contacts pick up. All of it runs inside the backend process. Deployed as a Vercel function, the backend still serves REST, but its WebSocket upgrade handler never receives traffic, so live calls need a long-running deploy such as Render or a local server.

Who can call what

The guard is registered globally, so every route is private unless it opts out with @Public(). Opting out does not mean unchecked. Routes the phone provider calls carry their own proof instead of a user token.

Needs a Bearer access token

  • Every route by default. The guard verifies the token, then loads the user, so a deleted account is refused.
  • Settings, where the provider keys live. Responses always return those keys masked.
  • Call recordings, which the backend proxies from the provider instead of linking to them.

Public, but signed per call

  • The sign-in endpoints: register, login, refresh, forgot-password and reset-password.
  • Twilio and Plivo answer, status and recording webhooks under /api/calling-campaigns.
  • The /twilio/stream and /plivo/stream media sockets.
  • Every URL handed to the provider carries an HMAC of the call id. A webhook with a bad token gets a 403, and a socket with a bad token is closed at its start frame.
Sessions in the browser: localAuth.ts keeps the access and refresh tokens in localStorage. When a request comes back 401, api.ts calls /auth/refresh once, saves the new pair and retries the original request. A second 401 surfaces as a session-expired error.

Which module does what

Each feature is a NestJS module under AgentReach-backend/src. The kind says how work reaches it: a REST route, a provider webhook, a media socket or a cron job.

ModuleKindOwnsCalls out toCollections
authauth/restSign-up, login, token refresh, password reset, profilenoneUser
settingssettings/restProvider keys, encrypted at rest, and a test button per providerSES, Twilio, Plivo, Gemini (tests only)SystemSettings
contactscontacts/restContacts, directories, CSV parsing and importnoneContact, ContactDirectory
templatestemplates/restEmail templates, AI drafts, reference PDFsGemini textTemplate
email-campaignsemail-campaigns/restCampaigns, recipients, schedule, launch, relaunchAWS SESEmailCampaign, EmailCampaignContact
ai-calling-botsbot/restCalling personas and their knowledge, from text or a PDFnone, embeddings are computed locallyAiCallingBot, AiCallingBotEmbedding
calling-campaignscalling-campaigns/restwebhookCampaigns, dialing, provider callbacks, recordingsTwilio or Plivo, Gemini Live preflightCallingCampaign, CallHistory
realtime-callingrealtime-calling/realtimeThe media sockets and the live Gemini sessionGemini LiveCallHistory
signalssignals/restcronWatches, collectors, playbooks, review queueNews RSS, EDGAR, job boards, Gemini textCompanyWatch, Signal, SignalMatch, Playbook, TriggeredOutreach
schedulerscheduler/cronLaunches campaigns whose scheduled time has passednone, it calls the campaign servicesEmailCampaign, CallingCampaign
historyhistory/restSent email and past callsnonereads only
analyticsanalytics/restDashboard metricsnonereads only

One outbound call, end to end

A calling campaign holds one CallHistory row per contact. Launching it dials each contact in turn. The conversation runs later, on the realtime gateway, which bridges the phone line to a Gemini Live session.

KEYBackend codeGemini LivePhone providerGateway toolDatabaseCron jobRequestLive audioWebhookDashboardPOST /:id/launchor /:id/relaunchCampaign scheduler@Cron every minuteonce scheduledAt passes0 · Launchlaunch()Needs Twilio or Plivo keys and contacts with phonenumbers. Keys containing "mock" or "test" simulatethe calls instead. Relaunch resets every row first.each contact, one after another1 · PreflightGeminiLiveSessionWrapperOpens a Gemini Live session and waits for setup,so a bad key or model fails this row with anerror before the phone ever rings.setup, then closeready2 · DialcreateTwilioCall()Places the call through the provider's REST API.The answer and status URLs carry an HMAC ofthe call id. The row becomes QUEUED.RESTTwilio or PlivoRings the contact. On answer,calls the signed webhook, thenopens a media socket to thebackend.μ-law 8 kHz both waysStatus and recording callbacksupdate the same CallHistory rowcontact answers3 · Answer webhookhandleTwilioAnswer()Marks the row IN_PROGRESS and returns<Connect><Stream> pointing at the socket:/twilio/stream?callId=…&token=…answerprovider opens the socket4 · Live callRealtimeCallingGatewayChecks the token, connects Gemini as soon as thesocket opens and asks for the greeting, holdingits audio until the stream starts.in μ-law 8 kHz → PCM16 16 kHz → Geminiout Gemini PCM16 24 kHz → μ-law 8 kHzAn adaptive noise gate decides when the callerspoke; barge-in clears queued playback.Hangs up at 10 min, after two silent 15 s checks,or past the token budget. Reconnects once with aresumption handle if the Gemini socket drops.media socketend_callHangs up after a closing line,only once the contact has spokenfetch_contextSearches the bot's knowledge;gives up after 2.5 sBot knowledgeAiCallingBotEmbedding384-dim hashed vectors, top 5audioGEMINI LIVEPreflight sessionConnects, waits for setup,then closesModel, set per campaign:gemini-3.1-flash-live-preview(default)gemini-2.5-flash-native-audio-preview-12-2025API key from Settings,else GEMINI_API_KEYCall sessionNative audio both ways,tool calls, transcripts,session resumptionsocket closes5 · CompletecompleteCall()Writes COMPLETED and the transcript. The campaigncompletes once no call is pending or live.CallHistoryone row per contactStatus, transcript and recording for each call;History and the dashboard read it from here.
Steps 0 to 2 run inside the launch request, once per contact. Steps 3 to 5 run when that contact answers, driven by the provider. The preflight session and the call session are separate Gemini connections: the first only proves the key and model work, then closes. Twilio Media Streams and Plivo Audio Streams share one JSON envelope and the same μ-law 8 kHz audio, so a single gateway serves both. Only the outbound frame shapes differ.
StepKindRuns inTalks toCallHistory after
0 · Launchlaunch()codeThe launch requestnonePENDING; relaunch resets every row
1 · PreflightpreflightGeminiLiveCall()modelThe launch requestGemini Live, setup onlyFAILED with the error, if setup fails
2 · DialcreateTwilioCall() / createPlivoCall()providerThe launch requestTwilio or Plivo RESTQUEUED, with the provider call id
3 · AnswerhandleTwilioAnswer()webhookPublic webhook, signedReturns the stream XMLIN_PROGRESS
4 · Live callRealtimeCallingGatewayrealtimeThe media socketGemini Live call sessionEach turn appended to the transcript
end_callcall-tools.tstoolA Gemini tool callnoneendCallReason; hangs up after the last audio
fetch_contextcall-tools.tstoolA Gemini tool callBot knowledge, 2.5 s timeoutnothing
5 · CompletecompleteCall()codeThe gatewaynoneCOMPLETED; the campaign completes when no call is left
Status, recordinghandleTwilioStatus()webhookPublic webhook, signednoneProvider status, duration, recording
Why the first words come quickly: the stream URL carries the call id, so the gateway starts connecting to Gemini the moment the socket opens, before the provider's start frame arrives. It asks for the greeting straight away and holds up to about 20 seconds of that audio until the stream id is known. The campaign's response-speed preset then decides who owns turn-taking. Fast uses manual activity detection driven by the gateway's noise gate; balanced and conservative let Gemini's own voice detection decide.

When a signal sends email on its own

Three collectors poll each active company watch every six hours: news RSS, SEC EDGAR and job boards. Operators can also add signals by hand or post SES bounces. Every signal then goes through IngestionService.ingest(), and only a narrow case sends email without a person approving it.

StepKindWhat happensStops here when
1 · DedupcodeHashes the raw signal.The same hash arrived in the last 14 days.
2 · ClassifymodelGemini picks the signal type and extracts entities. A keyword heuristic answers when no key is set or the call fails.Never.
3 · SavecodeStores the signal. A second source on the same company within 7 days raises its confidence one tier.Never.
4 · MatchcodeFinds contacts by email domain, rated high, or by normalised company name, rated medium.The type is news-other, or no contact matches.
5 · PlaybookscodePicks active playbooks for this signal type whose directories include the contact.No playbook applies. The match is still kept for the feed.

Fires on its own

  • The playbook's mode is auto
  • The match came from the contact's email domain, so its confidence is high
  • The contact and company pairing has not been suppressed
  • TriggerService's guardrails pass

Waits in the review queue

  • The playbook's mode is review
  • The match came from the company name only, even under an auto playbook
  • Approving runs the same TriggerService; rejecting records the note
  • Two rejections of the same contact and company suppress the pairing, so it stops coming back
TriggerService has the last word. It skips a contact who already got this signal, a contact inside the playbook's cooldown window, and a playbook that reached its daily cap. When it fires, it copies the playbook's template with the {{signal.*}} values filled in, creates a one-contact email campaign, launches it through the normal email pipeline and records a TriggeredOutreach row for attribution.

Where the data lives

MongoService wraps 16 Mongoose models in Prisma-style delegates such as findMany, findUnique, create and update, and maps Mongo's _id to id. Services read like a relational client while MongoDB stays the store. All accounts share one workspace, which is why sign-up is closed in production unless ALLOW_REGISTRATION is set.

CollectionWritten byRead by
UserauthAuthGuard, on every request
SystemSettingssettings, as one row with encrypted credentialsEvery provider call
Contact, ContactDirectorycontactsCampaigns, signal matching, playbooks
Templatetemplates; TriggerService saves signal copiesemail-campaigns, analytics
EmailCampaign, EmailCampaignContactemail-campaigns, scheduler, TriggerServiceHistory, analytics
AiCallingBot, AiCallingBotEmbeddingai-calling-botscalling-campaigns, the gateway
CallingCampaign, CallHistorycalling-campaigns, webhooks, the gatewayHistory, analytics, recordings
CompanyWatchsignalsThe signal poller
Signal, SignalMatchIngestionService, the review queueThe signals feed, the review queue
PlaybooksignalsIngestionService, TriggerService
TriggeredOutreachTriggerServiceSignal stats, the guardrails

Background work

Two cron jobs and two kinds of in-process work run outside any request. None of them is a separate worker; they share the backend process.

JobRunsWhat it doesWhen it fails
Campaign schedulercampaign-scheduler.service.tsEvery minuteLaunches email and calling campaigns whose scheduledAt has passed.Logs the error and returns the campaign to DRAFT, so it is not retried every minute.
Signal pollersignals/scheduler.service.tsEvery 6 hours, or Run nowRuns each enabled collector for each active watch.Logs and skips that collector. A poll that starts while one is running is dropped.
Email sendingrunBackgroundSending()After a launchSends to each PENDING recipient, 250 ms apart.A throttled send is retried once. A recipient that still fails is marked FAILED, and launching again retries only those.
Live callRealtimeCallingGatewayPer answered callHolds the gateway session and its Gemini connection.Ends at 10 minutes, after two silent 15 s checks, or past GEMINI_MAX_CALL_TOKENS, 600k by default.

Why it is built this way

  • Provider keys live in the database.

    Settings encrypts them with AES-256-GCM before saving and always returns them masked. Operators change keys without a redeploy; GEMINI_API_KEY in the environment is only a fallback.

  • Webhooks are public but signed.

    Twilio and Plivo cannot send a user token, so each callback URL carries an HMAC of its call id, keyed with JWT_SECRET. A forged URL gets a 403.

  • Gemini is checked before the phone rings.

    The preflight session catches a bad key or model on the call row, so a contact never answers into silence.

  • One gateway, two phone providers.

    Both providers stream μ-law 8 kHz in the same event envelope. Only the outbound audio, clear and mark frames branch on the provider.

  • Bot knowledge is embedded locally.

    Chunks become 384-dimension hashed word vectors, so training and fetch_context make no API call and stay fast enough for a live call. The trade-off is keyword recall rather than semantic search.

  • Signals only send on their own for the strongest match.

    An email-domain match under an auto playbook fires. Anything weaker waits for a person, and repeated rejections suppress the pairing.

  • Launching twice never double-sends.

    A second launch retries only FAILED recipients. Re-sending to everyone, including those already SENT, is a separate relaunch the operator has to choose.

Run it yourself

Each app runs on its own. The backend needs MongoDB and a JWT secret; provider keys are entered later in Settings.

# Backend: REST on :3001/api, Swagger on :3001/docs
cd AgentReach-backend
npm install
cp .env.example .env      # set DATABASE_URL and JWT_SECRET
npm run start:dev

# Frontend: app on :3000, these docs on :3000/documentation
cd AgentReach-frontend
npm install
# in .env.local: NEXT_PUBLIC_API_URL=http://localhost:3001/api
npm run dev
Live calls need the provider to reach you. Set PUBLIC_API_URL to a public HTTPS address, such as a tunnel to port 3001, and PUBLIC_WS_URL if the socket host differs. With Twilio or Plivo keys that contain “mock” or “test”, launches simulate the calls, so the rest of the flow can be exercised without a phone line.

File map

FileRole
Backend · AgentReach-backend/src
main.tsBoots Nest, sets the /api prefix, CORS and Swagger, and routes WebSocket upgrades. Exports a handler for Vercel and listens everywhere else.
app.module.tsRegisters Mongo, every feature module and both schedulers.
auth/auth.guard.tsThe global guard: Bearer token, user lookup, @Public() bypass.
auth/secrets.tsJWT secret rules and the HMAC call tokens on provider callback URLs.
settings/credential-encryption.tsAES-256-GCM for provider keys, and the masking used in every response.
mongo.service.tsPrisma-style delegates over the 16 Mongoose models in schemas/.
email-campaigns/email-campaigns.service.tsLaunch checks, background SES sending, throttling retry.
calling-campaigns/calling-campaigns.service.tsLaunch, Gemini preflight, dialing, webhooks, recordings.
realtime-calling/realtime-calling.gateway.tsThe live call: sockets, Gemini session, noise gate, timers, transcript.
realtime-calling/audio-codec.tsμ-law and PCM16 conversion and resampling.
realtime-calling/call-tools.tsend_call and fetch_context.
realtime-calling/call-monitor.hub.tsIn-process pub/sub for watching a call live. It is registered, but nothing publishes to it yet.
bot/bot.service.tsKnowledge chunking, local embeddings, cosine search.
signals/ingestion.service.tsThe signal pipeline, from dedup to playbooks.
signals/trigger.service.tsGuardrails and the one-contact email campaign.
signals/scheduler.service.tsThe six-hourly collector poll.
scheduler/campaign-scheduler.service.tsThe every-minute campaign launcher.
Frontend · AgentReach-frontend
src/lib/api.tsThe REST client: access token, refresh and retry, friendly errors.
src/lib/localAuth.tsTokens, profile and theme in localStorage.
next.config.tsRewrites /api/* to the backend.
src/components/docs/This page and its two diagrams.
src/lib/docs.tsContent for every other documentation page.