Newsletter Iterable Integration

This guide explains how newsletter subscriptions work in our application, how they integrate with Iterable’s email marketing platform, and the architectural decisions behind the current implementation.

What is This Integration?

The newsletter integration manages user subscriptions to Lonely Planet’s email newsletters through Iterable, a marketing automation platform. When users sign up for newsletters through forms on our website, their information is validated, stored in Iterable, and they’re automatically subscribed to our standard newsletter offerings.

Key Components

The integration consists of three main layers:

1. Client-side form component (newsletterForm) - Captures user input with validation and analytics
2. Server-side API endpoint (/api/external/newsletter) - Validates emails and manages Iterable API calls
3. Iterable platform - Stores user profiles, manages subscriptions, and delivers emails

What Problems Does This Solve?

Email List Management: Centralized management of newsletter subscribers across all Lonely Planet properties without requiring user accounts.

Email Validation: Server-side validation prevents invalid or fraudulent email addresses from entering our marketing database, maintaining list quality and delivery reputation.

Analytics Separation: Decoupling analytics tracking (Tagular) from subscription logic ensures that tracking failures don’t prevent subscriptions, and subscription failures don’t corrupt analytics data.

Authentication Independence: Users can subscribe to newsletters without creating accounts, lowering the barrier to engagement while still allowing authenticated users to manage preferences later.

How It Works

The Subscription Flow

When a user submits a newsletter signup form, several processes happen in sequence:

1. Client-Side Form Interaction

The NewsletterForm component tracks user interaction through analytics events:

// Form viewed (automatically tracked on render)
tagular("FormViewed", {
  formContext: {
    formId: "homepage-newsletter",
    formName: "NEWSLETTER-FORM",
    formType: "NEWSLETTER",
    formBrand: "LP",
  },
});

// Form started (tracked on first input focus)
tagular("FormStarted", {
  /* same context */
});

// Form submitted (tracked 3s after successful submission)
tagular("FormSubmitted", {
  field: [
    {
      fieldName: "userId",
      fieldValue: hashedEmail, // SHA256 hash for privacy
    },
  ],
  formContext: {
    /* same context */
  },
});

Why the 3-second delay? The FormSubmitted event waits 3 seconds to ensure the API call completes before firing analytics. This prevents recording submissions that ultimately fail.

Why hash the email? Email addresses are PII (Personally Identifiable Information). Hashing with SHA256 allows us to track unique users across sessions without storing raw email addresses in our analytics system.

2. API Request to Server

The form submits to /api/external/newsletter with this data:

{
  email: "user@example.com",
  source: "homepage-newsletter",      // Unique form identifier
  formName: "NEWSLETTER-FORM",        // Form type identifier
  formType: "NEWSLETTER"              // Category (hidden field)
}

Why separate source and formName? source identifies the specific form instance (useful when multiple newsletter forms exist on different pages), while formName groups related forms by type.

3. Server-Side Email Validation

Before touching Iterable, the server validates the email through SendGrid’s validation API:

const validation = await validateEmail({
  email: "user@example.com",
  token: SENDGRID_VALIDATE_API_TOKEN,
});

// SendGrid checks:
// - Email format (RFC 5321 compliance)
// - DNS MX records (domain accepts email)
// - Disposable email detection
// - Known invalid patterns

Why SendGrid instead of simple regex? Regex can validate format but can’t detect:

• Typos in domain names (gmial.com instead of gmail.com)
• Domains without mail servers
• Temporary/disposable email services
• Known spam trap addresses

SendGrid’s service checks these conditions, significantly improving list quality.

4. Iterable User Profile Update

If validation passes, the server creates or updates the user in Iterable:

await iterableApi("/users/update", {
  email: "user@example.com",
  dataFields: {
    source: "homepage-newsletter",
    formName: "NEWSLETTER-FORM",
    formType: "NEWSLETTER",
    signupDate: "2024-02-12T10:30:00.000Z",
    signupSource: "Lonely Planet Web",
  },
});

Why update instead of create? The /users/update endpoint is idempotent—it creates the user if they don’t exist, or updates existing fields if they do. This handles both new subscribers and returning users updating their preferences.

What are dataFields? Custom user attributes stored in Iterable. These fields enable:

• Segmentation (target users by signup source)
• Personalization (reference in email content)
• Analytics (track signup channels)
• Automation (trigger workflows based on source)

5. Signup Event Tracking

After updating the user profile, we track a “Sign Up” event in Iterable:

await iterableApi("/events/track", {
  email: "user@example.com",
  eventName: "Sign Up",
  dataFields: {
    source: "homepage-newsletter",
    formName: "NEWSLETTER-FORM",
    formType: "NEWSLETTER",
    signupDate: "2024-02-12T10:30:00.000Z",
    signupSource: "Lonely Planet Web",
  },
});

Why track an event after updating the user? Events and user profiles serve different purposes:

• User profiles store current state (latest source, current preferences)
• Events record historical actions (every signup, even from existing users)

This separation enables:

• Lifecycle analysis (track re-engagement campaigns)
• Multi-touch attribution (understand multiple signup touchpoints)
• Workflow triggers (different automation for first vs repeat signups)

6. Newsletter Subscription

Finally, we subscribe the user to all default newsletters:

await iterableApi("/users/updateSubscriptions", {
  email: "user@example.com",
  subscribedMessageTypeIds: [64347, 71315, 71309], // Production IDs
  validateChannelAlignment: true,
});

What are Message Type IDs? Iterable’s internal identifiers for specific email campaigns or newsletters:

• 64347 - Brand/Shop updates (product news, promotions)
• 71315 - Friday ATP newsletter (travel inspiration)
• 71309 - Picture Yourself newsletter (destination highlights)

What is validateChannelAlignment? This Iterable feature ensures subscriptions match the user’s preferred communication channels. If a user previously unsubscribed from email entirely, this prevents re-subscribing them against their wishes.

Why different IDs for prod/nonprod? Iterable workspaces are isolated environments. Production uses the “Lonely Planet” workspace with live subscriber lists. Non-production uses “Lonely Planet-Sandbox” with test data. Each workspace has its own Message Type IDs for the same logical newsletters.

Environment-Based Configuration

Message Type IDs are selected based on the FRONTEND_ENV variable:

const MSG_TYPE_IDS = {
  prod: {
    MSG_LP_BRAND_SHOP: 64347,
    MSG_NEWS_FRIDAY_ATP: 71315,
    MSG_NEWS_PICTURE_YOURSELF: 71309,
  },
  nonprod: {
    MSG_LP_BRAND_SHOP: 50633,
    MSG_NEWS_FRIDAY_ATP: 77535,
    MSG_NEWS_PICTURE_YOURSELF: 65637,
  },
}[FRONTEND_ENV ?? "nonprod"];

Why environment-based IDs? This prevents test signups from corrupting production subscriber lists and ensures development/staging environments interact only with sandbox data.

Recent Architectural Changes

Decoupling from Authentication

Previous behavior: Newsletter subscriptions were tightly coupled to user authentication. When users created accounts, the auth callback handler would automatically subscribe them to newsletters.

Current behavior: Newsletter subscriptions are handled independently:

1. Anonymous signups - Users can subscribe without accounts via the /api/external/newsletter endpoint
2. Authenticated subscriptions - Auth0 post-user-registration and post-user-login actions handle newsletter preferences for authenticated users
3. Preference management - The /api/external/newsletter/[iterable_id] endpoint allows authenticated users to update subscriptions later

Why this change?

Separation of concerns: Authentication and marketing subscriptions serve different business functions. Decoupling allows:

• Independent failure handling (auth failures don’t block subscriptions)
• Different scaling requirements (authentication is higher-traffic)
• Separate rate limiting and monitoring
• Different teams to manage each system

Lower barriers to engagement: Many users want newsletter content without creating accounts. The old approach forced account creation for newsletter access, reducing signup rates.

Cleaner authentication flow: Authentication code should handle identity verification and session management, not marketing logic. This improves maintainability and reduces complexity in critical auth paths.

Tagular Event Decoupling

Previous behavior: The formSubmit Tagular event was fired as part of the newsletter subscription logic, creating a dependency between analytics and functionality.

Current behavior: Tagular events (FormViewed, FormStarted, FormSubmitted) are tracked independently by the client-side component, with no server-side dependencies.

Why this change?

Reliability: Analytics failures shouldn’t prevent core functionality. If Tagular is unavailable or blocked by ad blockers, users can still subscribe successfully.

Performance: Client-side analytics can fire asynchronously without blocking server responses. The API endpoint responds immediately after Iterable confirms subscription.

Data integrity: Separating concerns ensures analytics data reflects actual user behavior (form interactions), while Iterable data reflects subscription outcomes. A subscription might fail validation, but we still want to track that someone attempted to sign up.

Auth0 Action Integration

Newsletter handling for authenticated users has moved to Auth0 post-registration and post-login actions:

// Auth0 Post-User-Registration Action (pseudocode)
exports.onExecutePostUserRegistration = async (event, api) => {
  const { user } = event;

  // Subscribe new user to newsletters via Iterable API
  await subscribeUserToNewsletters({
    email: user.email,
    userId: user.user_id,
    source: "auth0-registration",
    formName: "AUTH-REGISTRATION",
    formType: "AUTHENTICATION",
  });
};

Why handle this in Auth0?

Guaranteed execution: Auth0 actions run synchronously during authentication flows, ensuring newsletter subscriptions happen before users land in the application.

Centralized user management: All user provisioning logic lives in Auth0, including newsletter preferences. The Astro application receives fully-provisioned users.

Reduced application complexity: Astro code doesn’t need to check “did this user already subscribe?” or handle race conditions between authentication and newsletter flows.

Subscription Management

Anonymous User Flow

Authenticated User Flow

Preference Updates

Users with accounts can manage subscriptions through the authenticated endpoint:

// GET /api/external/newsletter/[iterable_id]
// Fetches current subscription preferences from MAPI

const response = await fetch(`/api/external/newsletter/${iterableId}`, {
  method: "GET",
  headers: {
    Authorization: `Bearer ${accessToken}`, // Passed through to MAPI
  },
});

// POST /api/external/newsletter/[iterable_id]
// Updates subscription preferences via MAPI

const response = await fetch(`/api/external/newsletter/${iterableId}`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${accessToken}`,
  },
  body: JSON.stringify({
    subscribedMessageTypeIds: [64347], // Only Brand/Shop, unsubscribe others
  }),
});

Why proxy through our API instead of calling MAPI directly?

Security: API keys and authentication logic stay server-side. Clients never see MAPI credentials.

Abstraction: If MAPI’s interface changes, we update one endpoint instead of every client calling it.

Monitoring: All subscription management traffic flows through our API, enabling centralized logging and error tracking via Airbrake.

Error Handling

Email Validation Failures

// Invalid email format or failed SendGrid checks
{
  "error": "Invalid Email"
}

User sees: “There was an error processing your request, please try again later.”

Why generic message? Specific validation details could help malicious actors identify validation logic. Generic errors protect against targeted attacks while still prompting users to check their input.

Iterable API Failures

// Network error, rate limit, or Iterable service issue
{
  "error": "Newsletter subscription failed"
}

Behind the scenes:

• Error logged to console with full details
• Airbrake notification sent with context
• User sees generic error to avoid exposing system details

Missing Required Fields

// Missing email or source parameter
{
  "error": "Invalid source or email"
}

When this happens: Usually indicates a client-side bug (form not passing hidden fields correctly). Should not occur in normal user flows.

Analytics and Tracking

Client-Side Events (Tagular)

All form interactions are tracked for funnel analysis:

// 1. Form impression
"FormViewed" - Tracks when form appears in viewport
  Payload: formContext (formId, formName, formType, formBrand)

// 2. User engagement
"FormStarted" - First focus on any form field
  Payload: Same formContext

// 3. Submission outcome
"FormSubmitted" - Successful API response (3s delay)
  Payload: formContext + hashed email for user tracking

Why track all three events?

• FormViewed - Understand form visibility and placement effectiveness
• FormStarted - Measure engagement rate (viewed vs started)
• FormSubmitted - Track completion rate (started vs completed)

This funnel reveals where users drop off:

• Low start rate = form not compelling or hard to find
• Low completion rate = friction in form fields or errors

Server-Side Events (Iterable)

Iterable’s “Sign Up” event tracks successful subscriptions:

{
  eventName: "Sign Up",
  dataFields: {
    source: "homepage-newsletter",
    formName: "NEWSLETTER-FORM",
    formType: "NEWSLETTER",
    signupDate: "2024-02-12T10:30:00.000Z",
    signupSource: "Lonely Planet Web"
  }
}

Use cases for this event:

• Welcome campaigns - Trigger welcome email series on first signup
• Re-engagement - Identify users who signed up but never opened emails
• Attribution - Track which forms/pages drive most subscriptions
• Segmentation - Target messaging based on signup source

Best Practices

When Adding Newsletter Forms

1. Use unique source identifiers: source should identify the specific form instance, not just the form type:

// Good
<NewsletterForm source="homepage-hero-newsletter" />
<NewsletterForm source="article-footer-newsletter" />

// Bad
<NewsletterForm source="newsletter-form" /> // Too generic
<NewsletterForm source="form-1" /> // Not descriptive

2. Include formName consistently: Use standard form names for grouping:

// Standard form names
formName = "NEWSLETTER-FORM"; // Marketing newsletters
formName = "PRODUCT-NEWSLETTER"; // Product updates
formName = "EVENT-NEWSLETTER"; // Event notifications

3. Preserve hidden fields: The API requires source, formName, and formType fields:

<input type="hidden" name="source" value={source} />
<input type="hidden" name="formName" value={formName} />
<input type="hidden" name="formType" value="NEWSLETTER" />

Why are these required? Missing hidden fields cause 400 errors and prevent analytics from attributing signups to specific forms.

Error Handling in Components

Always handle both validation errors and network errors:

const handleSubmit = async (e: Event) => {
  e.preventDefault();
  setIsLoading(true);

  try {
    const response = await fetch("/api/external/newsletter", {
      method: "POST",
      body: new FormData(e.target),
    });

    const data = await response.json();

    if (data.error) {
      setError({
        message:
          "There was an error processing your request, please try again later.",
      });
      setIsLoading(false);
      return;
    }

    // Show success state
    setApiData(Object.fromEntries(formData.entries()));
  } catch (err) {
    // Network error or JSON parse failure
    const { airbrakeClient } = await import("#utils/airbrake/browser");
    airbrakeClient.notify(err);

    setError({
      message:
        "There was an error processing your request, please try again later.",
    });
  } finally {
    setIsLoading(false);
  }
};

Testing Newsletter Forms

When testing in non-production environments:

1. Use test email addresses: Avoid real user emails in sandbox
2. Check environment: Verify FRONTEND_ENV matches your intent (nonprod message type IDs in staging)
3. Monitor Airbrake: Check for validation or API errors that might not surface to users
4. Verify in Iterable: Confirm users appear in the correct workspace with expected dataFields

Security Considerations

1. Never expose API keys client-side: All Iterable API calls must flow through server-side endpoints
2. Validate all inputs: Email validation prevents injection attacks and maintains list quality
3. Rate limiting: Consider implementing rate limits on the newsletter endpoint to prevent abuse
4. CORS restrictions: Ensure newsletter API endpoints only accept requests from your domains

Troubleshooting

”Invalid source or email” error

Cause: Missing required fields in form submission

Check:

• Are hidden fields present in form HTML?
• Is form data properly serialized before submission?
• Are field names exactly source, formName, and email?

Subscriptions not appearing in Iterable

Cause: Wrong environment or API credentials

Check:

• Does FRONTEND_ENV match your intended environment?
• Are you checking the correct Iterable workspace (prod vs sandbox)?
• Is ITERABLE_API_KEY valid and has correct permissions?

Email validation always fails

Cause: SendGrid API issues or invalid token

Check:

• Is SENDGRID_VALIDATE_API_TOKEN set correctly?
• Does the token have email validation permissions?
• Is SendGrid API responding (check server logs)?

Analytics events not firing

Cause: Tagular not loaded or client-side errors

Check:

• Is Tagular script loaded before form renders?
• Are there JavaScript errors in browser console?
• Are ad blockers interfering with analytics?

Success but no confirmation email

Cause: Iterable workflow configuration, not application issue

Check:

• Does the newsletter campaign have a welcome email configured?
• Are Iterable workflows active (not paused)?
• Is the user’s email bouncing or marked as spam?

Related Files and Components

API Routes

• /src/pages/api/external/newsletter/index.ts - Anonymous subscription handler
• /src/pages/api/external/newsletter/[iterable_id].ts - Authenticated preference management
• /src/pages/api/external/newsletter/README.md - API documentation

Components

• /src/components/newsletterForm/newsletterForm.tsx - Main form component
• /src/components/newsletterForm/newsletterForm.stories.tsx - Storybook examples
• /src/components/fields/input.tsx - Reusable input field component

Utilities

• /src/utils/airbrake/browser.ts - Client-side error reporting
• /src/utils/airbrake/node.ts - Server-side error reporting
• /src/cohesion/tagular.ts - Analytics event tracking

External Documentation

• Iterable API Documentation
• SendGrid Email Validation API
• Auth0 Actions