Designing Automated Pipelines for Striim

A case study about learning from $925k in lost revenue and 6,000 support tickets in a year

Designed Striim's automated pipeline solution that reduced setup time from hours to minutes. Led design across three iterations, created the company's first design system, and the first iteration alone generated $500k revenue and processed 100TB of data in a month.

01.The Context

01.The Context

Striim builds data pipelines that move information between systems (databases, cloud services) in real-time to power analytics and AI.

Most customers need two things: Initial Load or IL (copy all existing data) followed by Change Data Capture or CDC(sync ongoing changes). Think automatic photo sync - take a one-time complete snapshot of your existing photos (IL), then every new photo automatically appears in the cloud in real-time (CDC). One setup, zero maintenance.

The gap: Despite supporting both of these use cases individually, Striim was missing a key experience: Automated Pipelines that managed the complete IL→CDC process

Staff Data Engineer

Potential customer

"Overall Striim does have the features that we're looking for but there's quite a sizable manual work to maintain both the initial historical load and the CDC part. We decided to go with another solution."

02.The Crisis

02.The Crisis

The damage: $925k+ in lost revenue. 179 critical issues. 6,000 support tickets from botched manual setups over a year.

The root cause: Every customer needed our support team to manually coordinate Initial Load (copying existing data) and CDC (syncing ongoing changes) - a complex, error-prone process that was killing our growth.

The manual pipeline setup requires users to create and coordinate separate IL and CDC pipelines, exposing complex configurations that overwhelm SMB users and drive support ticket volume.

03.My Evolving Role

03.My Evolving Role

This project spanned three major iterations over several years, with my role evolving as the team structure changed.

V1: Sole designer

V2: Part of a 2-person team responsible for the configuration experience

V3: Returned to solo design responsibilities as we integrated the automation features into the main platform.

04.Key Contributions

04.Key Contributions

Design System Foundation: Pitched and designed Striim's first design system, now implemented throughout the entire product ecosystem. This foundation significantly accelerated subsequent design iterations across all Striim products.

Standalone Product Success: As sole designer, created a new product from scratch that generated $500k in revenue and processed 100TB of data in a month.

SMB-Focused Innovation: Designed Striim's first guided experience specifically for data engineers at small-to-medium businesses, transforming a technically complex enterprise process into an accessible self-service workflow.

A snapshot from the first iteration of the design system

05.Initial Discovery and Research

05.Initial Discovery and Research

Competitive Analysis: Analyzed wizard-like experiences from Qlik, Fivetran, Attunity, Amazon Glue, HVR, and Airbyte to understand industry standards for automated data pipeline creation.

User Research: Conducted interviews with internal data engineers as well as with consultants hired by Striim to understand pain points in the current manual process and validate design directions through usability testing.

06.The Journey:
Three Versions,
Hard Lessons

06.The Journey: Three Versions, Hard Lessons

06.The Journey: Three Versions, Hard Lessons

What started as a database migration tool evolved through three iterations, each teaching us that great design needs the right business strategy to succeed.

V1: StreamShift - The Proof of Concept

Competitive Analysis: Analyzed wizard-like experiences from Qlik, Fivetran, Attunity, Amazon Glue, HVR, and Airbyte to understand industry standards for automated data pipeline creation.

User Research: Conducted interviews with internal data engineers as well as with consultants hired by Striim to understand pain points in the current manual process and validate design directions through usability testing.

Responses to a question from the post-task questionnaire.

V2: BigQuery Focus - Following the Money

BigQuery represented 60% of Striim's revenue but was not supported by V1. We evolved the product to help users move their data dedicatedly to Google BigQuery as part of our partnership with Google.

Designs

Step 1: Configure the Target
Step 2: Configure the Source
Step 3: Select the Data to Move
Step 4: Monitor
Collaborative design effort between myself and the other designer showing live data flow progress and system health metrics during the IL-to-CDC transition process.

Validation

Internal testing with data engineers and external consultants helped validate the simplified experience and eliminate major pain points. The final product was tried by around 300 potential customers, but resulted in no business revenue.

Excerpts from the usability test sessions I conducted to validate the designs.

The Unraveling

Instead of integrating automation into our main platform, leadership launched V1 and V2 as a separate product.

The UX was headed in the right direction. The business strategy was catastrophic.

Market Confusion: ~10 Striim products suddenly existed. Customers couldn't choose. Sales team was overwhelmed.

Pricing Death Spiral: 4x more expensive than signing up for the storage solutions.

Feature Gaps: Missing 100+ connectors, transformations, and enterprise features that existed in our main product.

The Killer Blow: Existing customers started churning because they wanted automation, but it was only available in a separate product (with feature gaps).

V3: Reuniting the Products

The separate product failed within 18 months. We integrated automation back into Striim Enterprise, where it belongs.

An exploration of user flows and information architecture

Designs

Designs

Step 1: Configure the Source (Where Data Currently Lives)

Step 1: Configure the Source (Where Data Currently Lives)

Step 1: Configure the Source (Where Data Currently Lives)

Step 2: Configure the Target (Where Data Will Be Copied)

Step 2: Configure the Target (Where Data Will Be Copied)

Step 2: Configure the Target (Where Data Will Be Copied)

Step 3: Select the Data to Copy

Step 3: Select the Data to Copy

Step 3: Select the Data to Copy

Step 4: Review

Step 4: Review

Step 4: Review

Step 5: Monitor

Step 5: Monitor

Step 5: Monitor

Overview First

Overview First
Overview First

Details on Demand

Details on Demand
Details on Demand
Monitor the Initial Load progress
Monitor the Change Data Capture progress

What I Learned

What I Learned

What I Learned

Design success ≠ Product success. The automation wizard was genuinely better than anything we'd built, but great UX trapped in bad strategy still fails.

Integration beats separation. Creating separate products fragments your market instead of expanding it.

Context is everything. The same design that failed as a standalone product succeeded when integrated into the existing platform.

Consider the ecosystem. Product decisions ripple through your entire customer base, not just new users.

This project taught me that successful product design requires understanding business strategy, market positioning, and customer ecosystems. Sometimes the best design decision is enhancing what works rather than building something new.

————————————————————————————————
The automation features now serve hundreds of users within Striim Enterprise, finally delivering the original vision without strategic complications.

Design success ≠ Product success. The automation wizard was genuinely better than anything we'd built, but great UX trapped in bad strategy still fails.

Integration beats separation. Creating separate products fragments your market instead of expanding it.

Context is everything. The same design that failed as a standalone product succeeded when integrated into the existing platform.

Consider the ecosystem. Product decisions ripple through your entire customer base, not just new users.

This project taught me that successful product design requires understanding business strategy, market positioning, and customer ecosystems. Sometimes the best design decision is enhancing what works rather than building something new.

———————————————
The automation features now serve hundreds of users within Striim Enterprise, finally delivering the original vision without strategic complications.

Design success ≠ Product success. The automation wizard was genuinely better than anything we'd built, but great UX trapped in bad strategy still fails.

Integration beats separation. Creating separate products fragments your market instead of expanding it.

Context is everything. The same design that failed as a standalone product succeeded when integrated into the existing platform.

Consider the ecosystem. Product decisions ripple through your entire customer base, not just new users.

This project taught me that successful product design requires understanding business strategy, market positioning, and customer ecosystems. Sometimes the best design decision is enhancing what works rather than building something new.

—————————————————————————————
The automation features now serve hundreds of users within Striim Enterprise, finally delivering the original vision without strategic complications.