Key Design Decisions

Connect: progressive disclosure for complex setup

Connect had to handle an enormous range of authentication models (OAuth, API keys, IP whitelisting, SFTP credentials, direct database logins) and configuration parameters (host addresses, port numbers, query scheduling, regional language defaults). If we exposed every field up-front, the page would overwhelm anyone setting up their first connection.

The solution was progressive disclosure. We designed a flow where users picked their connection type, named the connection, entered core credentials, and selected the specific tables and columns they wanted to pull. Advanced options (like language defaults, scheduling parameters, and lower-level configuration) lived in a collapsible "Advanced" area that stayed out of the way until someone needed it. We also built in an inline connection-test that validated credentials before users committed to the rest of the setup, cutting down on the most common early-stage error.

Prep: making data transformation approachable without hiding the power

Prep was where business users ran into territory traditionally reserved for data engineers. Users needed to be able to filter, handle nulls, merge, split, pivot, summarize, rename, and convert data types across tables that could contain millions of rows. We needed non-technical users to feel confident here, without making the tool feel dumbed-down to power users.

Two design choices shaped Prep:

A column-first selection model. Users clicked on a column in the data table, and relevant transform tools surfaced in a side panel with clear icons for each operation. This kept the interaction direct: you touch the thing you want to change, then you choose what to do with it. No hunting through menus.

An interactive transform pipeline. Every transform a user applied was recorded as a step in a visual pipeline, showing the full lineage of the table. Users could insert new transforms between existing ones, delete steps mid-pipeline, and understand at a glance what had been done to their data. This was particularly important for teams where one person built the data prep and another person had to audit or modify it.

Among the transforms I designed directly were null handling, ignore, filter, and merge. Each one needed its own interaction model to feel appropriate to what it did.

Relate: a visual canvas instead of a join builder

Relate was the module where users connected their prepared sources together into a unified data model. The typical BI pattern for this at the time was a SQL-style join builder: a form or query-writer that let you specify joins between tables with dropdowns.

We chose a visual canvas instead.

Users would click a source to open a panel on the left showing all its columns, then click a second source to open a panel on the right. Linking two columns created a join between them, visualized on the canvas with a connecting line and a join-type icon (inner, outer, left, right). The joins were represented as Venn-diagram icons that showed, at a glance, which records the join would include.

As users connected more sources, the full network of relationships took shape as a star schema on the canvas — intentionally mirroring how data engineers already thought about dimensional models, but made visual and interactive so business users could understand it without a technical background.

The decision to go visual was deliberate. SQL join builders are efficient for people who already know what they want. But for users who were still figuring out how their data fit together, a visual canvas made the relationships discoverable. You could see what was connected to what, which sources were underutilized, and where gaps existed.

Designing for scale: preventing UI collapse at hundreds of sources

Some customers were uploading hundreds of loose Excel files plus formal data sources, which meant the Relate canvas could legitimately have hundreds of nodes at once. Most visual canvas interfaces fall apart at that scale.

We used a handful of patterns to prevent collapse:

• Adaptive node sizing. With a small number of sources, nodes rendered larger with visible source names. As the count grew, nodes shrank and names moved to hover tooltips, preserving legibility across scales.

• Color-coded source types. Prepared sources and raw sources rendered differently, helping users distinguish at a glance where each node sat in the data pipeline.

• Differentiated line styles. Standard joins, complex joins, read-only joins, and invalid joins each had their own line style on the canvas, so users could spot problems without clicking into individual relationships.