Thinking About Building An In-house Data Integration Solution?

Sixteen years ago, the road to commercialization of space was sketched on NASA’s old school drafting tables. Partly inspired by the realization that the Space Shuttle program was too costly and that NASA could not be both the supply and demand for space flight, NASA decided to partner with SpaceX. On Saturday, May 30, their collective continuity of purpose was realized and took commercial space travel one step closer to a regular occurrence: NASA astronauts launched in historic test flight of SpaceX Crew Dragon.

The significance of the launch is paramount for many reasons, one of which is a partnership forged on failure. The NASA of the 21st Century turned their back on building everything in-house and toward industry with the knowhow and desire to experience the small errors and cataclysmic failure-and embrace these experiences through testing. In other words, SpaceX’s strength in testing and failing routinely is built into their business plan, which allows them to derive valuable information and swiftly improve their hardware.

Learning from the Space Shuttle program, NASA is not in the market to purchase, own, and operate hardware. It took NASA 35+ years to figure out that when you are your own partner, you are the supply, demand, and limitations to what you can do.

Then NASA made a choice that can serve as a roadmap for any business.

In 2004, NASA decided to turn to industry to innovate and cut costs. Instead of telling SpaceX what to build, they gave them high-level requirements and told them to go. There were considerable bumps along the way, but NASA chose SpaceX for their unique ability to test, fly, fail, fix. Test. Fly. Fail. Fix…. to the point of docking with the ISS.

*Image courtesy of

In the age of new data control, compliance, tag management, first-party tracking, tag reduction, and major data privacy concerns and breaches, legislation is evolving in real-time. Fortune 50 companies, SMBs, FinTech, AdTech, LATAM companies-and others-are searching for partners to address and solve their data concerns just as NASA needed an innovative partner to move into the next age of commercialization of space.

Most businesses can’t afford to test, fly, fail, fix.

Building flexible data architecture for secure and compliant data integration is not easy. And not every company has 16 years to formulate their plan like NASA did. They don’t have time or resources to design and build architecture that’s flexible and stable enough to handle their data integration-while upholding compliance mandates, enabling tag management, switching to first-party tracking, and addressing other issues as they arrive. Improving website speed and adhering to regulations like CCPA, for example, don’t allow time to test, then fly, then fail, then fix, then test, then fly, then fail, then fix-not without costing something.

Even for companies that have dedicated resources to research and develop their data architecture, what happens when legislation and data regulations change? Do these companies have the resources to start all over again as new rules adapt and evolve with technology?

How ready for in-house data engineering are you?

We’ve found that our clients tend to fall into one of three categories:

  • Companies that build their own data routing architecture and spend two+ years developing and proofing the concept. As compliance and security standards evolve, however, they decide to outsource because they cannot afford the time and money to rebuild.
  • Companies that try to build, but quickly realize the time and money required to build, test, fly, and fix are far greater than a pre-built, proven and customizable solution.
  • Companies that choose to focus on what they do and immediately delegate the data architecture and control piece to industry innovators.

Of course, those buckets are relegated to clients.

There may be other companies that successfully build and maintain exactly what they need. Costs associated with building a successful data architecture flexible enough to change with ever-changing legislation vary. But if you extrapolate the R&D, the time required across multiple teams, and multiply it by the years needed to not just build but also prove the concept, the amounts become staggering.

In addition to the research and development costs, companies who aren’t compliant in time are subject to fines. According to a report from DLA Piper, there were 59,000 reported data breaches in the EU during the first nine months of GDPR enforcement. While this number sounds shocking, it shows that many more companies are actually coming forward and being transparent. Under the law, companies that do not disclose breaches to the public are subject to additional fines if caught. DLA Piper says that hiding data breaches “has become a high-risk strategy under GDPR.”

During this same period, 91 GDPR-related fines were issued to organizations operating within the EU. While most of the fines were relatively small, Google was given a €50 million fine by the French privacy authority. The company was accused of processing personal data for advertising services without prior authorization.

How to get it right the first time

NASA shares contracts across innovators Blue Origin, Dynetics, and SpaceX to ensure timely, safe, and repeatable space flight, effectively commercializing low-earth orbit.

Initiatives like better data control, data security, data compliance, site performance, or low cost per tracked user are difficult and complex. When you combine the challenge with the opportunity for businesses to remove IT risk/responsibilities from their scope and delegate the top-level requirements to proven and established solutions, the cost of managing everything in-house begins to heavily outweigh the benefits.

Let solutions like ours test, fly fail, and innovate. Then we’ll meet you on the launchpad, ready for liftoff.

If you have data, do business on planet Earth, and would like more information on proven partners who specialize in data compliance, data transfer restrictions, privacy, data breach notifications, and consent requirements, reach out to me directly at with “liftoff” in the subject line.Photo by James Beheshti on Unsplash.

Originally published at




Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium


Implement Cache Memory in Go

Working with Merge

8 Best Mobile App Development Programming Languages

Increase your Application Performance

Microservices diagram

Do You Even Know What A Bit Means?

Sana Paul, Nishtha Singh, Diksha Garg, Architects, Architecting, Architecture, Architectural, Design, Digital, Machine Learning, Artificial Intelligence, AI, Tech, Technology, Algorithmic Thinking, Algorithms, Cybernetics, Computing, Embodiment, Digital Landscape, IOT, Generative Design, Augmented Reality, Virtual Reality, AR, VR, Building Information Modelling, Analysis, BIM, Zeyka, Zeyka India, Architecture, Interior Design, Home Renovation, Construction, Tech, Design, Project Management

How to build a Power BI report

Kentico 12 Class Libraries with Modern .NET Core Features

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
MetaRouter Team

MetaRouter Team

More from Medium

The Modern-Day Doctor Lives in the Cloud — Data Dynamics, Inc.

The Chess World Championship challenger has made a mistake

What is the role of GLAMs in the Spread of Misinformation?

Database Management: tech against child trafficking