Thinking About Building An In-house Data Integration Solution?
Think about NASA.
Sixteen years ago, the road to commercialization of space was sketched on NASA’s old school drafting tables. Partly inspired by the realization that the Space Shuttle program was too costly and that NASA could not be both the supply and demand for space flight, NASA decided to partner with SpaceX. On Saturday, May 30, their collective continuity of purpose was realized and took commercial space travel one step closer to a regular occurrence: NASA astronauts launched in historic test flight of SpaceX Crew Dragon.
The significance of the launch is paramount for many reasons, one of which is a partnership forged on failure. The NASA of the 21st Century turned their back on building everything in-house and toward industry with the knowhow and desire to experience the small errors and cataclysmic failure-and embrace these experiences through testing. In other words, SpaceX’s strength in testing and failing routinely is built into their business plan, which allows them to derive valuable information and swiftly improve their hardware.
Learning from the Space Shuttle program, NASA is not in the market to purchase, own, and operate hardware. It took NASA 35+ years to figure out that when you are your own partner, you are the supply, demand, and limitations to what you can do.
Then NASA made a choice that can serve as a roadmap for any business.
In 2004, NASA decided to turn to industry to innovate and cut costs. Instead of telling SpaceX what to build, they gave them high-level requirements and told them to go. There were considerable bumps along the way, but NASA chose SpaceX for their unique ability to test, fly, fail, fix. Test. Fly. Fail. Fix…. to the point of docking with the ISS.
*Image courtesy of spaceflightinsider.com
In the age of new data control, compliance, tag management, first-party tracking, tag reduction, and major data privacy concerns and breaches, legislation is evolving in real-time. Fortune 50 companies, SMBs, FinTech, AdTech, LATAM companies-and others-are searching for partners to address and solve their data concerns just as NASA needed an innovative partner to move into the next age of commercialization of space.
Most businesses can’t afford to test, fly, fail, fix.
Building flexible data architecture for secure and compliant data integration is not easy. And not every company has 16 years to formulate their plan like NASA did. They don’t have time or resources to design and build architecture that’s flexible and stable enough to handle their data integration-while upholding compliance mandates, enabling tag management, switching to first-party tracking, and addressing other issues as they arrive. Improving website speed and adhering to regulations like CCPA, for example, don’t allow time to test, then fly, then fail, then fix, then test, then fly, then fail, then fix-not without costing something.
Even for companies that have dedicated resources to research and develop their data architecture, what happens when legislation and data regulations change? Do these companies have the resources to start all over again as new rules adapt and evolve with technology?
How ready for in-house data engineering are you?
We’ve found that our clients tend to fall into one of three categories:
- Companies that build their own data routing architecture and spend two+ years developing and proofing the concept. As compliance and security standards evolve, however, they decide to outsource because they cannot afford the time and money to rebuild.
- Companies that try to build, but quickly realize the time and money required to build, test, fly, and fix are far greater than a pre-built, proven and customizable solution.
- Companies that choose to focus on what they do and immediately delegate the data architecture and control piece to industry innovators.
Of course, those buckets are relegated to clients.
There may be other companies that successfully build and maintain exactly what they need. Costs associated with building a successful data architecture flexible enough to change with ever-changing legislation vary. But if you extrapolate the R&D, the time required across multiple teams, and multiply it by the years needed to not just build but also prove the concept, the amounts become staggering.
In addition to the research and development costs, companies who aren’t compliant in time are subject to fines. According to a report from DLA Piper, there were 59,000 reported data breaches in the EU during the first nine months of GDPR enforcement. While this number sounds shocking, it shows that many more companies are actually coming forward and being transparent. Under the law, companies that do not disclose breaches to the public are subject to additional fines if caught. DLA Piper says that hiding data breaches “has become a high-risk strategy under GDPR.”
During this same period, 91 GDPR-related fines were issued to organizations operating within the EU. While most of the fines were relatively small, Google was given a €50 million fine by the French privacy authority. The company was accused of processing personal data for advertising services without prior authorization.
How to get it right the first time
NASA shares contracts across innovators Blue Origin, Dynetics, and SpaceX to ensure timely, safe, and repeatable space flight, effectively commercializing low-earth orbit.
Initiatives like better data control, data security, data compliance, site performance, or low cost per tracked user are difficult and complex. When you combine the challenge with the opportunity for businesses to remove IT risk/responsibilities from their scope and delegate the top-level requirements to proven and established solutions, the cost of managing everything in-house begins to heavily outweigh the benefits.
Let solutions like ours test, fly fail, and innovate. Then we’ll meet you on the launchpad, ready for liftoff.
If you have data, do business on planet Earth, and would like more information on proven partners who specialize in data compliance, data transfer restrictions, privacy, data breach notifications, and consent requirements, reach out to me directly at Preston@MetaRouter.io with “liftoff” in the subject line.Photo by James Beheshti on Unsplash.
Originally published at https://blog.metarouter.io.