The Rise of Zero Copy

There are numerous reasons why zero-copy is coming up so much at the moment.

The first rather obvious one is that the volumes of data collected by businesses are huge and ever increasing (a recent study found that SMB hold on average 50TB, a number which balloons to 350TB for larger businesses). Even if network speed has increased and storage is as cheap as it’s ever been, moving around those kinds of volumes is not trivial.

Then, the improvements in cloud database technologies (such as Snowflake, Google Big Query, Redshift or Databricks) have enabled IT departments to build Data warehouses within their private cloud. Those databases might not provide full ETL capabilities, but the loading is usually not an issue. One might argue that data is theoretically available even if not “accessible” by the teams needing it the most (i.e. Marketing doesn’t have the required SQL skills).

At the same time, the legal landscape has carried on evolving towards a stricter framework. GDPR is the main one but not the only consideration either. Ultimately, where the data resides is important, and not sending it to a US SaaS vendor is something that most would prefer. It’s not just that the NSA can go snoop into your customer data, it’s also that data is the valuable fuel that those companies need to train their AI models. Data Sovereignty is probably a sub-trend which you’ll start hearing about.

So, from a storage perspective, businesses have a strong desire and incentive to keep ownership of their precious data asset, and technology is allowing them to achieve that goal in a cost-effective and scalable way.

Nonetheless, holding the data in a safe and technically sound place is only half the battle. The second half is to make it available and accessible to the teams and system that need it to operate. The issue being that if those teams or products start copying the data everywhere, the initial solution is at best undermined, and more realistically completely compromised.

The marketing teams and the ESP don’t actually need to full terabytes of data, they need insight, decisions or personalisation points. The Zero-copy approach!

Another issue addressed by Zero-copy is the fact that numerous SaaS vendors have moved to a charging model based on events. When you try to replicate the terabytes of ever-changing data into those platforms, the event-based cost start spiralling up. If instead you feed those tools just what they need to operate, the volume of events can be cut by 10 or even 100X! I’ve encountered too CMO forced to make bad business decision because of a looming event-driven bill.

There are also additional benefits to the approach, one of them is the slightly intangible data consistency.  Modern marketing requires integration between multiple platforms such as CRM, Email, Social media, or Analytics. The Zero Copy approach helps maintain consistency across these systems by limiting the redundant copies that could diverge in quality or become out of sync. Hard to measure at first but can help prevent data quality issues from arising in the future

Another similar benefit lies in the fact marketing increasingly demands real-time personalisation and responses. Here, the Zero Copy approach reduces latency by eliminating unnecessary data movement. If data is accessible in its source system, it is always fresh, up-to date and real-time.

Is it the silver bullet everyone has been waiting for? Maybe not, some data still need to travel,  some of the target systems have constraints that need to be taken into consideration, and tech stack are not perfect. Real-world is complex and the Zero-copy approach needs to be supported by tools (such as imagino) to avoid just being an unfulfilled ethos.