All posts by themetaphysician
I’ve been re-reading Mary Shelley’s Frankenstein—a ground-breaking work of fiction that she completed in 1817 at the age of 20. The book is quite different from the movie, in part because the creature that Victor Frankenstein creates is quite thoughtful and articulate. And, of course, his name is not Frankenstein.
Here’s the thing, though. Frankenstein doesn’t create his creature because doing so is good or right. He creates it because he can. And he spends the rest of his life trying—unsuccessfully—to undo what he did, because it turns out that the consequences of his experiment were rather foul.
The Frankenstein story is thus an apt metaphor for our present-day economy. We create what we create just because we can, not because we’ve thoughtfully considered the consequences of that creation. As a result of this carelessness, we’re likely to expend what’s left of our productive energies undoing much of what we’ve done—because what we’ve unleashed upon the world is multiple orders of magnitude more monstrous than anything Frankenstein ever did.
Why capitalists create
Let’s not kid ourselves. Sure, for PR and marketing purposes we couch our product development in terms of “meeting a need.” But the empirical evidence indicates otherwise. Sure, we may imagine a market “need” for our creations—but the real question in any business plan is profitability, not need.
In fact, if our concern were really need, we’d work in cooperatively to meet that need. But we don’t. Instead, we try to beat others to market. We keep secrets from them. We put out propaganda about how their creation is worse than ours—even when we know it’s actually damn good.
Besides, the big money isn’t in meeting needs. It’s in creating something no one needs and then convincing them they need it. We call this “making a market”—and it’s how real fortunes come about. No one needed Amazon or Facebook. But once they became available, they became desirable. And that desirability eventually feels like a need. It’s just like heroin and crack. No one needs them. But oh boy. Once you use them, it’s hard to stop.
Our “disruption” and “innovation,” in other words, is fully Frankensteinian. We create what we can just because we can. It’s only after we create it that we try to convince anyone that it’s something they need.
What capitalists create
Now that we’ve considered why we create, let’s look at what we create.
We create apps, services around apps, and tools to get apps to market. Again, we may call this “innovation” and “disruption”—but it is anything but. Just like Frankenstein, we cobble together pieces of code and process ganked from here and there. Then we claim that we are “innovative” because no one has ever created any monster exactly like ours.
I’m not saying we never improve a design or streamline a process. But let’s get over ourselves. A mortgage is still a mortgage. An e-tailer is still a retailer. A car is still a car. Nobody is financing genuine innovation. They’re financing incremental improvement and fast follows. Can you even name the last true innovation? I’d be interested in your answer.
Also, almost none of it works. What we create is enormously defective. It’s just that we’ve developed a tolerance for defect that could probably be classified as a cognitive pathology. Can you imagine having to reboot your car while you’re driving on I-95? Or having to pay for new stations on your radio until you’ve got 22 monthly charges hiding in 12 different invoices?
Our creations are such ugly monsters that we cannot secure our data, ensure uptime, or predictably control our operational costs. But, again, we accept this because—unlike the monster Frankenstein created—ours deliver food to your door and provide your niece with an entry-level job creating something called “content.” Plus, because everything is hidden behind a screen, few of us have ever seen how truly ugly our monsters are.
The result of our creation
The final proof of our economy’s Frankensteinian nature is empirically revealed is its results. If we were doing anything to build a better world, we would have seen some improvement by now. After all, we’ve been at this for almost half a century.
But our results are horrendous. Our single most existential issue, for example, is the climate crisis. Have our creations helped us in that regard? Not a whit. In fact, we even created a monster called “Bitcoin” that has the same carbon footprint as Argentina. And that’s just one tiny piece of the global carbon footprint our monsters generate.
How about income disparity—which is not only inhumane, but also a primary contributing factor to social unrest? Are our creations helping us there? Just the opposite. We’re actually exacerbating the problem of capital aggregation by disproportionately rewarding VCs and private equity while extracting more value from the working poor than ever.
The monstrosity of our creations is further revealed in the way we keep externalizing the consequences of our build-it-because-we-can mentality. Distributed computing becomes unwieldy so we offload to the cloud. The cloud then becomes totally unmanageable, so we create a new set of monsters to get control of the previous ones. And so on.
So keep patting yourselves on the back, my Dr. Frankensteins. But know this. History will judge this generation as a total failure because we didn’t actually succeed at anything. We are self-obsessed infants who will leave nothing to the future but the destruction of the planet, pathological cognitive overload, and no coherent spiritual belief beyond the AI fantasies of STEM-addled know-nothings.
“But Lieberman!” you protest. “Look at GDP! Look at the market! We’re doing amazing!!!”
No, we’re not. Just as Frankenstein’s monster was animated by external energy, so is our Frankenstein economy propped up by low interest rates, quantitative easing, insane tax legislation, and tax avoidance on a cosmic scale. If you think your buggy code is what’s propping up the market, then our Dunning-Kruger epidemic is even worse than I imagined.
Our economy is monstrous. Our business culture is monstrous. Our society is monstrous. And it is we who made them that way.
Do I have a constructive suggestion to offer in this scenario? Of course I do. But you may not want to hear it. It’s the same suggestion Shelley offered in her novel—and the same one declared by prophets for generations. Repent. We are not on the right path. Let’s stop building monsters for no other reason than that we can—and start addressing the real issues facing humanity in 2021. Feed the poor. Save the Earth. Do justice. Show mercy.
Every time we try to create a unicorn, we wind up creating yet another gotdam monster. So let’s stop it already. We’re not innovating. We’re just imitating. The real innovation isn’t making a buck. It’s re-making our world for the greatest good.
InformationWeek news desk interview with Rich Thompson of Cylance
NYC’s Gonna Be Just Fine
As a pandemic steamrolls us, media voices have expressed grave concern about the fate of New York City. Some have even called it a “ghost town.”
How foolish. Of course we care about—and must care for—the people of New York, especially those who earn their daily bread in healthcare, restaurants, building services, and other labors. But let’s get, as the expression goes, “hunnit.” The City’s fate is not contingent upon anyone’s joy or misery, life or death, presence or absence.
We don’t make the City what it is. The City makes us what we are.
History is natural
The piece of Earth we think of today as New York City has been exercising its will upon those who inhabit it since before White Europeans showed up to exploit and colonize it. In fact, before any humans at all lived here, the region’s rivers, wetlands, and woods teemed with biotic richness largely unmatched anywhere else.
The Lenape and other First Nations that settled here found themselves in near-Edenic environs: temperate climate, convenient grab-n-go fishing, plenty of construction materials, and easy transport. It’s a shame that anyone came along to trouble them.
But come along they did, immediately commencing to turn Manhattan—not only the region’s central island, but also what author Russell Shorto called “The Island at the Center of the World”—into a hub of global commerce.
Why New York, though? Boston, Baltimore, and Charleston also boast fine natural harbors that gave rise to cities. Why did we so dramatically outswell our coastal peers?
Shorto convincingly argues that the City had (and has) something those other harbors lack—a complex of inland waterways that perfectly complement its vast oceanic harbor. At a time when water transport was the only practical means of moving large volumes of goods and people over any distance, these inland waterways proved vital. Merchants could not only get large ships laden with goods to the City, they could also then use smaller vessels to easily distribute those goods to innumerable other locations around the island, up the Hudson, along Long Island Sound, and elsewhere inland via smaller rivers like the Hackensack and Raritan.
As James Fenimore Cooper wrote in his 1830 novel The Water Witch:
Nature has placed the island of Manhattan at the precise point that is most desirable for the position of a town. Millions might inhabit the spot, and yet a ship should load near every door.
Construction of the Erie Canal 150 miles north in 1825 further advantaged the City by making it the gateway to the entire Midwest, as well as to its more proximate water-fed sprawl.
Once so established, New York’s primacy simply continued by other means after water transport was supplanted. Rail lines radiated from where watercraft once converged. Roadways then did likewise—abetted by bridges and tunnels that were built as soon as we could figure out how to engineer them. Air routes followed thereafter.
Then the City got wired. Other cities may have taken advantage of digital’s location-independence to muscle in on the City’s markets, but the City stubbornly remains the City.
Its waters have been flowing for a long, long time. Its bedrock goes deep. Its paths are well trod. The outbreak of a virus is a small thing to such a piece of Earth.
Always dying, always alive
We’ve always had anxieties about the City. We worry when the quarter-Brooklyn that passes as our nation’s capital tells us to “Drop dead!” We worry when gentrification disorients and dislocates us.
But that anxiety isn’t actually about the City itself. It’s about losing our personal versions of the City. These versions are typically reified at a time in our lives when we’re most intensely urban, when the City most owns us and we’re sweetly but briefly deluded into imagining that we own some little swath of it.
For me, that period was the late 70’s-early 80’s and that swath stretched across SoHo, the Village, and the Lower East Side where in a single evening I could hear Dexter Gordon at Sweet Basil, Oliver Lake at Ali’s Alley, Monty Waters at Tin Palace, and still make the last show at CBGB—all on foot—before taking the heroic wee-hours ride home to 92nd and First, where a fourth-floor one-bedroom walkup cost me $250 a month.
That beloved City-version is four decades gone. Interestingly, though, much of it was at least as barren as the COVID-stricken City of today. I’d walk blocks without encountering a single gainfully employed soul. And there were plenty of empty places to squat or start an impromptu music venue, rent-free.
Even my home neighborhood (did we call it “Yorkville?”) felt empty, despite being fully populated by working folk. But the density was low because at the time there were no high-rises—and there was no Q train on Second.
The City, in other words, was a perfectly wonderful “ghost town.” What better companions could a young man have in those thin-aired hours before dawn than ghosts, anyway?
The City is always dying and always alive. It is beautiful and ugly. It is the signified that defies our signifiers. You can’t argue with Derrida in the City. Well, you can—but you’ll lose.
What, NY worry?
The version of the City contemporary Cassandras apparently fret about is one beloved by real estate speculators, landlords, and tourist-trappers. I’m less concerned about such people than perhaps I should be. But it seems hideously reductionist to assess the City’s circumstance with economic metrics alone. You might as well marry for money—which I’m sure many have done without realizing it.
After all, the City isn’t just rents and occupancies. I mean, look. My Mom grew up in the Bronx during the Great Depression. Was she worse off because she didn’t live in a glass cube and have concierge service? My Grandfather was able to support a wife and three kids on his single inconsistent income as a union painter. Is that awesome or what? And what about the education and culture his daughters experienced in a bygone City-version that—according to the reductionist metrics of capitalist Cassandras—was essentially comatose?
By the way, data-driven doomsayers seem oblivious to the fact that the City is actively growing a sixth borough. That borough may not be considered as such politically or legally, but it’s booming and it’s ontologically as New York as Staten Island or the North Bronx.
I’m referring to the stretch of municipalities on the west side of the Hudson from Edgewater to Jersey City. One is even called “West New York.” Look down your snooty provincial nose if you want, but that stretch is ethnically diverse and closer to Manhattan than most of the other four boroughs, thanks in part to the recently deceased Arthur Imperatore, Sr.—who saw the future and built a ferry to get there.
And please don’t take this the wrong way. 9/11 was a tragic, horrific day for thousands of us. But the City took the insult in stride. Twenty years later, subways still underserve commuters, Wall Street still sinks its vampiric teeth into the jugular of the global economy, and you can still spend on dinner what was once my monthly rent. Somewhere behind the City’s billion windows, human ova are being fertilized, agonies large and small are being suffered in isolation, and someone is hatching a million-dollar idea that someone else is going to totally rip off. The City goes on.
Our City, you see, is not like other cities. It is itself. Seattle is rainy. LA is sunny. But our City doesn’t give a good gotdam whether it’s raining or sunny or snowing or flooding or you’re winning or losing or living or dying or moving to Scarsdale or arriving from Sheboygan.
New York City is based. It may be the most based city that ever was.
The long, long run
It’s probably not a coincidence that my second favorite city is Venice. Venice is also a water city. And that’s putting it lightly. People have been predicting its demise for centuries. In fact, its heyday is so far in the past that it has been in decline since before the Dutch stole Manhattan with their transactional sleight-of-hand.
But Venice is also based as hell. Its ghosts are so badass that they’ll keep living there after the rest of us are long gone. I can only assume that anyone who counts New York City out has never been to Venice—or has failed to retain its central lesson.
So spare me your pearl-clutching. The City’s gonna be just fine. I don’t say that because I’m an optimist. I say that because I’m a realist. Of course, we’re doomed. But we’ve always been doomed. We’ve never been anything but doomed. But between now and such time as that doom manifests, there’s lots to be done. And you can have it all done to you in New York City.
The Datarchy Delusion
Why Quantism Is a Toxic Religion We Must All Abjure
“You can’t improve what you don’t measure.” We’ve all heard this mantra, which is often mistakenly attributed to management gurus like Peter Drucker and W. Edwards Deming. And many of us embrace this principle as the basis upon which to press forward with the advancement of data-driven analytics, machine learning, and AI.
There’s just one problem. It’s horsecrap. We improve stuff without measuring it all the time.
There are lots of ways to logically refute the poisonous myth of absolutist quantism. One way is to simply observe our own lives. I, for one, don’t log the total minutes per month I spend arguing with my fiancée. Nor do I track mean time between aggravated eyerolls. Nonetheless, through effort and communication, she and I have achieved a level of intimacy, trust, and delight that I’m sure would be the envy of many.
Another piece of empirical evidence is that much of human progress has had nothing to do with data. The discovery of vaccination, for example, was based on insight and bold action—not the analysis of quantitative data. Sure, data has been used to incrementally improve the efficacy of vaccines and vaccination programs. But the innovation of the vaccine itself was not based on data inputs.
A third refutation is simply to reference the purported disciples of data, Drucker and Deming. In his masterwork The New Economics, Deming actually calls the above pseudoprinciple “a costly myth.” And Drucker asserted that the only functions a manager can personally perform—relationships with people, development of mutual confidence, creation of community, etc.—couldn’t be measured at all.
Yet we persist in clinging to and promulgating the false gospel of quantification. In fact, as we disseminate our technologies across all aspects of human endeavor, we are aggressively poisoning those human endeavors with a dogma that is doomed to failure: that salvation can be found in governance by data—or, as I choose to call it, “datarchy.”
Why datarchy is failing
I’ll probably write a separate piece on how datarchy wreaks destruction, but let’s briefly consider a few basic problems with the fetishization of quant:
- Datarchy privileges the measurable above the important. Sure, data is super-useful for many purposes. But there’s a reason employers first use data to screen job applicants and then call the ones with the best data for an interview before making their final selection. Data alone is simply insufficient. We know this instinctively. Yet because we’ve gotten so good at capturing and analyzing data, we seem determined to pretend that that’s how we can solve our most pressing problems.
- Datarchy answers small questions and begs big ones. Market indices like the S&P and Dow register the expectations of an extremely small number of human beings with relatively narrow interests regarding a handful of public corporations. These indices have almost nothing to do with anything except flows of aggregated capital. Yet they’re treated as a reliable daily indicator of the state of the cosmos. It’s patently absurd. But because we have been indoctrinated to “believe” in data, we act as though these numbers yield knowledge about our economy that we in reality lack. Worse yet, our false belief makes us intellectually complacent—so we keep failing to ask fundamental questions about capital and its purposes.
- Datarchy doesn’t lead inexorably to the good. We have more data and use it in more ways than ever. Despite this, the climate crisis escalates unabated. Economic disparity keeps getting worse. Political institutions are failing. And so on. I know those of us working in data-related fields are tempted to claim that this is not our fault, that if people would just utilize the amazing tools we develop in more effective ways, everything would be fine. But that’s a cop-out. Being good at engineering isn’t an excuse for being bad at intellectual honesty, civic responsibility, and the ethical use of one’s gifts. Unless, of course, we as an industry simply want to plead guilty to moral bankruptcy while we cash out of our third startup.
And this is without addressing issues such as bad data, skewed data sampling, and poor data taxonomy.
Dealing in realities
None of the above is meant as an indictment of data-related technologies themselves. I’ve been working in the field for more than forty years. Wikipedia even credits me with coining the term “DataOps.” And I continue to advocate for the aggressive use of data for the purposes it is suited to serve.
But we can’t claim to be smart while acting dumb. When you hear someone in power say “We will go where the facts lead us,” you might think we’ve somehow succeeded as an industry. Unfortunately, just the opposite is true. Such an absurd statement only demonstrates that we’ve poisoned people with our own special brand of Kool-Aid. Facts don’t and can’t lead. Leaders lead. And they lead by their vision, values, and moral capacity—things which data does not and cannot provide.
So, yes, let’s keep using data engineering as a tool. But let’s wield that tool wisely—and not impute to it any inappropriate power and authority just because such hype may serve our commercial purposes. We can and must use data to make better decisions. However, it is also our responsibility to rein in datarchy by properly understanding and communicating its limits. Failure to do so will have dire consequences for which we—and we alone—will be accountable.
-Lenny Liebmann, 11/17/20
(Note: The term “datarchy” is entirely unrelated to any of the several companies using the trade name “Datarchy” in the U.S., Slovenia, India, and elsewhere.)
Greed as Addiction
Maybe let’s try re-framing “greed” as “addiction to financial gain.”
When we accuse a cadre of “greed,” we’re saying something about their character generally. This takes us out of the realm of productive problem-solving and into the realm of tribal othering.
When we talk about “addiction,” we may accomplish several things:
First, we de-other the greedy. Addiction is a common human affliction. We are all addicts of one kind or another – to drugs, food, gambling, approval, sex, Twitter, etc. Those addicted to gain are just like us save for the nature of their addiction.
Second, we rightly connect the nature of greed with its negative consequence. Addicts ultimately and always put their addiction first. You may think you love your kids, but if you’re on dope you will leave them in the car alone to cop. You may think you are dedicated to your career, but if you are a sex addict you will compromise your work in a heartbeat for another lustrush.
Addiction to gain is no different. It places profits before people, transactions before the ethics, quick wins before the long-term well-being pf oneself and others.
Third, it suggests solvability. Greed is incurable. Addictions are subject to therapeutic treatment. We find out what’s triggering our compulsion – and apply various measures to put something between our present moment and our next self-destructive impulse.
Our present culture fosters addiction to financial gain, just as it fosters other addictions. But there are countermeasures. “Greed” suggests a much more intractable personal pathology.
Finally, the framing of “addiction” allows us to distinguish the healthy range of material acquisitiveness from the unhealthy one. If you want a nice RV so you can take the family on nice vacations, you may have to work a little harder for that. We can rationally distinguish this from using your aggregated capital to enslave other human beings and burn the planet.
What we call “greed” really does seem to be a form of addiction, where the addict craves the rush of the next gain – and self-medicates against various negative feelings such as shame by pursuing that next rush. And the dominant narratives of “success” and “accomplishment” often feed such addiction.
Let’s do what we can to stop it and offer the gain-addict the opportunity for healing.
Technology Debt Crisis 2025
Capital loves code. Over the past decade, there’s been no better way to get fat returns on your cash than to invest in code. Tech stocks have buoyed markets. And code fever shows no signs of abating.
But the cool new code we write today becomes the massive technology debt of tomorrow. And there’s nothing sexy or profitable about debt service.
Don’t believe me? Just visit the data center of any global financial institution. There you’ll find a mainframe running COBOL programs from the 70’s. That’s right, the 70’s. You know, that time before computers.
Now multiply the technology debt from the 70’s by a few orders of magnitude. Think of all the code we’ve written in the intervening decades and all the places it’s running. Add to that all the code we’re writing now and will write in the coming years. The apps, the databases, the AI.
It’s unsustainable. It’s not an immediate pain-point. So like climate change, infrastructure, and all our other long-term challenges, technology debt is a can that capital just keeps kicking down the road.
Meanwhile, we keep producing massive future technology debt at an unfathomable rate. Worse yet, we encourage young people to become coders and add to the burden.
A lot of this code is bad. A lot of it is unsecure. And all of it – 100% — is future debt.
Technology debt is not an abstraction. It has a real cost. You have to allocate time, money, and talent to old code. Or, alternatively, you can ignore it – in which case you pay for your neglect with poor business performance, non-competitive customer experience, and outages.
Technology debt is why you pay Microsoft a tax just for owning a CPU and having to run Word. It’s why the IRS shuts down just to modify a few pieces of business logic. It’s why Walmart had to spend $3.3 million on Jet.com.
Of course, no one thinks they’re creating debt when they roll new code into production. After all, their code is brand spanking new. It has been birthed by state-of-the-art code cultures with all the right attributes: Agile, DevOps, Continuous Delivery.
But it’s still all debt. And you’ll never re-platform your way out of it. Heck, I’m writing this on a platform that dates from the Reagan administration.
Can we avoid technology debt entirely? Of course not. But we can:
• Price debt into the code we produce today – instead of persisting in our denial-based valuation of intellectual property.
• Better service the technology debt we already have, including our mainframe code – some of the most important code on the planet running on some of the most awesome machines on the planet.
• Stop being so juvenile and manic about coding. Not every kid needs to be a coder. We need plumbers and welders, too.
In fact, maybe we should ease up on coding for minute while we address what is perhaps our most egregious “technology debt” – the roads and bridges and airports and water/sewer systems we have neglected for decades.
We can’t code our way out of debt. We can, however, choose to more prudently manage it.
Big Data, Bad Metrics
As technology provides us with more access to more data, a lot of
attention is being directed towards leveraging that data to improve outcomes. The popular notion is that by gleaning insights from so-called “Big Data,” we can make better, faster fact-based decisions
—and that these decisions can move the needle on everything from business performance to human longevity.
This notion is borne out to some extent by empirical evidence. If you collect and analyze customer data, you can sell more. And if you measure your food intake, you can lose weight.
However, not everything that readily lends itself to measurement is important. And not everything that is important readily lends itself to measurement.
Measuring the Wrong Thing
In the June 2014 issue of Harvard Business Review, Clay Christensen and Derek van Bever make this very point about investment. They assert that flawed metrics often drive companies to eliminate jobs rather than create them. They demonstrate that ratios such as return on invested capital (ROIC) incentivize corporate decision-makers to simply whittle down denominators (by, say, outsourcing), rather than do the hard work of adding to the numerator. They further argue that a metric like ROIC is obsolete, because capital is not the scarce resource it once was.
The result is short-term investing that fails to generate growth or jobs. And these results will, if left unchecked, ultimately work against the interests of the very capitalists for whom return ratio metrics have become orthodoxy.
Similarly flawed metrics dominate the contact center industry, where managers chase KPIs such as shorter call-handling times or higher customer satisfaction scores—neither of which have much to do with desired business outcomes such as greater long-term customer loyalty or more market-aligned innovation. But, because these metrics are easily understood and easy to capture, they drive counter-productive and toxic behaviors.
How’m I Doing?
At a recent technology conference, I found myself sitting next to a bright young market analyst who waxed enthusiastic about the Quantified Self. He went on at some length about how the principles of data capture and analysis could be applied to the individual in order to improve physical, financial and even emotional outcomes.
My question to him was a simple one: “What is your ethical metric?”
The question was, of course, rhetorical. And it served its rhetorical purpose by leading to a discussion about whether we are in fact primarily physical, financial and/or emotional beings—or whether we might be in some way moral beings as well. We then discussed how we might rightly incorporate moral imperatives into a life in which physical, financial and emotional imperatives play such a dominant role.
Ethical goodness and long-term socioeconomic impacts don’t easily find their way into our spreadsheets and BI toolkits. So in our zeal for metric-driven personal and institutional behaviors, let us also exercise wisdom. Otherwise we may succeed in making our numbers—but fail in making our world.
DataOps: Why Big Data Infrastructure Matters
Many organizations approach Big Data almost exclusively from a data science perspective. They know that the massive data they now have access to contain high-value insights, and they’re trying to determine what kind of analytics they need to extract that insight.
But if you don’t also re-think your infrastructure, you’ll be in trouble. You can’t simply throw data science over the wall and expect operations to deliver the performance you need in the production environment—any more than you can do the same with application code.
That’s why DataOps—the discipline that ensures alignment between data science and infrastructure—is as important to Big Data success as DevOps is to application success.
Stale results are no results
Speed counts. In fact, some results are completely useless if they aren’t delivered in real time. So if your infrastructure can’t deliver near-wirespeed performance, you’ll never be able to reap the full potential business value of your ever-expanding data resources.
Of particular importance is data intake. Big Data projects often get derailed because everyone was too busy thinking about analytic processing workloads on the back end. But intake can be the thornier problem. It’s not easy to prep large volumes of disparate data for analytic processing. Data has to be validated and rationalized. And, again, this has to happen fast.
Incorporating legacy mainframe data into Big Data environments can be especially challenging. EBCDIC-to-ASCII conversion is a beast to normalize—and it can take days to execute that conversion if you don’t have the right infrastructure in place.
The cloud is not enough
Scalable, low-cost commodity cloud infrastructure—both from as-a-service providers and in the datacenter—is awesome. Unfortunately, that capacity can’t solve every performance problem.
In fact, infrastructure models that require you to move lots of data in and out of memory are going to create bottlenecks that will ultimately prevent you from achieving the real-time results your business demands.
So, sure, there’s a lot you can do with Hadoop running on cloud VMs. But you’re not going to make your Basel II reporting deadlines if those are the only cards you have to play.
No future for one-hit wonders
It’s also important to bear in mind that Big Data success isn’t about just getting one type of analytic result from one set of data sources. It’s about adaptively performing multiple types of operations—including operational analytics, predictive analytics, mobile data serving and transaction processing—on whatever combination of data sources may be relevant to any given business objective, today and in the future.
That means your infrastructure has to deliver superlative peformance on everything from social sentiment gleaned from unstructured content to anomaly detection gleaned from IoT telemetry.
In other words, infrastructure isn’t commodified just because VMs and open-source solutions are. Infrastructure that cost-efficiently delivers differentiated performance by adaptively aligning with ever-changing Big Data workloads will definitely provide a competitive advantage. And if you don’t have that, even the best data science can’t help you.
Enterprise Mobility and Time-Independent Work
Because we use the word “mobile” to describe the increasingly useful wireless devices now at our disposal, there is a tendency to understand their impact primarily in terms of location-independence. If we closely observe how work is actually performed in the enterprise, however, we may find that it is by leveraging these devices to make work time-independent that we are more likely to achieve transformative value.
Consider an account rep about to embark on a roadtrip to call on customers in a number of cities. Such a rep is no more “mobile” today than in the past. Physical movement from place to place still depends on trains, planes and automobiles. Other than perhaps providing access to information about a last-minute gate change, a mobile device does nothing to enhance our rep’s physical mobility.
But without enterprise mobility, this rep would have had to prepare for such a trip by putting together reports in printed form or as files loaded onto a laptop. This might mean extra hours front-loaded on a Sunday night or early Monday morning before the trip. Our rep would also have to be super-diligent about putting together all the right information before leaving—because it would be difficult or impossible to add to that information once on the road.
Enterprise mobility dramatically time-shifts this work. With the right combination of mobile BI and content access, our rep can defer information assimilation tasks until just before each appointment. This allows our rep to avoid the intensive front-loading of these tasks and to walk into each appointment with more timely content. Perhaps even more important, our rep can apply lessons learned on Monday and Tuesday to appointments on Wednesday and Thursday—introducing a kind of adaptive, improvisatory quality to account management that was never possible before.
Of course, like the rest of us, our rep can also check emails early in the morning, late at night, and during the sundry other moments the day affords in restaurants and taxis. So what has changed about our work is not that we can collaborate with colleagues on another coast. We’ve always done that. What is new is that someone in San Francisco can ask me a question at the end of their work day and get an answer from me even though, given the fact that there is a three-hour difference in our time-zones, I’m already kicked back after dinner watching a ballgame on TV.
This is not news to those of us who carry our workspace in a pocket and a shoulder bag. But it is something important to consider when planning and pre-imagining the new mobile enterprise. When work leaves the office, it also leaves clock—and this atemporality has profound implications that mere mobility does not.