As in 2015, the maturation of big data technologies has been a strong theme this year, but with more stress on implementations , and the gaining of business value, beyond science projects.
The broad Hadoop family of technologies, including MapReduce replacement parallel processing framework Spark , has made inroads in the UK public sector , as well as companies such as Amex Global Business Travel , Barclays and Sky Betting. And Now TV is among those companies putting NoSQL database technology to use in its case, to ease transmission of Game of Thrones.
Machine Learning is emerging as a central topic for the future. It is evident, here, in the stripe of cognitive computing represented byIBM Watson. Use cases are still incipient.
Data discovery suppliers, such as Tableau and Qlik, have captured much of the high ground in recent years, with the traditional players such as SAP, IBM, Oracle, and Microstrategy, responding by coming back onto their visualisation turf. But there is also a new wave of “third generation” suppliers germinating in Silicon Valley .
The Panama Papers data disclosure of this year , which showed how the rich, global elite hide their wealth, was made possible through the use of data visualisation software, Linkurious, on the basis of the graph database Neo4j, as provided by Neo Technology.
Arguably the most interesting data analytics story of this year, however, was how it enabled Leicester City to win the English Premier League a shock no one predicted.
Business Intelligence and data anlaytics are part of a broader data management story that we tell in Data management: fifty years of prospecting for business value , part of our Computer Weekly at 50 article series. It shows how business intelligence arose from data warehousing in the 1990s, and how it is developing today in the new, big data world of Hadoop and NoSQL databases.
1. Apache Spark grows in popularity as Hadoop-based data lakes fill upHowApache Spark is growing in popularity and finding real-time use across Europe, including in online betting and on railways, and with Hadoop.
2. NoSQL database technology finds use cases, but still minority sportFrom managing water meter data, through Bitcoin and video data, to web publishing, NoSQL database technology is finding real use cases . But it’s likely to be stuck at 10% of the market for some time.
3. Hadoop starts to trumpet way through UK public sectorRead how big data technology Hadoop is starting to appear in patches of the UK public sector, including GCHQ, HMRC, the Home Office and the NHS.
4. San Francisco data analytics early-stage firms point to third-wave BICalifornia start-up and early-stage data analytics companies are positing technologies that might point beyond second-generation business intelligence, such as Qlik and Tableau.
5. Foxy Leicester City FC won Premier League with data analyticsLeicester City Football Club was the surprise winner of this season’s English Premier League, thanks, in part, todata analytics.
6. Panama Papers technology shows data journalism at its bestHow the open source data analytics and visualisation software of the Panama Papers project empowers a new kind of journalism, according to the International Consortium of Investigative Journalists.
7. CW@50: Data management Five decades of prospecting for business valueData management and business intelligence have been at the heart of business value creation for decades. Read about how Computer Weekly has tracked their promise and tribulations. The article concludes: “Whatever may be the technical shape of future data management architectures, the field can only develop as one of the main bearers of business value, coming from IT, for corporate organisations well into the future”.