Capturing and maximising data as an asset
Paul Farnan, Enterprise Strategy Architect with Microsoft Services, discusses the exponential growth of data and how public services can optimise the opportunities that it presents.
Microsoft believe there is a real opportunity for government organisations to exploit data to improve public services for their departments and for citizens. This begins when organisations start to understand their data, and consider that data as an asset in the same way as their people, facilities or real estate. Paul Farnan is seeing this trend as he consults with large and small organisations, public and private sector. He helps them to understand key technology trends, and harness the potential to use data effectively within their organisation.
“The reason we see data as so important is that we believe it’s a foundational building block for information, knowledge and intelligence within organisations,” Farnan relates. The quantity of data being collected and stored is growing “at an exponential rate” as organisations take on new technology which can exploit data.
According to some estimates, the global amount of digital information is increasing 10-fold every five years with 85 per cent of that increase coming from new data types such as sensors, weblogs, RFID and ‘the internet of things’. The growing scale and scope of data presents “a huge opportunity” for government if it is harnessed correctly.
Three drivers for that growth stand out:
1. the processing power of devices and ‘always on’ connectivity allows data to be collected in a way that was not previously possible;
2. the reduction in storage costs has allowed data to move from an archived mode to real-time accessibility; and
3. the combination of enhanced processing power and lower cost in the cloud allows organisations to improve their analysis of data and identification of specific patterns.
The data generated which is of interest to organisations comes in various formats and incorporates:
• personal data created by employees such as documents or presentations;
• organisational data which is maintained and managed in applications and systems across the organisation; and
• community data which is typically sourced externally or via certified third party data sets, and world data which is readily accessible online.
All these types of data provide a wealth of untapped information where we now have the potential to transform into knowledge and intelligence that can be distributed and shared to produce greater efficiencies.
For example, in New York, Microsoft Services have helped to develop New York City’s situational awareness system. He explains: “The Mayor and the city’s emergency services basically aggregated thousands of pieces of data from various public sector resources and, as a result can now get a one-stop shop view of the entire state of play inside New York City.” The system’s dashboard gives a ‘point-in-time’ view of emergency situations across the city including immediate video footage, and the contexts of weather conditions, transport system usage and the resources available to respond. The value of combining all these disparate data points to address specific scenarios has delivered unprecedented benefits.
The importance and value of data being processed and maintained by government agencies and bodies has been recognised in recent Irish Government policy both in the Public Sector Reform Plan and in the more recent Civil Service Renewal Plan and there have been some pockets of progress and success. The challenge still remains in turning this policy into delivered outcomes on a broad scale across government. Departments need to think about what value can be created by combining traditionally siloed data sets. For example, within healthcare, data gathered on the type and frequency of treatment can be matched with diagnosis plans to ensure that treatment is balanced, and appropriate to an individual’s needs. Additionally, insights from anonymised medical histories can be used to predict prescriptive and individualised treatment plans.
Capabilities
Microsoft identifies three main platform capabilities that an organisation needs to put in place in order to realise the value of data.
The first centres around data consumption: “How it takes structured or unstructured data in from its current systems and also from external systems.” Secondly, the organisation must integrate and develop insights and queries from those data sets, a process that allows it to “ask questions that haven’t been asked before”. The third element covers how data is published and visualised, not just in a visual way but also in application programme interface (API) formats to allow other individuals and agencies to make use of them.
Focusing on the practical process, Farnan adds: “The first step is to encourage organisations to focus on the outcomes, identify where the most resource-intensive and costly processes are, and seek to optimise those processes through data points.”
An organisation also needs to consider which data points to gather from non-traditional sources. It may not have been technically feasible to collect those data points in the past but there is now an opportunity to pull them together e.g. through the use of sensors.
Following on from that, a data-oriented culture needs to be driven through the organisation. This mind-set shift involves bringing together multi-disciplinary teams – typically with a mix of technology, business, problem-solving, and research and analysis skills. All these stakeholders must have access to the data using tools that are easy to use and familiar to them. It is no longer acceptable for either the data or the tools to be a specialist IT capability. “We call this democratisation of data, giving access to the right data to the right people at the right time.”
Data classification
“Once you have the data collection element sorted,” he explains, “it’s a matter of classification understanding the data you collect in your systems, how they relate to each other and also how the data you collect relates to the data collected by other organisations.”
Data classification is very important for differentiating sensitive information from non-sensitive information. Microsoft generally classifies data according to low, medium or high business impact with that ranking related to the impact of exposure. Personally identifiable information, if exposed, would have a high business impact. Differing policies, governance and security models are applied to each of these levels to enable organisations to effectively balance the risks and rewards.
The final step is to “get the tools and start experimenting with the data quickly.” Farnan sees the days of 18-24 month data projects as a thing of the past and explains that organisations should be looking to unlock data in a matter of weeks. Everyday tools such as Microsoft Office 365, Excel and SharePoint offer a good starting point.
Microsoft encourages organisations to not only build data but share data (via open data APIs) as already happens between government organisations. APIs are an interface point into a data platform, allowing access in a machine-readable format: “We have seen examples in government departments today where sharing data via APIs has effectively eliminated huge amounts of enquiries over the counter at certain department offices, in one specific case virtually eliminating 50,000 enquiries annually.”
The potential value is not just an inherent gain for government departments’ efficiencies but also for improving public services in general. For example, geographical data such as GPS
co-ordinates can be mapped out against employment and Census data to illustrate local labour markets and their age profile, and help to shape job creation policy.
Organisations need to utilise the platforms they have in place today such as Microsoft SQL Server and look at how they can expand that capability to process even greater amounts of data. Farnan concludes: “Trusting the sources of your data and having a platform which provides secure, well-processed, clean data is critical as well as having both the API platform and the self-service tools to unlock the value inside those data.”
For further information, Paul Farnan can be contacted by email:
pfarnan@microsoft.com
Find out more about Microsoft Services here:
www.microsoft.com/microsoftservices