Conversation with Una Shortt

Una Shortt is a senior leader in the Data & Analytics Industry. Having discovered STEM by chance, she found a passion for engineering that has since accumulated in over 20 years in a dynamic career in Enterprise Data & Analytics Architecture and Data Governance. Una is currently Vice President of Data Platform as a Product at Schneider Electric, a company she joined in 2004 as a Data & B.I. Developer, and a key contributor to the evolution of the Digital Hub of Schneider Electric in Barcelona.

Let’s start this conversation with a travel in time. In 1996 you started to study Computer Systems. What was driving that choice?

Certainly, a travel back in time! I studied Computer Science by chance. The course of my choice, Journalism, was not available locally, and for economic reasons, I needed to choose a university where I could commute daily from home. And so, at a crossroads in my decision and with some much-welcomed mentoring from my older brother (who was 3 years my senior), I followed in his footsteps to study a BSc in Computer Science in University of Limerick, Ireland. I listened carefully to his advice, focused on learning how-to low-level code in year 1 and spent most of my day in the computer labs on campus for the 4 years. With retrospect, I am not sure I could have found a greater fit for me! The bases of logic & approach of coding stands to me to this day. So, I have a lot to thank my brother for.

In fact, the early years of your professional career were related to software development and architecture projects. When did you start feeling the call of Data and Business Intelligence?

What a great question! I guess I was always working with data but was considered a Software Engineer in my early career – programming with C++ and Pro*C primarily. I officially entered the world of Data & BI almost 20 years ago; it was a case of being in the right place at the right time. I took a chance by applying for a super interesting role in a vibrant company, knowing I didn´t perfectly fit the requirements. They were at the cutting edge of implementing Data & Analytics at scale in the 2000s. I was interviewed by someone who saw my potential, and under his mentorship, I learnt the foundations of Data & BI in a few short years. That company was APC, purchased by Schneider Electric in 2007. And where I continue to enjoy my adventures in data today.

In some respects, the role was easy; I was a low level coder, but could now use various market technology that meant a lot of the “heavy lifting” of data processing & visualisation was being managed. In fact, I missed the opportunity to code (although we had plenty SQL to write!). On the other hand, I discovered I was no longer in a pure IT world. We had to sell, market, promote what we were doing, influencing and convincing our internal stakeholders that we brought benefit beyond their usage of Excel. We were also a small team and we all played multiple roles – I was Project Manager, Business Analyst, ETL Engineer and Viz Expert. It was an exciting and challenging journey.

Let’s focus then on the Data Side. You had been working for a few years as Business Intelligence Manager, so tell me in your own words what is Business Intelligence

You know, it’s interesting, as the industry has evolved significantly since those days. But yes, my first leadership role in Data was as “Business Intelligence Manager”.  Focusing within a specific Business Unit at the time, we managed the E2E Data Lifecycle – not only analytics. And that is true to the definition of Business Intelligence, as without good quality data, from trusted sources, transformed with business logic, we cannot reach the Decision Insights expected as an outcome. So in summary, I would say B.I. has been the “founding father” of Data Driven Decision Making - as it was through those first views of B.I Reports that businesses learnt all the dependent methodologies and standards required in delivering trusted Data Insights.

And what do you need to have a successful Business Intelligence? Which profiles are needed? And which platforms?

This is an interesting question, Manuel. There is a full suite of skills needed to delivery Business Insights – from the knowledge of the various data domains to data analysis & design, data mapping, data quality, data engineering and visualisation – not to forget the very important infrastructure teams, Project Managers, Scrum Masters and Communication specialists.

With the Data Industry being so vibrant, we are limiting ourselves if we look for a very specific list of data or technology experience on a resume. So yes, we have technologies & platforms we use – but we may find it difficult to find someone with an exact match. Instead, we need to look at candidates holistically, to consider their abilities and attitude also, which broadens our opportunities to hire a more diverse team.

Specifically in Schneider Electric, we complement this with a global, multi-hub strategy enabling us to attract top talent globally. Barcelona is an important location for us in Digital talent, and thus we have Barcelona as one of our global Digital Hubs.

Tell me your 3 Dos’ and Don'ts to build a BI system

Do

• Do your research on Data Risk & Regulations – the world is ever changing, with regional and country regulation for data

• Do provide governed autonomy – we have spoken about the power of centralized/decentralized models for 15 years. It has not changed – what has changed, is the Governance you need to instill to be successful in this model, and the scale with which we are now in

• Do be clear on what the system is and is not – know your product and assure your stakeholders are clear on what features & capabilities it should provide, and what it should not.

Don’t

• Don’t build and “expect them to come” – you must understand what data the business needs. This will also secure their support in cataloguing the data and securing data quality

• Don’t automatically build custom solutions or frameworks where an out of the box solution exists. If there is no market solution, consider discussing with your key vendors to influence their product design roadmap: If you need a specific data platform feature, it is highly likely other companies need it too.

• Don’t ignore your key stakeholders – assure you identify the various personas & roles for your platform. Once you identify them, enable them to contribute to the Product Feature definition and evolution that fits their role

One of the main challenges for any BI team is the quality of data and facing complaints such as “this report is wrong, it does not reflect real numbers” or two different teams having different sources of truth. What is your recommendation to tackle this issue?

Governance, governance, governance. Governance in assuring ownership of data, how we create & share the data, how we secure the transformation rules applied to the data, and how we quality control & publish the data. To help our teams, we should assure clear governance guidelines on how data is produced and consumed; and our Data Strategy at Schneider Electric has really begun to nail this down in recent years with our Data Golden Rules – which includes key principles on building data for reuse, data cataloguing & quality, authoritative sourcing for specific datasets and consumption via our group data platforms.

In the last years there has been a significant shift from classic ETLs to ELT models. What is your take on that?

I have enjoyed this transition, as moving to ELT (Extract, Load, Transform) is hugely beneficial when we consider DevOps models. When we perform ETL (Extract, Transform, Load), the transformation (the “T” of ETL) is happening “on the fly”. This becomes a challenge when tracing full data lineage in root cause analysis (RCA). Whereas if we load (the “L” of ETL) the data first, exactly replicating the source system at a specific point in time, we can more easily do full data lineage in our RCA. Having worked in Operations for many years, I am a proponent of the ELT methodology - servicing almost 60,000 analytics users globally, data traceability and lineage is key for issue management. This also helps assure reuse across everything we build. The more we transform prior to entry to the data lake, the more we decrease the opportunity for reuse of that data by various stakeholders.

And what do you think about Zero-ETL trend? What is your definition, by the way

Well, Zero-ETL should simplify our data pipelines while naturally better securing near real time analysis. But in effect, we could rebrand it to EL – Extract Load, a mechanism for moving data sets from source to target with no transformation. While this is a new buzz word, we have been doing this across the business for many years, but now it is being better enabled through evolved technologies.

From my experience, I see a lot of value in simplifying our data pipelines, to ease our DevOps in issue management. Building a data pipeline that passes source data as-is from A to B (“the EL of ELT”) clearly reduces “time to insights”, better enabling near real time and simplifies the support required. But we need to consider is the value in the data itself, as it was in the source system? If so, why not access the data directly at the source, in “direct to transaction” analytics? Or do you need to merge that data with other data sets from other systems, refining common rules for data consolidation and quality? And thus, transformation will need to be applied to the data upstream.

So ultimately, I believe we still need the “T” for Transformation, but without doubt, separating the first step as Extract and Load can bring simplification in our Data Operating Model.

You also have significant experience on the Data Architecture side, so I am curious to know your perspective on the proliferation of Data Lakes projects and even further evolutions such as DataLakehouse, Data Mesh, etc.

A centralized Data Lake/Lakehouse architecture has its place and can help secure the data foundations within an organization: as you learn your stakeholders, their use cases, what data is available, how to secure it, what common denominators - like master data, data hierarchies - should exist and how you should govern data within your Data Lake. But these components outgrow a centralized technical architecture as more adoption occurs across data domains in the organization. Data Mesh can be the game changer: if we are clear on what it requires to scale, transforming our thinking to product, data as a product.

We need to secure who are the owners of the data products – the Data Domain Owners - in the company. Yet not all data is easily segregated by Data Domain, and even when so, data is usable cross domain. Therefore, participating Data Domains in the Data Mesh must be prepared to service other domains with their own DevSecOps teams.

To determine this cross-Domain architecture, we need to consider how data will be accessed, shared and governed across the Data Mesh. And if leaving the centralized Data Lake for Data Mesh, it is worth every organization rechallenging their definition of their E2E Data Architecture: what each product should, and should not, serve. We can then secure a clearly defined microservices architecture with tools & technologies, delivering governed data products by domains, enabled through a common data platform architecture.

You are currently responsible of “Data Platform as a Product”. Could you give some details of the scope of your role and the methodology you follow?

My scope extends across Group Data & Analytics Platforms at Schneider Electric, with primary scope as the Enterprise Data Lake and Analytics toolkit, applying agile methodology. In my career, I have owned the technical execution of various streams of our Data & Analytics Strategy, including the leadership of the first Enterprise B.I. organization in the company. Moving into Product has been a transformation, as it is a whole new way of thinking: removing technology discussions from the equation. As I mentioned earlier, Product Thinking is critical in Data Management and quite a transformation for many companies. Focusing on the personas and roles of the product, a product mindset will assure the product is useful, while bringing clarity on why the product exists in the first place – assuring we think of the product holistically, securing its usability, features and business value outcomes.

And which profiles are you working with?

Variety is what really drives a successful team, and we have that within our Product team. With a strong mix in the team of Data Risk, Data Governance, Data Architecture, Data Services & Delivery experience. It is a valuable combination with the different perspectives enabling more diverse discussions on the product and planning increment sessions. We then partner closely with the Platform DevSecOps teams to secure the Product User Journey.

Which Technological Stack are you using?

We use a combination of technologies, assuring a “portfolio of tools” (not a single vendor model) to get the right balance in features from best of breed technologies. Our core foundation is AWS, utilising much of their service offering in our Data Lake, with Tableau our primary visualisation tool. We then use a variety of tools for data transformation (use case dependent), complemented by frameworks built internally. As the market for Data & Analytics is quite mature, we always target to reuse technologies we already have when fulfilling features or look to the market for latest trends. Even if I mentioned my love for low level code, with such a large scope, we try to avoid building solutions from the ground up if there is already a suitable product on the market, simplifying the DevSecOps cycle for our teams. We instead focus our innovation efforts in areas where we are extending existing market software or building unique features to Schneider.

Compliance and ethics are also becoming quite relevant to the Data world. Which tools do you recommend to tackle these challenges?

I will probably surprise you with my answer, to say the greatest tools in compliance and ethics in an organization are people and cross-organizational processes. Companies continue to rely a lot on human decisions in design and execution – so we need to assure compliance and ethics in the Data Industry by design. Technology can only facilitate so much – organisations need to assure that the processes & policies exist that secure the compliance E2E, such as Data Retention, Data Residency, Data Privacy and Data Classification. And the Policies must be living documents, where employees working with data have an awareness of compliance and ethics in their day-to-day operations.

Last question: in the STEM world there is a significant gender gap. What do you think we should do to close this gap?

It is key we can secure strong diversity across STEM industries, to bring that edge in Thought Leadership we all value – be it male or female. And here we all have a role to play. We must acknowledge the great work of many organisations over the past 10 years or so, and volunteers from our industry, bringing engineering to younger children – particularly through coding camps and in-school mentoring and coaching. Reaching both girls & boys early in the school system, at primary level, is key: while encouraging girls through sharing female role models in the history of STEM, inviting moms who may work in STEM to speak at the school, giving opportunities to deliver STEM skills. This is the moment to capture the minds of young people, to share with them the energy, possibilities, and creativity of the industries within STEM.