Obsessed with tying effort to impact, Mark Holt has given all his teams at thetrainline.com 100% customer-centric KPIs that are measured monthly. The aim? To accurately track the effectiveness of each team in moving the dial for customer benefit, and to change what isn't working to what does work. When did you start your current role?February 2014. What is your reporting line? To CEO. Do you meet with and discuss business strategy with the CEO every week? Yes. Are you a member of the board of directors? Yes. What other executive boards do you sit on? None. Does your organisation have a CDO? No. What non-technology responsibilities do you have in the organisation? Accountable for growth of our mobile business – one of our top growth strategies. I also have product responsibilities for various components of the customer experience including journey search. How many employees does your organisation have? 280 permanent, 500 total. Does your organisation carry out significant trade in the EU? Yes. How many users does your department supply services to?It's a B2C product, so we provide services to over six million visits each month. How do you ensure that you have a good understanding of your business and how your customers use your business's products?We structure each of our development teams into customer-centric clusters that are durable and contain all the skills necessary to deliver (product owners, BAs, developers, QAs, support, UX, etc). Each team is responsible for driving either one or two clearly defined KPIs that are 100% customer-centric. These teams use qualitative and quantitative insights to drive improvements in customer experience. The quantitative insights are gleaned from tooling such as:• application performance management, which provides insight into real end-user errors and end-to-end performance• AB testing using multivariate tools to gain 99% statistical confidence that changes are improving user experience (or not)• consolidated application logging (using ELK), and looking at the major drop-offs in conversion from specific pages that point to usability issues• several clusters include a dedicated insights manager role that brings together funnel data with cohort analysis and other back-office data sources to help ensure we focus our efforts on the most valuable opportunities From a qualitative side, we bring customers into our office almost every week to sit with a cross-section of the team, including both UX people and programmers. These sessions are videoed and more tooling is used to show where users are clicking or are getting stuck. Finally team members regularly get out to train stations to meet and demonstrate prototypes of new UX to real customers. thetrainline.com technology strategy and agenda Is your organisation being disrupted by the internet, mobility or technology-oriented start-ups? No. Are you empowered by your organisation to disrupt from the inside? Yes. Describe a disruptive measure you’ve led or played a major part in? Complete reorganisation of all development into cross-functional, product-centric development teams, with the goal of localising decision-making to the closest point to the customers, and empowering all team members to innovate and 'do the right thing'. It's a massive cultural change that has had significant benefits and far-reaching impacts. Dedicated product management and user experience (UX) teams have been introduced, along with NewRelic (APM) and centralised log management to gain deeper insights into real user experience. It's a big cultural improvement that has helped teams focus on site performance (highly correlated to revenue) and reliability. I've also made sure developers get out to stations to interact with real customers. What major transformation project has been recently completed or is under way at your organisation? As above, plus:• ongoing migration from owned infrastructure to 100% cloud• reference Architecture project to transform software architecture to microservices• ongoing cultural change to focus on building software that is secure from outside• moving towards continuous delivery, with some teams already deploying multiple times per day• bringing mobile app development in-house, with huge increase in innovation and time to market• changing to a culture that focuses on end-user performance as a key success factor What impact will the above transformation have on your organisation? • very significant improvements in time-to-market for innovation• increased agility around infrastructure• improved joy for development teams How has your leadership style contributed to the outcomes of the transformation project?I am obsessed with tying effort to impact. All teams now have 100% customer-centric KPIs that are measured on a monthly basis and reports distributed. This means that we can accurately track the effectiveness of each team in moving the dial for customer benefit. I have an obsession with moving quickly and reducing time to market. I am also instilling a culture where change is a positive (still working on this one). What key technologies do you consider enable transformation? Transformation is about people, not really technology. But technologies that provide people with deeper insights (application performance management, crash analysis, log management, deep tagging) make a huge difference to transformation of customer experience. Are you increasing the number of cloud applications or infrastructure in use at your organisation? Yes. What is your information and data analytics vision for the organisation? As explained above, we are obsessed with quantitive insights. We capture and analyse the maximum data we can get our hands on. Many of our teams have dedicated insights managers who provide customer insights. How is mobile and social networking impacting operations and customer experience? Mobile is the future for our business. Social networks are used extensively for support; we have a dedicated social team who provide support via these channels. Describe your strategic vision towards shadow IT and BYOD. How do you influence and engage executives and employees around choice? Having a good understanding of KPIs is critical. I don't really recognise shadow IT because I don't feel that I have to own everything to do with technology. I want to know that the architecture is sensible and that appropriate business decisions are being made. What strategic technology deals have been struck and with whom? • Amazon Web Services• NewRelic Who are your main suppliers? Thoughtworks, CapGemini, JKT, BT, Vodafone and Oracle. thetrainline.com IT security and budget Has your organisation detected a cyber intrusion in the last 12 months?Yes. Has cyber-security risen up your management agenda?Yes. Does your organisation understand the potential cyber-security threats it faces?Yes. Has this led to an increase in your security budget? Yes. What is the IT budget? Significant. How much is the IT operational spend compared to the revenue as a percentage? 4%. What is the strategic aim of the CIO and IT operations for the next financial year? 1. Security by design2. Improve qualitative and quantitive insights3. Continuous delivery4. Migrate to AWS public cloud5. Migrate to micro services architecture6. Improve performance and reliability7. Ensure that all the tech teams are having funSpecific KPIs are defined for each of these, and the KPIs form part of my direct reports, and their direct reports' appraisals. Are you finding it difficult to recruit the talent you need to drive transformation? Yes. Has recruitment and retention risen up your agenda as a CIO? Yes. Are you looking for recruits in the EU to fill the skills shortage you have? Yes. Does your IT organisation operate an apprenticeship scheme?No. thetrainline.com technology department How would you describe your leadership style?Enthusiastic! I love new technologies, and I love people that get things done Explain how you’ve supported and developed your senior leadership team to support your overall objectives and visionReorganisation of leadership team and introduction of new roles (security director) since joining. How many employees are in your IT team?200. What is the split between in-house/outsourced staff? 80% in-house. Does your team include key skilled workers from the EU? Yes.