TM Forum is a global industry association that drives collaboration and collective problem-solving to maximize the business success of communication and digital service providers and their ecosystem of suppliers.

The Forum’s vision is to help communications service providers and their suppliers to digitally transform and thrive in the digital era. They do this by providing an open, collaborative environment and practical support which enables their members to rapidly transform their business operations, IT systems and ecosystems to capitalize on the opportunities presented in a rapidly evolving digital world.

As a neutral, non-profit member organization, TM Forum represents over 850 member companies generating US$2 trillion in revenue and serving five billion customers across 180 countries.

“My responsibilities include analytics and business intelligence,” says Richard May, VP Technology and Community, TM Forum. “We’re a membership organization covering 160,000 individuals, and we need to understand their engagement. If a company is more engaged, it’s more likely to be active with us, and so gain more value. It’s also more likely to renew membership. So engagement is a critical success factor.” The operational insights provided each day are used across TM Forum, from tracking specific engagement measures to high-level operational dashboards used by the leadership team.

Challenges: ‘improving agility’

TM Forum had deployed a variety of solutions to collect data, including member and partner data, feeding into a legacy data store. The organization was working towards a strong central point for data integration, data improvement and analytics, with output transferred to the Birst visualization and BI platform for dashboards and reports. With new data sources brought in via direct transfer, this set-up required time-consuming and resource-intensive data preparation.

May says, “We started with a best practice approach but it was becoming cumbersome, hampering our agility. If we wanted to measure something new then accessing data from the source, blending it with existing data and getting it to users took 2-3 months. So there was an agility issue, it took too long, plus the cost of maintaining that. If we didn’t change anything, it remained stable. But once you introduced new things, you started seeing instability, with an erosion of trust in the users.”

“Our model was the wrong way around: we were very light on data insights, dashboarding and reporting, and instead spent more time, resources and money on developing the data set, building custom SQL jobs to get data from different sources, when technology was already available to easily pull data in and perform ETL. We also realized it might take 18 months to make the changes necessary.”

Why Datalytyx?

TM Forum wanted a scalable solution that was less brittle, delivered higher performance and integrated all data within a single platform. This would enable TM Forum to realize the value of existing data and quickly assimilate and leverage new sources. “We wanted a partner that understood what we were trying to achieve and could build a solution with strong foundations, and who also offered expert support where we lacked knowledge internally. Datalytyx offered its Palladium platform-asa-service and it’s a perfect fit. This included flexible resources, meaning we can call on a developer or data scientist as and when needed. We’re not hamstrung by our internal capabilities.” 

Palladium: performance, service, value

The Palladium platform-as-a-service uses Snowflake as its cloud data store, providing a secure, scalable and high performance solution. Virtually any type of data is ingested rapidly, with automatic scaling based on storage and compute resources required, and allowing for simple data sharing without any negative impact on performance. “We were agnostic on the choice of database,” says May. “We wanted cloud-based technology that could scale to our needs and potentially offer a better cost mode but also new features, like plugging in different visualization tools with ease and its Time Travel capability. Snowflake was a better fit.”

The platform also leverages Talend data integration technology, providing connectors for almost any data source and enabling fast development of data processing routines to support high quality output for analytics and reporting. May says, “We previously used lots of stored procedures to bring data in, and provide transformation logic. We’d built these in a non-standard, custom way. We also had single points of failure in our support structure. We built Talend versions of those procedures and the Datalytyx team added a lot of value. Datalytyx was also sensitive to our budget and worked hard to deliver what we needed. Having a partner with that flexibility is so important.”

Outcomes: processing times reduced from 12 hours to 20 minutes

TM Forum has significantly reduced the time required to prepare data, gaining a next-generation future-proofed architecture. May says, “Datalytyx has trained people who understand our set-up and requirements; they can make changes to our Talend jobs quickly and easily rather than a developer perhaps taking weeks.”

“Automated by Talend, execution by Snowflake takes a fraction of the time compared to our old jobs. With some monthly processes, it used to take 12 or 13 hours to process data. Now it takes 20 minutes on the Palladium platform. In testing, we identified pre-existing problems with SQL jobs that are now resolved. So we have more accurate data.”

May adds, “This is a step change, and we are already providing improved value and better visualization to the business. Because we’ve taken care of the back-end, we can focus on value-added areas like visualization. Datalytyx is looking after Snowflake and Talend while also helping with the visualization piece. We’ve successfully flipped the model from being too focused on maintaining data sets and access to being able to move forward with greater agility. We’ve made resource savings but, more importantly, are now in a strong position to deliver faster and better insights to users.”