African telco takes an 'extreme sport' approach to B2C billing development

To be one of only three developers responsible for designing and building a B2C billing system from scratch, and then roll it out in one go over 16 hours to two million customers, could be the stuff of nightmares.

Or you could see it is a type of extreme sport for developers; a sure test of your mettle. Leon Nortje, Development Team Lead, Billing, Rain, falls into the latter camp.

Nortje (center), Thashil Maharaj, Senior Full Stack Engineer, Rain (right), and Michael Du Plessis, Senior Software Engineer, Rain (left), caught up with Insight to discuss how they used TM Forum’s Open APIs to build a new billing system from the ground up for South Africa’s Rain.

Nortje’s first challenge was to convince Rain’s board that it made business sense to build a new billing system in-house using TM Forum Open APIs.

A focus on simplicity and customer-centricity helped his case. He presented the new system as three blocks: accounts, customers and invoices.

“We wanted to change the focus of the billing system towards the service side, instead of billing and collection,” explains Nortje.

There was also a pressing need within Rain for flexible product development.

“Our billing platform at that stage couldn't handle new products. Now we can push a new product out in the day or two,” says Nortje. For example, “if customers want more SIM cards, they no longer need to buy a new product. Instead we can change the cardinality. And it works.”

Indeed, “our whole architecture revolves around, technically, how the product catalog works,” he explains. “That is where all of our business rules sit in how we define structure and offerings and all the rules around an offering.”

Importantly, the new system, which went live in April 2025, is considerably less costly than the one it replaced.

“The salary of the three of us for [the equivalent of] two years ... is less than the cost of running the previous billing system for one month,” says Nortje.

It addition, it took only two to three months for the new system to deliver a return on investment in equipment.

“We were running 80%, 90% CPU on massive boxes,” says Leon. “We've trimmed down boxes to a third of what it was, and it's not even reaching 60% CPU yet.”

Developing backwards

For the first two years of the project Nortje worked alone. Among his tasks was rebuilding the templates for 35 TM Forum APIs for which he used Intent Architect.

“It passed 92% of the CTK (common tool kit) test cases without any logic behind it. You can add, you can insert, you can delete, you can do the basic stuff.”

By the middle of 2024 Nortje was ready to start collaborating with other developers on the business logic.

He favoured using TM Forum Open APIs off the shelf to build the system because they are standardized, and have been fine-tuned for specific functions and road-tested in commercial systems by TM Forum members.

But it meant developers would have to design the system backwards from the APIs, while keeping an eye on the big picture. That did not chime with every developer’s way of thinking.

“A lot of new developers look at the code, and it's like, oh no, there's too many variables,” explains Nortje, adding: “As a developer, you want to strip it down because it's way over-bloated.”

But stripping away variables brings its own form of complexity and takes time. In addition, removing what you don’t need today risks closing doors to what your business might want tomorrow.

Fortunately Du Plessis and Maharaj were ready to take up the challenge, which entailed a very tight delivery deadline of February 2025, and joined Nortje in September 2024.

Tried and tested

Close adherence to TM Forum models helped them forge ahead.

“We've locked down on the models. We are very strict. The models that are defined in TM Forum, we use that religiously,” says Nortje. “If it says many to many, we make it many to many, and we'll figure out why later.”

Initially, for example, they thought that having separate service manager, resource manager and product manager components was overkill. “But if you start dividing it properly, like TM Forum did, it 100% makes sense that you've got a tool that just does resource allocation,” says Thashil.

In addition, “if a lot of companies have already used it we may need it in the future,” says Leon. And it means “expansion is not a problem because TM Forum has thought of all niggly bits.”

All systems go

The team succeeded in developing the core billing functionalities for three Rain Mobile, Work and Home currently live on website products by end of February. They then had to transfer legacy products and their customers to the new system.

There were of course bumps in the road. The team could not take a phased approach to mapping and moving accounts, payments and invoices because it was impossible “to strip pieces out of the existing system,” explains Michael. “So even if we could get the account part to move out, we would have had to copy everything back into their system to map it back. We [tried] it for two months. It was a mess.”

Nonetheless by March comparative tests of performance between the new and old systems revealed only 0.25% billing discrepancies. They scheduled the new billing system to go live over the Easter weekend of 2025.

Customers could use still data services, but otherwise on 17th April “we shut everything down and we started deploying from top to bottom.”

Over the course of 16 hours the three members of the team rolled out the new system, replacing websites, customer support, as well as the mobile app for customers.

“We replaced all the entry points into the business. We replaced the whole billing system, and the OSS charging system that was part of the old billing system. And we got everything running,” says Leon.

“We used REST endpoints to be compatible with TM forum, and then we also had a gRPC endpoint to make API communication quicker, more responsive and stream-based.”

In addition to cost and efficency gains and the ability to roll out customer-centrice products much more quickly, business divisions now have relevant data at their fingertips.

“The company's changed a lot, because so much of everyone's work was around processing on ... database systems,” says Michael. “Now people are focusing on their areas more.”