AI costs rise for Australian businesses

The Business

Topic:AI

Anna Volkova from software platform HiBob says the business cost of AI can be weighed up in human ways. (ABC News: Daniel Irvine)

In short:

Businesses are looking for ways to measure the return on the billions being spent on AI. Some say they are "years away" from judging bottom-line impacts.

In this story:

Australian businesses paying to use AI are weighing up what value they are getting from the disruptive technology.

"It's actually only really a relatively new thing that we have started to speak about," Anna Volkova, the head of people and culture at HR software company HiBob, told ABC News.

The platform has seen wins from using artificial intelligence, but it has been difficult to judge that against the mounting cost.

With AI capabilities changing so quickly, Ms Volkova said the business can "never catch up" with the need to redesign how it works to get the most out of it.

This has made it hard to work out precisely if the money is being spent effectively.

"We'll probably start to see impact to our bottom line in maybe two or three years, and I think most businesses are in that boat."

Despite the cost of AI tokens tumbling, businesses are spending more than ever. (ABC News graphic)

Jeremy Pell, whose firm Elastic helps companies to manage the cost of AI, agreed with that estimate.

"We're in a really interesting time at the moment," he said.

Jeremy Pell says companies have not yet reaped the bottom-line benefits. (ABC News: Berge Breiland)

Last week, Assistant Minister for Technology and the Digital Economy Andrew Charlton described the value flowing out of Australia via AI as a "distant sucking sound".

In a speech at the ANU Crawford School of Public Policy, he said Australians were already spending $5-8 billion each year on artificial intelligence.

That is just the start, with the government estimating local spending on AI could "plausibly" reach $20 billion to $40 billion annually within a decade.

Tokens adding up to billions

Even a few years ago, AI was not a line item companies had in annual budgets.

Now, in the current reporting season for ASX-listed companies, you can measure it in billions.

The technology spend at Australia's biggest bank, the Commonwealth Bank, increased from $2.3 billion in the 2025 financial year to $2.4 billion in this most recent financial year. It lists savings too.

"Measured gross benefits from AI use cases, including reinvested capacity, were [around] $200 million in FY26," its results presentation noted.

On Wall Street, investors are asking whether the billions invested by US tech giants in AI will ever be justified by returns, creating concern about an AI-fuelled stock market bubble.

Stu Scotis, AI leader for consulting firm Deloitte, said the equation of how to spend an IT budget is getting tougher.

Stu Scotis says there are more uses of AI for increasingly complex tasks. (ABC News: Patrick Rocca)

Clients are weighing up long-term information technology costs — like storage and refreshing laptops — with a new, runaway expense.

"What we used to plan for as a static activity is now somewhat variable," he said.

Mr Scotis said current conversations are around "tokens": units of data processed by AI, or essentially the amount of grunt used to compute tasks.

Complex requests use more tokens, simple ones fewer.

The cost of tokens has plummeted in recent years, with increased competition, more efficient chips and more supply.

An unprecedented amount of data centre construction has increased the computing power available in the world.

Adding in more energy-efficient microchips, increased competition and the technology improving how much it can compute per token, it is now cheaper than ever.

As the below chart from research institute Epoch shows, the cost of the most powerful models has fallen the fastest.

Epoch used the "Massive Multitask Language Understanding" benchmark to rank AI large language models.

The price for large language models GPT-4 Turbo level or better (the most powerful, shown by the blue line) fell from $US15 ($20.90) for one million tokens in November 2023, to 17.5 US cents (24.4 cents) per million in February last year.

Research institute Epoch found the cost of the most high-powered tokens has fallen the fastest. (Supplied: Epoch)

Despite this trend, what companies are spending on AI is rising.

That is because firms are finding new tasks for AI to take on, leading to increasingly difficult computations, requiring more tokens per request.

AI not necessarily cheaper than workers

Mr Scotis put it another way: Human workers are paid in dollars, digital workers — or AI models — are paid in tokens.

"You want to get value for the digital workers and their output," he explained.

That means companies investing in AI need to examine the "value, outcome or growth" that comes from the tasks they complete, just as companies evaluate the work of human employees.

Elon Musk's latest venture sent global stock markets to dizzying heights, but its performance since does not augur well for the company or the future of the AI boom.

He said the human workers should not be too worried about being replaced.

"The cost of a digital worker doesn't necessarily mean it's cheaper than a human," he said.

"The complexity of the task, the length of time which a digital worker runs can, in some instances, be more expensive than a person."

Mr Scotis said firms needed to make a judgement "as to where you want to invest that type of work in your organisation, and does it make sense to do so".

It is a question being weighed in boardrooms right now.

How do companies pay for AI?

There are multiple ways that companies pay for artificial intelligence.

Some use "seats", paying per employee that accesses the technology.

This model is often used with platforms like Microsoft's Copilot, where AI abilities can be unlocked in existing programs, like writing documents in Word or juggling numbers in Excel spreadsheets.

The most common way to pay is by tokens.

The frequency with which employees are using AI models and the amount of tokens used in the queries is a new and increasing cost for businesses — which many of them will not have been paying for in even last financial year's report.

"That's the cost metric these businesses are grappling with," said Elastic's Jeremy Pell.

Different model, different prices

Another variable is that some of the cutting-edge or "frontier" AI models use more tokens.

That has led to companies using different models for different tasks, to avoid paying top dollar for tasks a more basic model can handle.

AI spending by major US firms is raising alam bells for some investors. (ABC News: John Gunn)

Some firms have also become more deliberate about the use of AI, sometimes capping the amount of tokens employees can use in a certain time frame.

"It's a matter of making sure that you are driving the business forward and these initiatives are driven and tied to the business's return on investment … It's working out which bits are useful."

'Scattergun' AI approach

AI expert Jon Whittle has warned the "scattergun" approach business has taken to using AI means the cost calculations are becoming more difficult, even as the price of tokens falls.

"I don't think companies are being very strategic right now about their AI use cases," he told The Business.

"They're just trying to find any place where they can use it, and that's why they're having costs increase so much."

Jon Whittle says the cost of tokens has come down dramatically. (ABC News: Simon Winter)

Professor Whittle, a former technical lead at NASA, director of CSIRO's AI capability and founder of Australia's National AI Centre, said a particular issue was that AI itself did a poor job at estimating the amount of tokens required.

Research has shown estimates can differ "by a factor of 30" running the same request on the same technology.

"It turns out that even the AI itself is not very good at predicting the cost that a particular AI task will take."

Speed increases as AI crunches data faster

From her Sydney office, Anna Volkova says the major advantage her firm currently gets from AI is speed.

Anna Volkova says decisions around AI are essentially choices to spend in the hope it will be worth it. (ABC News: Daniel Irvine)

By connecting artificial intelligence to the software tools the company already uses, staff have been able to do more, at greater pace.

But the bill is mounting.

The podcast that helps you unpack the latest business and finance news.

She said, eventually, the business would likely need to evaluate usage "by need", not "limiting responsible experimentation" but looking for uses that maximise what staff get out of using AI as a tool.

"I think the things that we're looking for in terms of really seeing the return for our people … is it making their work better, easier?" she asks.

It turns out the questions business leaders are asking about AI are very human.

Watch The Business, Mondays to Thursdays 8:44pm on ABC News Channel, after the Late News on ABC TV, and anytime on ABC iview

See more ABC coverage in your search results

Promotion

Practice Listening

Practice Listening