Acceptable Collateral

By Sebastian Scandura

Over the past several weeks, three of the most important regulators with a hand in Australian AI policy have released new guidance.

On 7 April 2026, the United States National Institute of Standards and Technology released a concept note for a new AI Risk Management Framework profile on Trustworthy AI in Critical Infrastructure, extending the NIST AI RMF deeper into sector-specific territory.

On 9 April 2026, the Australian Signals Directorate published formal guidance on the cyber security implications of frontier AI models, naming Anthropic's Claude Mythos Preview and OpenAI's GPT-5.5 as the inflection point that prompted it. ASD updated that guidance again on 30 April 2026.

On 30 April 2026, the Australian Prudential Regulation Authority released the final targeted amendments to CPS 230 and an industry-wide letter on AI use. APRA warned regulated entities that their current governance, risk management, assurance, and operational resilience practices are not sufficiently keeping pace with the scale, speed, and complexity of AI adoption.

I have read all of them carefully, because risk advisory is what I do. There is something missing across the three documents, and across the broader Australian conversation that has built up around them.

·  ·  ·

The consumer is not in the room.

Read CPS 230 and you will find a document about how regulated entities should manage the AI vendors they use. Read the ASD guidance and you will find a document about how organisations should defend their own technology stacks against AI-accelerated attack. Read the NIST profile and you will find a document about how critical infrastructure operators should govern the AI systems they deploy. Each is competent. Each is current. Each is also written from inside the organisation looking out.

The people whose data is the actual asset under management appear in these documents, when they appear at all, as a category. As a persona. As a data field. They do not appear as humans with a stake in what happens next.

This is not new. It is also not benign.

·  ·  ·

The visible edge of the current AI rush is the layoff list. In February 2026, WiseTech Global announced it was cutting around 2,000 jobs, almost a third of its 7,000-person global workforce, as part of an AI integration programme. The chief executive's framing was uncompromising: "The era of manually writing code as the core act of engineering is over." Two weeks later, on 11 March 2026, Atlassian announced 1,600 redundancies, explicitly to redirect capital toward AI investment. Around the same time, Block, the parent of Afterpay, cut more than 4,000 roles to "move faster with smaller, highly talented teams." Sydney is now reportedly the third-ranked global city for tech job losses, behind only San Francisco and Seattle.

These are the people the news stories cover. They are also not the people I am writing about today.

The less visible edge of the AI rush is the consumer. The customer who never sees the inside of a board paper, never reads a CPS 230 amendment, and never receives notification when the credit decision, the insurance claim outcome, the support call routing, the fraud-prevention trigger, or the loan default flag was assessed by a model whose decision boundary was last calibrated by a vendor whose foundation model was last updated by a third party whose accountability ends at a service agreement.

That consumer's data is the asset that makes the model work. That consumer's reasonable expectation of privacy and proper handling is the implicit warranty that allows the model to be deployed at all. That consumer is the human being whom every one of these governance frameworks is supposed to ultimately protect.

That consumer, in the current configuration, has no seat at the table.

·  ·  ·

Inside boardrooms, the conversation has moved on. Over the last two years, AI has stopped being a discipline and started being a credential. Executives use the language of agentic AI in meetings the way they once used the language of cloud transformation. Board members ask about an organisation's AI strategy the way they used to ask about digital strategy: not because they want a substantive answer, but because the question is now expected. Procurement tenders, including in regulated sectors, have started carrying explicit sections asking respondents to demonstrate "innovation," which in practice means demonstrating AI features. The word "innovation" in a 2026 request for tender is, in many cases, a euphemism for a checkbox compliance test against an AI shopping list.

None of this is in itself wrong. Boards should be asking about AI. Procurement should be testing for capability. Executives should be reading the regulations. But the centre of gravity of the conversation has moved inside the firewall, and the conversation about what is happening to the people on the other side of the firewall has not kept up.

·  ·  ·

Frameworks exist, and they are worth naming. The OECD AI Principles, adopted in 2019 and refreshed in 2024, are technology-neutral guardrails for AI use across member states. ISO/IEC 42001:2023 provides the first international standard for AI management systems. The European Union's AI Act is now phasing into binding obligations. The NIST AI Risk Management Framework, with its generative AI profile, its agentic AI profile, and the new critical infrastructure profile in concept, is the most operationally detailed of the lot. Australia's Voluntary AI Safety Standard, published in 2024, provides domestic guardrails. The Council of Europe AI Convention adds a treaty-level layer. The Australian Signals Directorate's own Ethical AI in the ASD framework governs how ASD itself uses AI.

These are not bad documents. They are useful documents. But, with one exception, they share a common centre of gravity. They look inward. They tell organisations how to protect themselves, how to govern their own AI use, how to manage their own risk surface. They do not, as their first principle, ask what is happening to the people the AI is acting upon.

The exception is the UNESCO Recommendation on the Ethics of Artificial Intelligence, adopted by 193 member states in November 2021. The Recommendation states explicitly that "the protection of human rights and dignity is the cornerstone." Its eleven policy action areas begin with mandatory ethical impact assessments for high-risk AI systems and run through data governance, environmental impact, gender equality, and human oversight. Of the major international AI ethics instruments, the UNESCO Recommendation is the closest to keeping a human, rather than a persona or a data field, at the centre of the page.

That is not a high bar. It should be the floor.

·  ·  ·

The Australian Human Rights Commission has been making a related point repeatedly through 2025 and into 2026: that voluntary action by industry, however good in faith, cannot replace enforceable rules that ensure consistency, accountability, and protection. The Commission's most recent statement noted that "the pace of reform, particularly in areas like privacy law, has been cripplingly slow. By the time consensus is reached, the technology will have already moved on."

The Productivity Commission has, in the meantime, advised the Government to pause work on mandatory guardrails for high-risk AI until gaps in existing law are fully understood. The Government's own final report on AI and the Australian Consumer Law, released late in 2025, concluded that the framework is fit for purpose. The submissions from civil society, including Good Ancestors' argument that AI developers should be required to internalise the risks of harm from their systems rather than shifting accountability down to Australian businesses deploying them, were noted and largely set aside.

These are policy choices. They are made by intelligent people in good faith. They are also, taken together, choices that leave the consumer holding the residual risk that the regulatory architecture has chosen not to absorb.

·  ·  ·

What would a risk professional do differently?

Five things, none of them radical.

One. Every organisation deploying AI on data that includes personally identifiable information should be required to publish, in a place a consumer can actually find, what the AI is being asked to do, what data it sees, what decisions it informs, and what redress is available if it gets the decision wrong. Not in a privacy policy that nobody reads. In a plain-English notice a customer can read in the time it takes to make a coffee.

Two. The "human in the loop" claim should be auditable. Not asserted in a vendor brochure. Tested by a regulator. With consequences for organisations that claim it and cannot demonstrate it.

Three. Ethical impact assessments should follow the UNESCO model and be mandatory for high-risk AI systems, not optional. The pushback that mandatory assessments stifle innovation is the same pushback that was made against mandatory food labelling in the 1980s. We adapted.

Four. The procurement tender sections currently asking vendors to demonstrate "innovation" should be required to also ask vendors to demonstrate consumer-facing controls. If the AI is good enough to deploy, it is good enough to be transparent about.

Five. Boards should be receiving, in their quarterly AI papers, not just an inventory of where AI is being used and what risks it introduces to the organisation, but a parallel inventory of what risks it introduces to the people who interact with the organisation. The two are not the same risk. They should not appear on the same page as if they were.

·  ·  ·

The ceiling, not the floor.

These five points are the floor. They are what enforcement should look like if the regulatory architecture were doing the work it needs to do.

The ceiling is different. The ceiling is what an organisation chooses to do when no regulator is making it.

There is a useful precedent. In high-stakes safety environments such as aviation, nuclear operations, and intensive-care medicine, any member of the team has the authority to call "stop." A junior cabin attendant can halt a flight if they see something wrong on the apron. An ICU nurse can pause a procedure if a patient's vitals shift. A reactor operator can scram the core. The authority is structural. It does not depend on rank or persuasion or whether the room agrees on the day. It exists because the cost of being wrong is too high to leave to politeness.

AI deployment in regulated environments is now operating at that scale of consequence. The decisions are smaller, individually, than a missed runway clearance. The volume is much larger.

The organisations that will stand apart over the next five years will be the ones that build a parallel structure inside their own governance. Call the role what you like. End-User Advocate. Consumer Risk Officer. AI Ethics Director. The label is less important than the mandate. The mandate is this: a board-level or board-adjacent position with explicit authority to represent the people on the other side of the firewall, and explicit power to call stop on AI deployments that put those people at unacceptable risk.

·  ·  ·

Not advisory. Not consultative. With teeth.

This is not regulatory burden. It is competitive advantage. The organisation that builds this role honestly will tell its customers a true story: that someone in the building, at the highest level, has the job of asking "what if this is wrong for the people we serve?" and the power to act on the answer. That is not gratuitous. That is the kind of ethic that builds goodwill. The customer who knows their concerns have a real seat at a real table is the customer who renews the contract, forgives the small mistakes, and tells their friends. Goodwill compounds. Distrust does too.

The cost of building this role is far smaller than the cost of the first major AI failure that becomes a news story. The organisations that move first will write the playbook. The ones that wait will read it.

·  ·  ·

Risk theatre is the appearance of risk management without the substance of risk management. There is a version of the current AI governance conversation that is sliding in that direction. Frameworks that look impressive. Letters that read seriously. Boards that ask the right questions and receive the right-looking answers. Regulators that publish the right-looking guidance.

The consumer, in the meantime, remains a data field.

·  ·  ·

The technology will not pause to be governed. The regulators will not legislate fast enough. The organisations that lead will lead because they decided to, not because they were told to.

If you sit on a board, or run an enterprise, or carry the data of people who have trusted you with it, the question is simple. Who in your building has the authority to say "stop, and listen to me" on behalf of the people you serve? Not who is consulted. Not who is informed. Who can stop the deployment.

If the answer is "no one," that is the first thing to fix. Before the next AI vendor onboarding. Before the next board paper. Before the next quarter.

Sincero Risk works privately with executives and boards on the substance of risk and AI governance, including the design of advocacy and stop-authority roles. If this piece has put a question in your mind that you do not yet know how to answer, that is a good place to start a conversation.

sincerorisk.com

Sebastian Scandura is the founder of Sincero Risk, an independent advisory practice based in Canberra. The practice offers risk advisory, growth strategy, and project assurance services to executives, boards, and founders. Views in this piece are his own.