In spite of surging interest in AI agents across Southeast Asia, the promise of scalable growth is currently stalled by a critical hurdle: weak AI governance and risk management, identified as a key challenge in a June 2025 IDC Asia-Pacific InfoBrief sponsored by UiPath.
Unchecked agent autonomy, in particular, is proving to be a major liability across industries. Seemingly minor errors can cascade into major consequences: algorithmic mishaps in finance can wipe out billions, and missteps in healthcare can directly threaten patient safety.
Too often, enterprises treat governance as an afterthought. Companies only realise after deployment that it is not the large language models (LLMs) that fail, but the inadequate scaffolding around them that turns autonomy into a major enterprise risk.
Hence, error handling, context management, and audit trails can no longer be treated as peripheral concerns. Unlocking the real value of agentic ecosystems lies in enforcing control, transparency, and human oversight.
Designing agents that can fail safely
By nature, LLMs behave non-deterministically. The same prompt can yield a different and potentially biased output on every run. Integrating non-deterministic processes directly into core business operations creates systemic exposure in areas such as accountability and security. The only viable path forward is to design for safe failure instead of pure autonomy. Systems must be engineered to restrict agents from acting on ambiguous or unverified outputs, bounding non-deterministic behaviour within safe limits.
Critically, organisations should avoid embedding agents within traditional frameworks unless an extremely compelling business case overrides the risk. Agents introduce variables such as potential escalations and nuanced error states that require careful handling.
Organisations should also rethink how they design AI agents, particularly when they produce undesirable outputs. If an agent generates a poor result, simply retrying will not guarantee a correct or improved outcome. A second attempt is just as likely to fail, wasting processing cycles without addressing the underlying problem.
The focus should instead shift from retry mechanisms to fail-safe agent design. Robust checks must be built directly into the agent’s logic to validate and correct ambiguous outputs. Rather than giving agents free rein over deterministic tasks, organisations should bound risk by requiring the agent to act through trusted automations or verified APIs. This ensures the critical execution step is handled by a predictable process, preventing the agent from acting on unverified outputs.
Starting small and scaling smart
Reliable, scalable agentic systems cannot rely on a monolithic “do-everything” agent. This is a necessity driven by risk and complexity. A single, overly broad agent is inherently brittle: it requires a vast, general prompt that rapidly degrades accuracy and makes errors difficult to isolate.
Instead, multiple specialised, single-purpose agents can give organisations tighter control. This allows for controlled scaling, simplifies debugging by isolating failures to individual components, and enables the reuse of specialised capabilities across different enterprise functions.
Beyond design, organisations must also adopt phased deployment of AI agents to manage risk. They can begin with one or two medium-scale internal processes that pose limited risk from financial, cybersecurity, or data privacy standpoints. This initial phase focuses on establishing baseline performance and understanding real-world variability without exposing critical systems. Only after confirming success at this controlled level should the organisation proceed to gradual integration.
Controlled escalation allows teams to become familiar with the challenges involved in managing inter-agent dependencies, orchestration, and controlled failure across an expanding, autonomous ecosystem.
Humans in the loop: The key to controlled autonomy
Achieving the right balance between autonomy and control is an ongoing challenge, as the parameters may shift frequently. Organisations must calibrate agency carefully and grant greater autonomy only when agents demonstrate consistent accuracy and reliability.
The necessary course of action is to deliberately keep humans in the loop. Agents must be restricted from high-stakes actions such as approving complex financial transactions without human supervision. Escalations for human review also feed into agent memory, improving performance in future runs. This controlled-agency model ensures agentic workflows remain trustworthy, operating within clearly defined guardrails that preserve security, predictability, and performance.
Execution can be delegated to specialised agents, but governance requires a centralised control plane that provides visibility, auditing, and management of non-deterministic processes. This approach keeps agents reliable, accountable, and integrated as stable components of the digital workforce, with humans firmly in the driver’s seat.
By combining focused, single-purpose agents with deliberate human oversight and centralised governance, organisations can build scalable, dependable agentic systems, maximising automation potential while maintaining accountability at every step.
