With the emergence of AI agent services like Meta's 'Muse' and OpenAI's 'DALL-E', experts are calling for user protection measures to extend beyond AI responses to include actual actions and authority control.
On November 12, the Broadcast and Media Commission and the Korea Information Society Development Institute (KISDI) held the '2026 AI User Protection Conference' in Seoul. In his keynote address, Professor Lee Sang-geun of Korea University emphasized that "to protect users in the age of agents, safety measures must be integrated throughout the entire process of delegating and recovering agent service authority."
Professor Lee categorized the risks associated with AI agent services into three types: agents manipulated by external attacks, agents that overstep boundaries in achieving goals, and poorly delegated agents.
He noted that the risk of external attacks increases when sensitive data access, unreliable external inputs, and external communication capabilities converge, stating, "We must control to prevent these three elements from combining simultaneously."
To enhance user protection, he stressed the need for safety measures throughout the entire process from delegation to recovery.
When delegating authority, it is crucial for the agent to confirm who it is representing and what actions the user has permitted. Professor Lee suggested that indicators such as "the agent is the transaction entity" should be clearly displayed during payments.
He also recommended separating the AI model that monitors actions from the model that executes tasks, explaining that since monitoring AI can also make errors, structural safety measures like authority restrictions should be implemented.
He analyzed that human verification procedures should vary based on the risk of actions, as requiring approval for every action could reduce automation efficiency. He stated, "Only public data should be used, and human verification is necessary for actions that could cause significant harm, such as payments, sending external emails, and handling private financial and medical information."
Professor Lee emphasized the need to broaden the scope of cancellation and recovery, citing the increasing number of transactions executed without human approval due to multiple agents operating simultaneously.
In his keynote address, Park Tae-woong, a member of the National AI Strategy Committee's Public AX Division, identified six areas of user harm related to AI agent services: information harm, relational harm, transactional harm, data and security harm, delegation harm, and remedy harm. He explained that the protection scope should include not only misinformation and personal data leaks but also actions that exceed the delegation scope and the absence or delay of remedy procedures.
Park argued that laws and regulations for user protection should define the safety levels to be achieved rather than specifying particular technologies, holding businesses accountable for outcomes. He pointed out, "The structure should not allow for excuses such as 'I am not responsible because it was an AI' or 'I am not responsible because the user clicked to confirm.'"
He also stressed the need to establish rules to verify the identity, authority, and accountability of agents, ensuring that it is clear who operates and manages the agent and that actions exceeding the delegation scope can be immediately recognized and halted.
Park concluded, "When transactions occur that deviate from the user's explicit instructions, the agent provider and the payment business should bear primary responsibility."
* This article has been translated by AI.
Copyright ⓒ Aju Press All rights reserved.