On October 10, 2024, our laboratory was honored to invite Professor Zhang Zhuosheng from the School of Electronic Information and Electrical Engineering of Shanghai Jiao Tong University to hold a high-level academic forum report. This report was presided over by Mr. Wang Hao, the tutor of the laboratory. The following is the poster of the academic forum report:

图1
Figure 1 Invitation poster

Title: Autonomous agents based on large models

Report Time:14:00-15:00 on October 10, 2024

Report Location:Room 304, Building 3, East District Graduate School, Shanghai University

Speaker:Zhang Zhuosheng

Meeting Minutes: Describe the workflow and principle of the application of large model autonomous agents to GUI screen operation, and realize operating system control, third-party software control, social media, online shopping, etc. through a series of action instructions and tool calls. This paper analyzes the problem of weak GUI environment perception of existing models. This paper analyzes the recent popular research on multimodal GUI agents. This paper analyzes the common security risks of large model agents, and lists various attack scenarios during the operation of agents.

图2
Figure 2 Recording of the reporting process