Agent是大模型的重要應用方向，而ReACT是學術界提出的重要方法，本文介紹ReACT論文，然後通過llama_index ReActAgent來分析ReACT的執行過程。
ReACT
《REACT: SYNERGIZING REASONING AND ACTING IN LANGUAGE MODELS》，由Shunyu Yao等人撰寫，發表在2023年的ICLR會議上。論文探討了如何在大型語言模型（LLMs）中結合推理（reasoning）和行動（acting）的能力，以提高模型在語言理解和交互式決策制定任務中的性能。
主要觀點：
- 大型語言模型（LLMs）在語言理解和交互式決策制定任務中表現出色，但它們的推理（例如鏈式思維提示）和行動（例如行動計劃生成）能力通常被分開研究。
- 作者提出了一種新的方法ReAct，它通過交錯生成推理軌跡和特定於任務的行動，使兩者之間產生更大的協同效應。推理軌跡幫助模型誘導、跟蹤和更新行動計劃，以及處理異常情況，而行動則允許它與外部資源（如知識庫或環境）接口並收集額外信息。
具體步驟：
ReAct（Reasoning and Acting）是一種結合了推理（reasoning）和行動（acting）的方法，旨在提高大型語言模型（LLMs）在解決複雜任務時的性能和可解釋性。ReAct的具體步驟如下：
- 定義任務和目標：首先，明確模型需要解決的任務和目標。這可能包括問題回答、事實驗證、文本遊戲或網頁導航等。
- 生成推理軌跡：模型生成一系列推理軌跡，這些軌跡是模型內部的思考過程，用於解決問題。這些推理軌跡可能包括分解任務目標、提取重要信息、執行常識推理、指導搜索重構、跟蹤進度和處理異常等。
- 執行行動：模型根據推理軌跡執行一系列行動。在ReAct中，行動是通過與外部環境（例如Wikipedia或其他知識庫）的交互來完成的。行動可以是搜索信息、選擇產品、選擇選項或購買等。
- 交替推理和行動：在解決任務的過程中，模型會交替進行推理和行動。這意味着模型在執行每個行動後可能會生成新的推理軌跡，然後再執行新的行動，以此往復。
- 更新上下文：每次行動後，模型會更新上下文信息，這包括之前的行動、觀察結果和新生成的推理軌跡。這種上下文更新幫助模型跟蹤任務的當前狀態，併爲未來的推理和行動提供信息。
- 生成任務解決軌跡：通過上述步驟，模型生成一個包含行動、觀察和推理軌跡的任務解決軌跡。這個軌跡不僅展示了模型如何解決問題，而且提供了模型決策的透明度，使得人類用戶可以理解和評估模型的行爲。
- 評估和調整：在實際應用中，模型生成的任務解決軌跡可能會被人類用戶評估和調整。用戶提供的反饋可以用來進一步指導模型的行爲，或者在模型自身無法正確解決問題時進行干預。
ReAct的核心在於通過交錯的推理和行動步驟，使模型能夠在執行任務時動態地進行推理和行動，從而提高任務解決的準確性和效率。這種方法特別適用於需要與外部環境交互並從中獲取信息以支持決策的任務。
以llama_index ReActAgent來看ReAct 執行過程
我們編寫一個簡單的ReActAgent程序，計算乘法，同樣的沒有openai的賬號，我們用google的Gemini。
from llama_index.core.agent import ReActAgent
from llama_index.core.tools import FunctionTool
from llama_index.llms.gemini import Gemini
# define sample Tool
def multiply(a: int, b: int) -> int:
"""Multiply two integers and returns the result integer"""
return a * b
multiply_tool = FunctionTool.from_defaults(fn=multiply)
# initialize llm
llm = Gemini(api_key="AI...", transport="rest")
# initialize ReAct agent
agent = ReActAgent.from_tools([multiply_tool], llm=llm, verbose=True)
resp = agent.query("計算85乘以9")
print(resp.response)
- 第一步，將tools放入prompts，要求LLM按照要求做Thought，然後出書符合要求的response：
[ChatMessage(role=<MessageRole.SYSTEM: 'system'>, content='\nYou are designed to help with a variety of tasks, from answering questions to providing summaries to other types of analyses.\n\n## Tools\nYou have access to a wide variety of tools. You are responsible for using\nthe tools in any sequence you deem appropriate to complete the task at hand.\nThis may require breaking the task into subtasks and using different tools\nto complete each subtask.\n\nYou have access to the following tools:\n> Tool Name: multiply\nTool Description: multiply(a: int, b: int) -> int\nMultiply two integers and returns the result integer\nTool Args: {"type": "object", "properties": {"a": {"title": "A", "type": "integer"}, "b": {"title": "B", "type": "integer"}}, "required": ["a", "b"]}\n\n\n## Output Format\nPlease answer in the same language as the question and use the following format:\n\n```\nThought: The current language of the user is: (user\'s language). I need to use a tool to help me answer the question.\nAction: tool name (one of multiply) if using a tool.\nAction Input: the input to the tool, in a JSON format representing the kwargs (e.g. {"input": "hello world", "num_beams": 5})\n```\n\nPlease ALWAYS start with a Thought.\n\nPlease use a valid JSON format for the Action Input. Do NOT do this {\'input\': \'hello world\', \'num_beams\': 5}.\n\nIf this format is used, the user will respond in the following format:\n\n```\nObservation: tool response\n```\n\nYou should keep repeating the above format until you have enough information\nto answer the question without using any more tools. At that point, you MUST respond\nin the one of the following two formats:\n\n```\nThought: I can answer without using any more tools. I\'ll use the user\'s language to answer\nAnswer: [your answer here (In the same language as the user\'s question)]\n```\n\n```\nThought: I cannot answer the question with the provided tools.\nAnswer: [your answer here (In the same language as the user\'s question)]\n```\n\n## Current Conversation\nBelow is the current conversation consisting of interleaving human and assistant messages.\n\n', additional_kwargs={}), ChatMessage(role=<MessageRole.USER: 'user'>, content='計算85乘以9', additional_kwargs={})]
上述數據沒有格式化，我們格式化後來看:
大體上就是讓LLM做COT思考，然後根據用戶當前的question，選擇合適的tool進行計算。
- LLM 思考後，回覆：
assistant: Thought: The current language of the user is: chinese. I need to use a tool to help me answer the question.
Action: multiply
Action Input: {"a": 85, "b": 9}
可以看到，給出了Action的tool，以及action的input
- llama_index 會對上面的輸出進行解析
process_actions，得到：
[ActionReasoningStep(thought='The current language of the user is: chinese. I need to use a tool to help me answer the question.', action='multiply', action_input={'a': 85, 'b': 9}), ObservationReasoningStep(observation='765')]
- 得到action step後，會進行執行，調用函數計算
cur_step_output = 765
- 然後，開始next_step:
[TaskStep(task_id='1d8db1d0-f5e3-4bfa-bcef-8fc405f958ca', step_id='91e9a97a-5ae1-4900-bf77-690865976902', input=None, step_state={'is_first': False}, next_steps={}, prev_steps={}, is_ready=True)]
- 來看step如何執行，同樣的拼接prompt，更新上下文，然後讓大模型COT:
[ChatMessage(role=<MessageRole.SYSTEM: 'system'>, content='\nYou are designed to help with a variety of tasks, from answering questions to providing summaries to other types of analyses.\n\n## Tools\nYou have access to a wide variety of tools. You are responsible for using\nthe tools in any sequence you deem appropriate to complete the task at hand.\nThis may require breaking the task into subtasks and using different tools\nto complete each subtask.\n\nYou have access to the following tools:\n> Tool Name: multiply\nTool Description: multiply(a: int, b: int) -> int\nMultiply two integers and returns the result integer\nTool Args: {"type": "object", "properties": {"a": {"title": "A", "type": "integer"}, "b": {"title": "B", "type": "integer"}}, "required": ["a", "b"]}\n\n\n## Output Format\nPlease answer in the same language as the question and use the following format:\n\n```\nThought: The current language of the user is: (user\'s language). I need to use a tool to help me answer the question.\nAction: tool name (one of multiply) if using a tool.\nAction Input: the input to the tool, in a JSON format representing the kwargs (e.g. {"input": "hello world", "num_beams": 5})\n```\n\nPlease ALWAYS start with a Thought.\n\nPlease use a valid JSON format for the Action Input. Do NOT do this {\'input\': \'hello world\', \'num_beams\': 5}.\n\nIf this format is used, the user will respond in the following format:\n\n```\nObservation: tool response\n```\n\nYou should keep repeating the above format until you have enough information\nto answer the question without using any more tools. At that point, you MUST respond\nin the one of the following two formats:\n\n```\nThought: I can answer without using any more tools. I\'ll use the user\'s language to answer\nAnswer: [your answer here (In the same language as the user\'s question)]\n```\n\n```\nThought: I cannot answer the question with the provided tools.\nAnswer: [your answer here (In the same language as the user\'s question)]\n```\n\n## Current Conversation\nBelow is the current conversation consisting of interleaving human and assistant messages.\n\n', additional_kwargs={}), ChatMessage(role=<MessageRole.USER: 'user'>, content='計算85乘以9', additional_kwargs={}), ChatMessage(role=<MessageRole.ASSISTANT: 'assistant'>, content="Thought: The current language of the user is: chinese. I need to use a tool to help me answer the question.\nAction: multiply\nAction Input: {'a': 85, 'b': 9}", additional_kwargs={}), ChatMessage(role=<MessageRole.USER: 'user'>, content='Observation: 765', additional_kwargs={})]
- 大模型回覆：
assistant: Thought: I can answer without using any more tools. I'll use the user's language to answer
Answer: 765
- 上述的回覆已經沒有新的action了，因此解析不出來action，next_steps就爲空了，計算結束，返回答案
- 這裏的計算比較簡單，一步action就搞定了，對於需要多步action的，會按照下面的流程圖，遞歸的執行，直到得到
Final Answer- 思考Thought
- 得到要執行的Action
- 執行Action，得到觀測結果 Observation
- 將上下文，連同Observation，讓大模型繼續思考Thought
- 直到沒有Action，給出最後的Filnal Answer
