App Builder AI Agent
We have started developing an AI agent for App Builder and are making it available to early testers. The AI agent should still be considered alpha, and it may still change considerably. We look forward to receiving your feedback via our forum, community catchups, or onboarding calls.
How it works
The App Builder AI agent knows your app's parameters and can receive a context string from Grasshopper. You can generate this string parametrically in Grasshopper, and provide relevant context about your app to the agent. If you use the App Builder parameter reference component in Grasshopper to define tooltips for your parameters, these are available to the AI agent and provide further context to improve the agent’s understanding of your app.
You can download an example Grasshopper model from here.
The AI agent widget can be placed in your containers like any other widget. The user can enter chat messages, and optionally include a screenshot of the scene or another image as further context. The chat history is shown below.
Give it a try:
Bookshelf example
This example uses a context string and parameter tooltips, therefore providing a minimum level of context to the AI agent:
Your model
You can try the agent using any of your models using the following URL pattern. Please be aware that the only context available to the AI agent is the parameter names. As long as you don’t provide a context string and parameter tooltips from Grasshopper, the agent won’t perform well.
Implementation details
The source code of the AI agent is part of App Builder Open-Source SDK. You can find the source code of the agent widget here.
Limitations
API key visibility
Currently, the AI agent communicates directly from the browser with OpenAI’s API, which prevents the API key from being protected. You can provide the API key via a property in your theme file or as a URL parameter, as described below. We intend to extend the AI agent by a backend application that will act as a proxy and remove the need to expose the API key. We will likely offer the possibility of charging for LLM usage via our credit system.
Limited parameter types
The AI agent currently supports the following parameter types:
Number (float, integer, even integer, odd integer)
Boolean
String (see Direct Text Input)
String list (value lists)
Support for further parameter types like Color and File will be added.
Distribution
The AI agent is not part of App Builder's main distribution yet. Therefore, when opening apps from the platform, you must replace main with agent in the URL. Once we consider the agent to be in beta, it will be merged into the main distribution.
Configuration
You can configure the AI agent using properties in your theme file or as URL parameters. The following table shows the available properties and provides examples.
Property | Description |
|---|---|
| Your OpenAI API key. |
| The model to use for the chat completion API calls. |
| The maximum history of chat messages to use for chat completion API calls. |
| Set this to  |
Theme property only | Setting this property overrides the default system prompt. |
Theme property only | This overrides the context string provided from Grasshopper. Useful for debugging and experimenting, or if your Grasshopper model does not output a context string at all. |
Theme property only | Array of names of parameters that should be made available to the AI agent. This can be provided from Grasshopper as well. |
Theme property only | Array of names of parameters that should NOT be made available to the AI agent. This can be provided from Grasshopper as well. |
| Langfuse is an LLM engineering platform that allows to trace and evaluate LLM applications. Set these properties to enable tracing of the AI agent to Langfuse. |
Examples
A theme file for providing the OpenAI API key, specifying a different LLM, overriding the system prompt, and switching on debug mode:
{
"version": "1.0",
"themeOverrides": {
"components": {
"AppBuilderAgentWidgetComponent": {
"defaultProps": {
"openaiApiKey": "YOUR_OPENAI_API_KEY",
"model": "chatgpt-4o-latest",
"systemPrompt": "You are a helpful assistant that can modify parameters of a 3D configurator and answer questions about the 3D configurator based on the user's input and the context provided. You may answer questions by the user without changing parameters.",
"debug": true
}
}
}
}
}You can achieve a similar result (without overriding the system prompt) using this URL: