Is there an existing issue for this?
Describe the problem
Use Case / Background
I am currently developing an Android 16 app (com.example.googleio2026) that integrates with the new AppFunctions SDK to log and track daily health data (hydration tracking).
When a user asks the assistant for a data summary (e.g., "Show me my last 7 days progress"), returning text alone isn't optimal for the UX. I have implemented a system-level @AppFunction that successfully draws a bar chart onto a headless canvas, saves it to a local cache file, and passes the asset back via an android.net.Uri wrapped inside an @AppFunctionSerializable class.
The Problem
When testing this behavior using the official Sample/Test Agent App (the chat companion UI), the app successfully executes the function but only prints out the raw string data or data class layout in the chat bubble. It currently does not resolve or display the android.net.Uri image asset inline.
While the data contract verifies perfectly via raw adb shell cmd app_function outputs, it is difficult to evaluate the visual conversational flow of agentic UI experiences locally.
Proposed Enhancement
It would be incredibly helpful if the Sample Agent App's chat interface could natively support rendering image assets returned by functions.
Specifically:
- Detect if an element inside an
@AppFunctionSerializable return payload is an android.net.Uri.
- Check if the mime-type/extension is an image asset (e.g., PNG, JPEG).
- Utilize an image loading library (like Coil or Glide) within the Compose chat bubble structure to display the image inline next to the text summary, managing the necessary temporary URI read permission grants (
AppFunctionUriGrant).
Code Example of the Payload
@AppFunctionSerializable
data class WeeklyProgressResponse(
val textSummary: String,
val barChartUri: Uri // <-- Requesting the test app UI to parse and display this asset
)
class HydrationStatsFunctions : AppFunctionTarget {
@AppFunction
suspend fun getPastWeekProgress(context: AppFunctionContext): WeeklyProgressResponse {
// ... business logic generating local chart cache file
return WeeklyProgressResponse(
textSummary = "You hit your hydration target 5 out of the last 7 days!",
barChartUri = fileProviderUri
)
}
}
### Describe the solution
I would like the Test Agent App (the ChatApp testing interface) to natively support and render image content URIs that are returned within `@AppFunctionSerializable` objects.
Specifically, the requested solution should:
1. Detect if any parameter inside the returned `@AppFunctionSerializable` data payload contains an `android.net.Uri`.
2. Inspect the content type/extension. If it evaluates to a supported image asset (such as image/png or image/jpeg), the ChatApp's Compose layout engine should resolve the asset.
3. Automatically handle the platform-level URI security boundary requirements (such as checking/requesting an `AppFunctionUriGrant`).
4. Display the image inline directly next to or below the regular `textSummary` data block inside the user's conversation bubble stream using an image-loading framework (e.g., Coil or Glide).
### Additional context
Thank you for providing these tools to the community! This enhancement would make local agent testing significantly closer to what the real-world consumer Gemini assistant app will experience.
### Code of Conduct
- [x] I agree to follow this project's Code of Conduct
Is there an existing issue for this?
Describe the problem
Use Case / Background
I am currently developing an Android 16 app (
com.example.googleio2026) that integrates with the new AppFunctions SDK to log and track daily health data (hydration tracking).When a user asks the assistant for a data summary (e.g., "Show me my last 7 days progress"), returning text alone isn't optimal for the UX. I have implemented a system-level
@AppFunctionthat successfully draws a bar chart onto a headless canvas, saves it to a local cache file, and passes the asset back via anandroid.net.Uriwrapped inside an@AppFunctionSerializableclass.The Problem
When testing this behavior using the official Sample/Test Agent App (the chat companion UI), the app successfully executes the function but only prints out the raw string data or data class layout in the chat bubble. It currently does not resolve or display the
android.net.Uriimage asset inline.While the data contract verifies perfectly via raw
adb shell cmd app_functionoutputs, it is difficult to evaluate the visual conversational flow of agentic UI experiences locally.Proposed Enhancement
It would be incredibly helpful if the Sample Agent App's chat interface could natively support rendering image assets returned by functions.
Specifically:
@AppFunctionSerializablereturn payload is anandroid.net.Uri.AppFunctionUriGrant).Code Example of the Payload