I figured out why my Spring AI MCP server failed to support image injection.
By default, the tool call result is of type text
, not image
.
@JsonInclude(JsonInclude.Include.NON_ABSENT)
public record TextContent( // @formatter:off
@JsonProperty("audience") List<Role> audience,
@JsonProperty("priority") Double priority,
@JsonProperty("type") String type,
@JsonProperty("text") String text) implements Content { // @formatter:on
public TextContent {
type = "text";
}
public String type() {
return type;
}
public TextContent(String content) {
this(null, null, "text", content);
}
}
public static McpServerFeatures.SyncToolRegistration toSyncToolRegistration(ToolCallback toolCallback) {
var tool = new McpSchema.Tool(toolCallback.getToolDefinition().name(),
toolCallback.getToolDefinition().description(),
toolCallback.getToolDefinition().inputSchema());
return new McpServerFeatures.SyncToolRegistration(tool, request -> {
try {
String callResult = toolCallback.call(ModelOptionsUtils.toJsonString(request));
return new McpSchema.CallToolResult(List.of(new McpSchema.TextContent(callResult)), false);
} catch (Exception e) {
return new McpSchema.CallToolResult(List.of(new McpSchema.TextContent(e.getMessage())), true);
}
});
}
To solve this, I created a custom toSyncToolRegistration
method, using the returnDirect
field to decide whether to return the tool result as raw JSON.
Apparently, Spring AI mcp sdk doesn’t seem to use this field in the @Tool
annotation.
public static McpServerFeatures.SyncToolRegistration toSyncToolRegistration(ToolCallback toolCallback) {
var tool = new McpSchema.Tool(toolCallback.getToolDefinition().name(),
toolCallback.getToolDefinition().description(),
toolCallback.getToolDefinition().inputSchema());
return new McpServerFeatures.SyncToolRegistration(tool, request -> {
try {
String callResult = toolCallback.call(ModelOptionsUtils.toJsonString(request));
if (toolCallback.getToolMetadata().returnDirect()) {
return ModelOptionsUtils.jsonToObject(callResult, McpSchema.CallToolResult.class);
} else {
return new McpSchema.CallToolResult(List.of(new McpSchema.TextContent(callResult)), false);
}
} catch (Exception e) {
return new McpSchema.CallToolResult(List.of(new McpSchema.TextContent(e.getMessage())), true);
}
});
}
Here is my image tool implementation:
@Tool(name = "image_tool", description = IMAGE_TOOL_DESC, returnDirect = true)
public McpSchema.CallToolResult imageTool(@ToolParam(description = TOOL_PARAM_DESC) String... urls) {
if (urls == null || urls.length == 0) {
return new McpSchema.CallToolResult(Collections.emptyList(), true);
}
List<McpSchema.Content> list = new ArrayList<>();
for (String url : urls) {
McpSchema.ImageContent image = new McpSchema.ImageContent(
List.of(), 1.0D, "image", urlToBASE64(url), "image/jpeg"
);
list.add(image);
}
return new McpSchema.CallToolResult(list, false);
}
With this setup, I’m now able to return images as part of tool responses by setting returnDirect = true on the tool definition.