providerplaneai
    Preparing search index...

    Class OpenAIImageAnalysisCapabilityImpl

    OpenAIImageAnalysisCapabilityImpl: Implements image analysis for OpenAI using the Vision API and tool schema.

    Uses OpenAI Vision via the Responses API to analyze images and emit structured JSON results using a tool schema.

    IMPORTANT:

    • OpenAI provides semantic understanding only
    • No reliable bounding boxes, coordinates, or confidence scores
    • All outputs are normalized defensively before exposure

    Image analysis request type

    AIResponse containing normalized images

    Error if prompt is missing or analysis fails

    Implements

    Index

    Constructors

    constructor

    Properties

    OPENAI_IMAGE_ANALYSIS_SCHEMA OPENAI_IMAGE_ANALYSIS_TOOL

    Methods

    analyzeImage analyzeImageStream

    Constructors

    Properties

    OPENAI_IMAGE_ANALYSIS_SCHEMA: {
        properties: {
            description: { description: string; type: string };
            imageIndex: { description: string; type: string };
            objects: {
                description: string;
                items: {
                    properties: { label: { type: string } };
                    required: string[];
                    type: string;
                };
                type: string;
            };
            safety: {
                properties: {
                    categories: { additionalProperties: { type: string }; type: string };
                    flagged: { type: string };
                };
                required: string[];
                type: string;
            };
            tags: { items: { type: string }; type: string };
            text: {
                items: {
                    properties: { text: { type: string } };
                    required: string[];
                    type: string;
                };
                type: string;
            };
        };
        required: never[];
        type: string;
    } = ...

    OpenAI-compatible schema for semantic image analysis.

    NOTE: This schema intentionally avoids spatial guarantees (bounding boxes, confidence scores).

    OPENAI_IMAGE_ANALYSIS_TOOL: {
        description: string;
        name: string;
        parameters: {
            properties: {
                description: { description: string; type: string };
                imageIndex: { description: string; type: string };
                objects: {
                    description: string;
                    items: {
                        properties: { label: { type: string } };
                        required: string[];
                        type: string;
                    };
                    type: string;
                };
                safety: {
                    properties: {
                        categories: { additionalProperties: { type: string }; type: string };
                        flagged: { type: string };
                    };
                    required: string[];
                    type: string;
                };
                tags: { items: { type: string }; type: string };
                text: {
                    items: {
                        properties: { text: { type: string } };
                        required: string[];
                        type: string;
                    };
                    type: string;
                };
            };
            required: never[];
            type: string;
        };
        type: string;
    } = ...

    OpenAI Responses API tool definition.

    Purpose:

    • Instructs the model to emit a function_call named image_analysis
    • Ensures output matches the schema for safe parsing
    • Used in both non-streaming and streaming API calls

    Methods

    Settings

    Member Visibility

    On This Page

    Constructors
    Properties
    Methods