# Sending Images to a Model API — Multimodal AI

Source: https://www.skillbyai.com/en/multimodal-ai/mm-api-image-input

> Build a chat request that mixes text and an image, using a URL or a base64 data URL.

## Messages with multiple content parts

Hosted multimodal APIs extend the familiar chat format: instead of a single string, a message's **content** becomes a list of **parts**, each with a type such as text or image. An image part can usually reference a public **URL** the provider fetches, or carry the bytes inline as **base64**, often wrapped in a data URL such as `data:image/png;base64,...`. The exact field names differ between providers, so check the documentation of the one you use; the OpenAI-style `image_url` part shown here is also accepted by many OpenAI-compatible servers, including local ones. Some APIs offer a detail or resolution setting that trades accuracy for token cost. Base64 inflates the payload by about a third, and providers enforce limits on image size, count and format, so resize images on your side before sending. For repeated use of the same large file, some providers offer a file upload endpoint so you send the bytes once.

## A multimodal request end to end

Your code prepares the image, wraps it with the text in a message, sends it to an API or local server, and parses the reply.

![Four connected stages: prepare image, build message parts, model server, parsed response.](assets/figures/multimodal-ai/section-7-map.svg) — Figure 7.1: prepare image (resize, encode) -> build message with text + image parts -> hosted API or local server -> parse and validate the response.

## Building an image + text message

The image is encoded in memory as base64 and placed next to the question. Printing the part types confirms the structure before any network call.

```python
import base64, io, json
from PIL import Image

img = Image.new("RGB", (64, 64), "white")      # stand-in for a real photo
buf = io.BytesIO()
img.save(buf, format="PNG")
b64 = base64.b64encode(buf.getvalue()).decode("ascii")

message = {
    "role": "user",
    "content": [
        {"type": "text", "text": "What is written on this receipt?"},
        {"type": "image_url",
         "image_url": {"url": f"data:image/png;base64,{b64}"}},
    ],
}
print([part["type"] for part in message["content"]])
print(message["content"][1]["image_url"]["url"][:22] + "...")
print(len(json.dumps(message)) > len(buf.getvalue()))   # base64 grows the payload
```

Output:

```
['text', 'image_url']
data:image/png;base64,...
True
```

## Ask for structure, then validate it

For extraction tasks (bills, ID cards, forms) ask the model for JSON with named fields and validate the result with a schema library such as Pydantic. Treat a parse failure as a retry or a human-review case, never as data.

**Quiz:** Why should you resize a 12-megapixel phone photo before sending it to a vision API?

- [ ] APIs only accept square images
- [ ] Base64 cannot encode large files
- [ ] Resizing adds EXIF metadata the model needs
- [x] Smaller images upload faster, fit size limits and usually cost fewer image tokens

*Answer:* Smaller images upload faster, fit size limits and usually cost fewer image tokens. Large images waste bandwidth, can exceed limits, and often cost more tokens without improving answers for typical questions.