Image Recognition

Current models recognize images: they read text in a photo, identify objects, animals, and plants, and interpret charts. Attach a picture to the conversation and you no longer have to describe what is in it.

Try it right now. Photograph anything on your desk, send it into your chat, and ask what it is. Not every service can do this, and one try shows whether yours can.

What people send as a photo

A photo works anywhere you would otherwise be retyping someone else's text or hunting for words to describe something.

A nameplate on equipment

Photograph the plate on an air conditioner and ask for the make, model number, capacity, year, and country of manufacture. If internet search is on in the service, it will find the manual by the model number as well.

A receipt or an invoice

A photo turns into a table of expenses. Send it and ask for the line items with amounts.

A paper report with a table

You no longer have to type it up. The model returns the contents in whatever form you ask for.

A part with no name

A part whose name you do not know can be shown. If the model recognizes it, you get a name, and with a name it is easier to find a replacement.

It does not recognize every part. Models do worse with things that rarely appeared in their training data. A part from rare or highly specialized equipment will more likely go unrecognized or be named wrong.

It is still worth trying. Check the name the model offered against a catalog or with a seller before you order the part.

Product packaging in a store

Photograph the ingredients or the specifications and ask what matters for your case.

Symptoms and over-the-counter medicine

Describe your symptoms to the model and ask which over-the-counter products are used for them and how they differ. That is a place to start when the problem does not seem serious to you.

A model's advice does not replace a professional, and neither does its answer to a legal question. In its usage policies, OpenAI places medical and legal advice tailored to a specific person among the things that require a licensed professional.

A photo adds to the description of symptoms. Take a picture of the package or of the pharmacy shelf. Ask what the active ingredient is, whether it fits your symptoms, and what it should not be combined with.

With that answer in hand, you walk up to the counter with your questions ready.

Behind the counter stands a pharmacist, and the questions are for them. The package itself asks for this: the American Drug Facts label has a Warnings section that spells out when the choice goes to a professional.

The FDA puts it plainly: if you read the label and still have questions, ask a doctor, a pharmacist, or another health care provider.

An error window on the screen

You do not have to retell it in your own words. You show the window itself, and the analysis runs off what is written in it.

A plant, an insect, a bird

A model will identify a plant, a leaf, an insect, or a bird from a photo. That comes in handy at work and outside it: for an agronomist, a buyer, or someone in a garden.

A traffic situation

A photo of the damage or of where the cars ended up helps describe the incident when you have to file a report.

Where the answer cannot be taken at face value

A venomous snake, a mushroom, an unfamiliar berry: here a mistake costs your health. The model will name a species and may get it wrong, so the answer is a lead for further searching, not grounds for picking the thing up.

The same goes for medicine. Models are not meant to read images such as CT or MRI scans, and you do not get a diagnosis from them off a photo.

Numbers, names, and dates taken from a picture get checked by eye. Anthropic explicitly asks users to verify what is read from images wherever the cost of an error is high.

How to take the photo

Shoot straight on and get closer. A blurry, rotated, or too-small frame is read with errors.

Say what to do with the photo. An attached image does not set a task: ask for the line items, the species, a comparison with another photo.

Which photos go in which account

The split between accounts from the chapter "Personal and Work Access" applies as well. Send pictures from your own life to your personal account: a plant from the garden, a store receipt, a part from a home appliance. Work documents, screens of work systems, and company equipment do not go into a personal account.

In a corporate account the rule is reversed: work photos belong there. Where your company draws the line is its decision. Ask your manager or whoever handles security what may be photographed and sent.