The Vision API can detect and extract information about entities in an image, across a broad group of categories.
Labels can identify general objects, locations, activities, animal species, products, and more. If you need targeted custom labels, Cloud AutoML Vision allows you to train a custom machine learning model to classify images.
Labels are returned in English only. The Cloud Translation API can translate English labels into any of a number of other languages .
For example, the image above may return the following list of labels:
| Description | Score |
|---|---|
| Street | 0.872 |
| Snapshot | 0.852 |
| Town | 0.848 |
| Night | 0.804 |
| Alley | 0.713 |
Label detection requests
Set up your Google Cloud project and authentication
Detect Labels in a local image
You can use the Vision API to perform feature detection on a local image file.
For REST requests, send the contents of the image file as a base64 encoded string in the body of your request.
For gcloud
and client library requests, specify the path to a local image in your
request.
REST
Before using any of the request data, make the following replacements:
- BASE64_ENCODED_IMAGE
: The base64
representation (ASCII string) of your binary image data. This string should look similar to the
following string:
-
/9j/4QAYRXhpZgAA...9tAVx/zDQDlGxn//2Q==
-
- RESULTS_INT
: (Optional) An integer value of results to
return. If you omit the
"maxResults"field and its value, the API returns the default value of 10 results. This field does not apply to the following feature types:TEXT_DETECTION,DOCUMENT_TEXT_DETECTION, orCROP_HINTS. - PROJECT_ID : Your Google Cloud project ID.
HTTP method and URL:
POST https://vision.googleapis.com/v1/images:annotate
Request JSON body:
{ "requests": [ { "image": { "content": " BASE64_ENCODED_IMAGE " }, "features": [ { "maxResults": RESULTS_INT , "type": "LABEL_DETECTION" } ] } ] }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "x-goog-user-project: PROJECT_ID " \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://vision.googleapis.com/v1/images:annotate"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = " PROJECT_ID " }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://vision.googleapis.com/v1/images:annotate" | Select-Object -Expand Content
If the request is successful, the server returns a 200 OK
HTTP status code and
the response in JSON format.
A LABEL_DETECTION
response includes the detected labels, their score,
topicality, and an opaque label ID, where:
-
mid- if present, contains a machine-generated identifier (MID) corresponding to the entity's Google Knowledge Graph entry. Note thatmidvalues remain unique across different languages, so you can use these values to tie entities together from different languages. To inspect MID values, refer to the Google Knowledge Graph API documentation. -
description- the label description. -
score- the confidence score, which ranges from 0 (no confidence) to 1 (very high confidence). -
topicality- The relevancy of the ICA (Image Content Annotation) label to the image. It measures how important/central a label is to the overall context of a page.
{
"responses": [
{
"labelAnnotations": [
{
"mid": "/m/01c8br",
"description": "Street",
"score": 0.87294734,
"topicality": 0.87294734
},
{
"mid": "/m/06pg22",
"description": "Snapshot",
"score": 0.8523099,
"topicality": 0.8523099
},
{
"mid": "/m/0dx1j",
"description": "Town",
"score": 0.8481104,
"topicality": 0.8481104
},
{
"mid": "/m/01d74z",
"description": "Night",
"score": 0.80408716,
"topicality": 0.80408716
},
{
"mid": "/m/01lwf0",
"description": "Alley",
"score": 0.7133322,
"topicality": 0.7133322
}
]
}
]
}
Go
Before trying this sample, follow the Go setup instructions in the Vision quickstart using client libraries . For more information, see the Vision Go API reference documentation .
To authenticate to Vision, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Java
Before trying this sample, follow the Java setup instructions in the Vision API Quickstart Using Client Libraries . For more information, see the Vision API Java reference documentation .
Node.js
Before trying this sample, follow the Node.js setup instructions in the Vision quickstart using client libraries . For more information, see the Vision Node.js API reference documentation .
To authenticate to Vision, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Python
Before trying this sample, follow the Python setup instructions in the Vision quickstart using client libraries . For more information, see the Vision Python API reference documentation .
To authenticate to Vision, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Additional languages
C#: Please follow the C# setup instructions on the client libraries page and then visit the Vision reference documentation for .NET.
PHP: Please follow the PHP setup instructions on the client libraries page and then visit the Vision reference documentation for PHP.
Ruby: Please follow the Ruby setup instructions on the client libraries page and then visit the Vision reference documentation for Ruby.
Detect Labels in a remote image
You can use the Vision API to perform feature detection on a remote image file that is located in Cloud Storage or on the Web. To send a remote file request, specify the file's Web URL or Cloud Storage URI in the request body.
REST
Before using any of the request data, make the following replacements:
- CLOUD_STORAGE_IMAGE_URI
: the path to a valid
image file in a Cloud Storage bucket. You must at least have read privileges to the file.
Example:
-
gs://cloud-samples-data/vision/label/setagaya.jpeg
-
- RESULTS_INT
: (Optional) An integer value of results to
return. If you omit the
"maxResults"field and its value, the API returns the default value of 10 results. This field does not apply to the following feature types:TEXT_DETECTION,DOCUMENT_TEXT_DETECTION, orCROP_HINTS. - PROJECT_ID : Your Google Cloud project ID.
HTTP method and URL:
POST https://vision.googleapis.com/v1/images:annotate
Request JSON body:
{ "requests": [ { "image": { "source": { "gcsImageUri": " CLOUD_STORAGE_IMAGE_URI " } }, "features": [ { "maxResults": RESULTS_INT , "type": "LABEL_DETECTION" }, ] } ] }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "x-goog-user-project: PROJECT_ID " \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://vision.googleapis.com/v1/images:annotate"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = " PROJECT_ID " }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://vision.googleapis.com/v1/images:annotate" | Select-Object -Expand Content
If the request is successful, the server returns a 200 OK
HTTP status code and
the response in JSON format.
A LABEL_DETECTION
response includes the detected labels, their score,
topicality, and an opaque label ID, where:
-
mid- if present, contains a machine-generated identifier (MID) corresponding to the entity's Google Knowledge Graph entry. Note thatmidvalues remain unique across different languages, so you can use these values to tie entities together from different languages. To inspect MID values, refer to the Google Knowledge Graph API documentation. -
description- the label description. -
score- the confidence score, which ranges from 0 (no confidence) to 1 (very high confidence). -
topicality- The relevancy of the Image Content Annotation (ICA) label to the image. It measures how important/central a label is to the overall context of a page.
{
"responses": [
{
"labelAnnotations": [
{
"mid": "/m/01c8br",
"description": "Street",
"score": 0.87294734,
"topicality": 0.87294734
},
{
"mid": "/m/06pg22",
"description": "Snapshot",
"score": 0.8523099,
"topicality": 0.8523099
},
{
"mid": "/m/0dx1j",
"description": "Town",
"score": 0.8481104,
"topicality": 0.8481104
},
{
"mid": "/m/01d74z",
"description": "Night",
"score": 0.80408716,
"topicality": 0.80408716
},
{
"mid": "/m/01lwf0",
"description": "Alley",
"score": 0.7133322,
"topicality": 0.7133322
}
]
}
]
}
Go
Before trying this sample, follow the Go setup instructions in the Vision quickstart using client libraries . For more information, see the Vision Go API reference documentation .
To authenticate to Vision, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Java
Before trying this sample, follow the Java setup instructions in the Vision API Quickstart Using Client Libraries . For more information, see the Vision API Java reference documentation .
Node.js
Before trying this sample, follow the Node.js setup instructions in the Vision quickstart using client libraries . For more information, see the Vision Node.js API reference documentation .
To authenticate to Vision, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Python
Before trying this sample, follow the Python setup instructions in the Vision quickstart using client libraries . For more information, see the Vision Python API reference documentation .
To authenticate to Vision, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
gcloud
To detect labels in an image, use the gcloud ml vision detect-labels
command as shown in the following example:
gcloud ml vision detect-labels gs://cloud-samples-data/vision/label/setagaya.jpeg
Additional languages
C#: Please follow the C# setup instructions on the client libraries page and then visit the Vision reference documentation for .NET.
PHP: Please follow the PHP setup instructions on the client libraries page and then visit the Vision reference documentation for PHP.
Ruby: Please follow the Ruby setup instructions on the client libraries page and then visit the Vision reference documentation for Ruby.
Try it
Try label detection below. You can use the
image specified already ( gs://cloud-samples-data/vision/label/setagaya.jpeg
)
or specify your own image in its place. Send the request by selecting Execute.
Request body:
{
"requests": [
{
"features": [
{
"maxResults": 5,
"type": "LABEL_DETECTION"
}
],
"image": {
"source": {
"imageUri": "gs://cloud-samples-data/vision/label/setagaya.jpeg"
}
}
}
]
}
