Quick start: Extract information from an image using AI

1. Take a photo of the new car on a mobile device.
2. Upload the image to your AppSheet app.

Using AI, AppSheet extracts data from the photo, such as car make, model, color, and year.

In this Quick start, you'll build an automation that uses AI to update the inventory at a car dealership. Specifically, you'll create an AppSheet database and use it to create an app, then build an automation using AI to extract information from a car photo and categorize the car by body style.

To build an automation to extract information from a car photo and categorize the car by body style, do the following steps:

1. Create an AppSheet database
2. Create an app using the AppSheet database
3. Build an automation to extract information from a photo using AI
4. Customize the input form
5. Run the automation in the app preview

Step 1: Create an AppSheet database

First, you'll  create an AppSheet database and add the columns you need to store information about the car, such as make, model, year, description, and so on, and an Image column that will be used to upload a photo of the car.

To create an AppSheet database, do the following:

1. Sign in to  AppSheet .
   The My Apps page is displayed.
2. Select Create > Database > New database.
   A new AppSheet database is created and opened in the database editor
3. Click within the name, Untitled database, and change the name of the database to: Car inventory
   
   
4. Change the name of the Table 1 table:
   1. Position your cursor in the Table 1 tab and select More  > Table settings.
      
      
   2. Change the table name to:  Car inventory
      
      
   3. Leave all other fields set to the default values.
   4. Click Save.
5. Delete the dummy rows in the table, as follows:
   1. Click in column 1 and drag the cursor to highlight all rows in the table. 
      
      
   2. Right-click on the selected rows and select Delete rows.
      
      
      A confirmation dialog displays to confirm the delete operation.
   3. Click OKto confirm the operation.
6. Edit the columns in the table as indicated in the following table:
   Edit column

   Steps

   Change Titleto Make

   Do the following: 

   1. Position your cursor in the  Titlecolumn header and select More > Edit column

      

   2. Change the Namefield to Makeand click Save.

      

   Change Assigneeto Model

   Do the following:  

   1. Position your cursor in the Assignee column header and select More  > Edit column.
   2. Change the  Namefield to Model.

   3. Key the Typefield set to Name.
      This indicates that the content is a proper name. Alternatively, you could set this to Text.

   4. Click Save.

   Change Statusto Color

   Do the following:  

   1. In the Statuscolumn header, select More  > Edit column.
   2. Change the  Namefield to Color.
   3. Leave the Typefield set to Enum and the Item typefield set to Dropdown.
   4. Edit the three Enum options as follows:
      1. Double-click in the Not startedoption field and change the name to  White.
      2. Double-click in the In Progress option field and change the name to Green. Then, click the color setting to the left of the name and choose the green color block.
         
         
      3. Double-click in the Complete option field and change the name to Yellow. Then, click the color setting to the left of the name and choose the yellow color block.
   5. Add more Enum color options, as required, and an Other option to account for colors that aren't defined:
      1. Click Add option and enter Orange. 
         The color setting should be set to orange automatically. If it's not, then, click the color setting to the left of the name and choose the orange color block.
      2. Click Add option and enter Red. 
         The color setting should be set to red automatically. If it's not, then, click the color setting to the left of the name and choose the red color block.
      3. Click Add option and enter Blue. Then, click the color setting to the left of the name and choose the blue color block.
      4. Click Add option and enter Silver. Then, click the color setting to the left of the name and choose the silver (or grey) color block.
      5. Click Add option and enter Black. Then, click the color setting to the left of the name and set the color to black by clicking Set custom color, in the Hexfield entering  #000000, and clicking Select. 
         
         
      6. Click Add option, enter Other. Then, click the color setting to the left of the name and choose the white color block.
         Note:  "Other" will be used to catch colors that aren't defined in the list of options. You could then update the Enum options later to include any colors that were set to Other.
         The Color Enum settings appear as follows:
         
      7. Click Save.

   Change Dateto Year

   Do the following:  

   1. In the Datecolumn header, select More  > Edit column.
   2. Change the  Namefield to Year.
   3. Change the Typefield to Number.
   4. Click Save.
      Note: When prompted, click Yesto confirm the column type change. The information doesn't apply in this case since there's no data in the table.
7. Add a new Description column, as follows:
   1. Click + Add columnin the column header.
   2. Change the  Namefield to Description.
   3. Set the Typefield to Text .
   4. Click Save.
      
      
8. Add a new Image column, as follows:
   1. Click + Add columnin the column header.
   2. Change the Namefield to Car photo.
   3. Set the Typefield to Attachments > Image.
   4. Click Save.
      
      

The columns in the AppSheet database appear as shown:

Next, you'll create an app using the AppSheet database.

Step 2: Create an app using the AppSheet database

To create an app using the AppSheet database, do the following:

1. In the top-right corner of the database editor, click Apps.
   
   
   
   The Apps using Car inventory pane displays. No apps are listed.
2. Click New AppSheet app.
   The AppSheet app is created and opened in the app editor in a new tab.
3. In the app editor, in the left navigation, click  to display the Data pane (if it isn't already displayed).
   
   
   
   The  Car inventorytable is displayed. Notice that the source is the Car inventory table in the AppSheet database that you created in the previous step.
   
   
4. To change the name of the app, click in the app name field in the top left corner of the app editor and change the name to Car inventory.
   
   
5. To edit the Year column properties, click  Edit adjacent to the Yearcolumn.
   
   
6. Turn off  Show thousands separator -- since this value will depict the year the car was built which doesn't require a separator.
   
   
7. Click Done.

Next, you'll build an automation to extract information from a photo using AI.

Step 3: Build an automation to extract information from a photo using AI

To build an automation to extract information from a photo of a car using the Extract AI task , do the following:

1. In the app editor, go to  Automation .
   
   
2. Click +in the top header of the Bots pane.
   
   
3. In the Add a new bot dialog, click Create a new bot.
   
   
   
   A new empty bot is created and displayed in the center pane of the app editor.
4. Edit the name of the bot.
   1. Position your cursor over the New Bot in the left navigation pane.
   2. Select  More  > Rename.
      
      
   3. Rename the bot to Update car inventoryand press Enter.
5. In the new bot flow in the center pane, click Configure event.
   
   
6. For Event name, enter Update car inventory  and click Create a new event.

   

   The Settings pane opens in the right side of the app editor.

7. Configure the event by defining a condition to ensure that a car photo is uploaded before triggering the automation. as follows:
   1. Click in the Conditionfield.
      The Expression Assistant opens.
   2. In the text box enter: ISNOTBLANK([Car photo])
      
      
      
      This expression ensures that Car photo column is not blank and contains a photo of the car.
   3. Click Save.
   4. Leave all other settings set to the defaults.
      
      The Settings pane appears as follows:
      
      
8. In the bot flow in the center pane, click + Add a step.
   
   
9. For Step name, enter Extract from photo and click Create a new step.
   
   
   
   The Extract from photo task is added and the configuration settings are displayed in the right pane.
   
   You might need to select the task in the center pane to view the configuration settings.
10. Edit the task configuration settings in the right pane, as follows: 
    1. Set the task type set to AI task.
    2. Leave the AI task field set to Extract. This AI task type extracts information from a photo, PDF, or text.
    3. Leave the Input columnfield set to Car photo. This is the Image column that is used to upload a car photo.
    4. Under  Output:
       1. Select  Save to tableto save the extracted information to the AppSheet database.
       2. Notice that the first column is added and set to Make. 
          Leave this column selection as is.
       3. Click Addand select Model in the drop-down, if it's not already selected.
       4. Click Addand select Color in the drop-down, if it's not already selected.
       5. Click Addand select Year in the drop-down, if it's not already selected.
       6. Click Addand select Descriptionin the drop-down, if it's not already selected.
          
          The Output section appears as follows:
          
          
    5. In the Additional instructionsfield, enter:
       
       If not sure of Make, Model, or Year, make best guess
       If not sure of Color, set to "Other"
       For Description, add 2 to 3 sentences
       
       The Settings pane appears as follows:
       
       
11. Click  Saveto save the app.
    
    

Next, you'll customize the input form that is used by app users to upload a car photo.

Step 4: Customize the input form

When you created your app, AppSheet auto-generated two views--a form view to add a new car to the inventory and a detail view to view the car details. In this step, you'll edit the form view to customize the fields that display in the input form that is used by app users to upload a car photo. 

To customize the input form, do the following:

1. Go to  App  > Views.
   
   
2. In the left navigation pane, under the  System-Generatedviews click Car inventory_Formto open its configuration in the center pane. 
   
   
3. In the center pane, in the Column order field select Manual. 
   
   
4. In the column order list, select all columns except Car photo and click Remove.
   
   

Notice that the input form is updated in the app preview to only show the Car photo field.

Next, you'll run automation in the app preview.

Step 5: Run the automation in the app preview

To run the automation in the app preview, do the following:

1. Before you proceed, you'll need a photo of a car. Take a photo of your own car or download one from the internet.
2. In the app preview, Click in the Car photofield.
   
   
3. Navigate to the car photo in you local drive and click Open.
4. Click Save.
   
   
   
   The car photo image is uploaded. AppSheet syncs the app, as indicated by the sync notification:
   
   
   
   As AppSheet syncs the app, AI extracts information from the photo to populate the remaining fields. When the sync completes, the car is added to the inventory.
5. Click Car inventoryto navigate to the car inventory list.
6. After the sync completes, click the new car item in the list to view the details. 
   
   
   
   Notice that the Make, Model, Color, Year, and Description fields have been auto-populated .
   
   

Congratulations! You have built an automation using AI to extract information from an image. 

What's next? 

• Extend this automation to categorize the car by body style. See Quick start: Categorize information using AI .  
• Explore more Gemini-based  quick starts to learn about AI in automations.
• Explore other  quick starts to learn about other AppSheet features.