Register Custom Device Actions

    • Custom device actions allow devices to perform special abilities not covered by standard traits.

    • Defining a custom device action requires a pattern to match user queries, an associated custom action, text spoken back to the user, and a command name sent to the device with parameters.

    • Custom device actions are defined within an action package file, such as actions.json , and are fulfilled locally.

    • The gactions command-line tool is used to deploy and test your action package.

    • To handle a custom action on the device, you need to modify the sample code to include a handler that processes the command and parameters.

    Your device may have special abilities not covered by the current set of traits -- such as a "blink my light" trait for a device that can blink its lights. You can define custom actions for your device that specify the commands sent to your device to trigger special abilities.

    To define a custom device action, you need the following:

    • A pattern to match against the user query
    • A custom device action to associate with a matched query
    • Text spoken back to the user if the device supports the action
    • A command name that is sent back to your device, along with any parameters

    You create the custom device action by putting this information into an action package . Action packages define the format for the Assistant responses. Unlike the Actions SDK, custom device actions are fulfilled locally; you do not specify an endpoint to process requests and provide responses. Custom device actions are not conversational in nature.

    Create an Action Package

    Using the following as an example, create a file (such as actions.json ) that defines a test command: blinking an LED. Copy the file from the sample code you downloaded in a previous step :

    cd assistant-sdk-python/google-assistant-sdk/googlesamples/assistant/library/
     
    cp ~/assistant-sdk-python/google-assistant-sdk/actions.json .

    Example

     { 
 "manifest" 
 : 
 { 
 "displayName" 
 : 
 "Blinky light" 
 , 
 "invocationName" 
 : 
 "Blinky light" 
 , 
 "category" 
 : 
 "PRODUCTIVITY" 
 }, 
 "actions" 
 : 
 [ 
 { 
 "name" 
 : 
 "com.example.actions.BlinkLight" 
 , 
 "availability" 
 : 
 { 
 "deviceClasses" 
 : 
 [ 
 { 
 "assistantSdkDevice" 
 : 
 {} 
 } 
 ] 
 }, 
 "intent" 
 : 
 { 
 "name" 
 : 
 "com.example.intents.BlinkLight" 
 , 
 "parameters" 
 : 
 [ 
 { 
 "name" 
 : 
 "number" 
 , 
 "type" 
 : 
 "SchemaOrg_Number" 
 }, 
 { 
 "name" 
 : 
 "speed" 
 , 
 "type" 
 : 
 "Speed" 
 } 
 ], 
 "trigger" 
 : 
 { 
 "queryPatterns" 
 : 
 [ 
 "blink ($Speed:speed)? $SchemaOrg_Number:number times" 
 , 
 "blink $SchemaOrg_Number:number times ($Speed:speed)?" 
 ] 
 } 
 }, 
 "fulfillment" 
 : 
 { 
 "staticFulfillment" 
 : 
 { 
 "templatedResponse" 
 : 
 { 
 "items" 
 : 
 [ 
 { 
 "simpleResponse" 
 : 
 { 
 "textToSpeech" 
 : 
 "Blinking $number times" 
 } 
 }, 
 { 
 "deviceExecution" 
 : 
 { 
 "command" 
 : 
 "com.example.commands.BlinkLight" 
 , 
 "params" 
 : 
 { 
 "speed" 
 : 
 "$speed" 
 , 
 "number" 
 : 
 "$number" 
 } 
 } 
 } 
 ] 
 } 
 } 
 } 
 } 
 ], 
 "types" 
 : 
 [ 
 { 
 "name" 
 : 
 "$Speed" 
 , 
 "entities" 
 : 
 [ 
 { 
 "key" 
 : 
 "SLOWLY" 
 , 
 "synonyms" 
 : 
 [ 
 "slowly" 
 , 
 "slow" 
 ] 
 }, 
 { 
 "key" 
 : 
 "NORMALLY" 
 , 
 "synonyms" 
 : 
 [ 
 "normally" 
 , 
 "regular" 
 ] 
 }, 
 { 
 "key" 
 : 
 "QUICKLY" 
 , 
 "synonyms" 
 : 
 [ 
 "quickly" 
 , 
 "fast" 
 , 
 "quick" 
 ] 
 } 
 ] 
 } 
 ] 
 } 
 
      actions 
 . 
 json

    The previous example uses the following information to define the custom device action:

    • A pattern to match against the user query ( blink N times )
    • The custom device action to associate with a matched query ( com.example.actions.BlinkLight ) for organizational purposes
    • Text spoken back to the user if the device supports the action ( Blinking N times )
    • A command name ( com.example.commands.BlinkLight ) that is sent back to the device, along with any parameters (a number and possibly a description of the speed)

    To define the query pattern, note the following:

    • You can use schema.org-defined types in the query pattern.
    • The types [...] array defines the list of custom types (for example, $Speed ).
    • You can use custom types in the query pattern. The user can speak any of the synonyms in your custom type to match the query pattern.
    • When a synonym does match, the type instance ( speed ) would return the normalized key ( SLOWLY ). There can be multiple entities in case, for example, there are different lights that support different speeds of blinking.
    • Parts of the request TTS pattern can be optional. For example, use ($Speed:speed)? in the query pattern to make this part optional.
    • $ type .raw (for example, $speed.raw ) in the response TTS is replaced by the word(s) the user actually spoke.

    For descriptions of many of these fields, see the ActionPackage documentation.

    Deploy the Action Package

    After you build your custom device action in an Action package, you can make the Action package accessible to the Assistant.

    While you can perform the steps in this section on your device, it may be easier to do them on your development system. The following commands do not require a virtual environment to run.

    1. Download the gactions command line tool .

    2. Remove any existing credentials from the same directory as the gactions tool.

      rm creds.data

    3. Save your Action package to Google by using the gactions CLI. Replace project_id with your Actions Console project ID .

      ./gactions update --action_package actions.json --project project_id

    4. The first time you run this command you will be given a URL and be asked to sign in. Copy the URL and paste it into a browser (this can be done on any system). The page will ask you to sign in to your Google account. Sign into the Google account that created the project in a previous step .

    5. After you approve the permission request from the API, a code will appear in your browser, such as "4/XXXX". Copy and paste this code into the terminal:

      Enter the authorization code:

      If authorization was successful, you will see a response similar to the following:

        Your 
 app 
 for 
 the 
 Assistant 
 for 
 project 
 my 
 - 
 devices 
 - 
 project 
 was 
 successfully 
 updated 
 with 
 your 
 actions 
 .

    6. Deploy your Action package into test mode by using the gactions CLI. You must have saved your Action package to Google at least once before running this command. Test mode enables the Action package on your user account only.

      ./gactions test --action_package actions.json --project project_id

      Currently, you cannot test the project using the Actions simulator.

    7. To update the Action package, use the gactions update command.

    8. (Optional) You can create localized Action packages to support many different languages and locales at the same time in a single project.

    Modify the sample

    Do the steps in this section on the device.

    nano hotword.py

    Add a handler for your custom action. You can use the handler below if you want to use the sample Action Package above.

     import 
  
 time 
 # Add this to the imports near the top of the file 
 ... 
 if 
 event 
 . 
 type 
 == 
 EventType 
 . 
 ON_DEVICE_ACTION 
 : 
 for 
 command 
 , 
 params 
 in 
 event 
 . 
 actions 
 : 
 print 
 ( 
 'Do command' 
 , 
 command 
 , 
 'with params' 
 , 
 str 
 ( 
 params 
 )) 
 # Add the following lines after the existing line above: 
  if 
 command 
 == 
 "com.example.commands.BlinkLight" 
 : 
 number 
 = 
 int 
 ( 
 params 
 [ 
 'number' 
 ] 
 ) 
 for 
 i 
 in 
 range 
 ( 
 int 
 ( 
 number 
 )): 
 print 
 ( 
 'Device is blinking.' 
 ) 
 # GPIO.output(25, 1) 
 time 
 . 
 sleep 
 ( 
 1 
 ) 
 # GPIO.output(25, 0) 
 time 
 . 
 sleep 
 ( 
 1 
 )

    Run the sample

    Run the modified source code.

    python hotword.py --device-model-id my-model

    Try a query. For the example above, try the following:

    Ok Google, blink 5 times.

    Note that the query needs to match the query pattern in the Action Package.

    Design a Mobile Site
    View Site in Mobile | Classic
    Share by: