Console
    Contact Us Start free

    Getting Started: Task Queues

    Learn how to use task queues and the App Engine Image API to resize images.

    Task queues execute code outside of direct user interaction, allowing tasks to happen in the background. This guide uses a Task Queue to perform tasks after adding an image to Cloud Storage . The tasks to be performed in the Task Queue are:

    1. Retrieve the image file that just got uploaded to Cloud Storage.
    2. Resize it to a thumbnail image using the Image API .
    3. Store the resulting thumbnail in Cloud Storage.

    The App Engine Java 8 runtime also supports Java's native image manipulation classes such as AWT and Java2D .

    Before you begin

    1. Configure your development environment and create your App Engine project .

    2. This guide uses the Apache Commons IOUtils library . To include the IOUtils library to your App Engine project:

      Add to your pom.xml :

       < dependency 
>  
< groupId>commons 
 - 
 io 
< / 
 groupId 
>  
< artifactId>commons 
 - 
 io 
< / 
 artifactId 
>  
< version>2 
 .5 
< / 
 version 
>
< / 
 dependency 
>

    Importing libraries

    The sample code provided with this guide uses the following imports:

      import 
  
 com.google.appengine.api.images. Image 
 
 ; 
 import 
  
 com.google.appengine.api.images. ImagesService 
 
 ; 
 import 
  
 com.google.appengine.api.images. ImagesServiceFactory 
 
 ; 
 import 
  
 com.google.appengine.api.images. Transform 
 
 ; 
 import 
  
 org.apache.commons.io.IOUtils 
 ;

    Creating a task queue

    While App Engine provides a default task queue, you can create different task queues for different types of work. For example, you can create one task queue to resize images and another to update your app's database.

    To add queues, create the queue.xml file in your App Engine project's WEB-INF directory. A taskqueue must specify a name and an execution rate:

     < ? 
 xml 
  
 version 
 = 
 "1.0" 
  
 encoding 
 = 
 "UTF-8" 
 ? 
>  
< queue 
 - 
 entries 
>  
< queue 
>  
< name>resize 
 - 
 image 
< / 
 name 
>  
< rate>60 
 / 
 h 
< / 
 rate 
>  
< / 
 queue 
>  
< / 
 queue 
 - 
 entries 
>

    This example queue, named resize-image , defines a rate of execution of 60 times per hour, or once a minute. To see the full list queue options see the queue.xml reference .

    A task queue has two components, the task requester and a task handler. The requestor adds a task to the queue and sends it to the task handler.

    Adding tasks to a queue

    To add a task to a queue:

    1. Create a task queue object using QueueFactory.getQueue() , making sure you specify the queue name defined in the queue.xml :

        Queue 
  
 imageResizeQueue 
 ; 
  
 // Taskqueue queue 
 @Override 
 public 
  
 void 
  
 init 
 () 
  
 throws 
  
 ServletException 
  
 { 
  
 // Setup Cloud Storage service 
  
 gcsService 
  
 = 
  
 GcsServiceFactory 
 . 
 createGcsService 
 ( 
  
 new 
  
 RetryParams 
 . 
 Builder 
 () 
  
 . 
 initialRetryDelayMillis 
 ( 
 10 
 ) 
  
 . 
 retryMaxAttempts 
 ( 
 10 
 ) 
  
 . 
 totalRetryPeriodMillis 
 ( 
 15000 
 ) 
  
 . 
 build 
 ()); 
  
 // Initialize the queue object with the specific queue 
  
 imageResizeQueue 
  
 = 
  
 QueueFactory 
 . 
 getQueue 
 ( 
 [ 
 QUEUE 
 - 
 NAME 
 ] 
 ); 
  
 // Cloud SQL connection setup 
  
 try 
  
 { 
  
 final 
  
 String 
  
 url 
  
 = 
  
 System 
 . 
 getProperty 
 ( 
 "cloudsql" 
 ); 
  
 // Cloud SQL server URI 
  
 try 
  
 { 
  
 conn 
  
 = 
  
 DriverManager 
 . 
 getConnection 
 ( 
 url 
 ); 
  
 // Connect to the database 
  
 Statement 
  
 createTable 
 ; 
  
 // SQL statement 
  
 // Batch SQL table creation commands 
  
 createTable 
 . 
 addBatch 
 ( 
 createContentTableSql 
 ); 
  
 createTable 
 . 
 addBatch 
 ( 
 createUserTableSql 
 ); 
  
 createTable 
 . 
 addBatch 
 ( 
 createImageTableSql 
 ); 
  
 createTable 
 . 
 addBatch 
 ( 
 createBlogPostImageTableSql 
 ); 
  
 conn 
 . 
 createTable 
 . 
 executeBatch 
 (); 
  
 // Execute batch 
  
 } 
  
 catch 
  
 ( 
 SQLException 
  
 e 
 ) 
  
 { 
  
 throw 
  
 new 
  
 ServletException 
 ( 
 "Unable to connect to Cloud SQL" 
 , 
  
 e 
 ); 
  
 } 
  
 } 
  
 finally 
  
 { 
  
 // Nothing really to do here. 
  
 } 
 }

    2. Add tasks to Queue object. As shown in the code sample, imageResizeQueue.add() adds a task to the imageResizeQueue object:

        try 
  
 { 
  
 // Add a queued task to create a thumbnail of the uploaded image 
  
 imageResizeQueue 
 . 
 add 
 ( 
  
 TaskOptions 
 . 
 Builder 
 . 
 withUrl 
 ( 
 "/tasks/imageresize" 
 ). 
 param 
 ( 
 "filename" 
 , 
  
 filename 
 )); 
 }

      Specify the URI of the task handler using TaskOptions.Builder.withUrl() , along with and any parameters sent to the handler.

      In this example, the URI is /tasks/imageresize and the parameter is a variable called filename that contains the filename of the image to be processed.

    Creating a task handler

    Once you have added a task to the queue, the task handler mapped to the URI /tasks/imageresize will run. A task handler is a Java Servlet that attempts to perform the task until it is successful.

    In this example, the task handler does three tasks:

    • Retrieve the image specified by the caller from Cloud Storage.

    • Transform the image using the App Engine Image API, in this sample, to a thumbnail image.

    • Store the transformed image (thumbnail) in Cloud Storage.

    To create the task handler:

    1. Add an annotation that maps the handler to the URI /tasks/imageresize :

         
 @WebServlet 
 ( 
 name 
  
 = 
  
 "imageResize" 
 , 
  
 description 
  
 = 
  
 "Task queue handler" 
 , 
  
 urlPatterns 
  
 = 
  
 "/tasks/imageresize" 
 ) 
  
 public 
  
 class 
 imageResize 
  
 extends 
  
 HttpServlet 
  
 { 
  
 // Task handler functionality 
  
 }

    2. Set up a connection to Cloud Storage as documented in the Using Cloud Storage guide and retrieve the image from Cloud Storage:

         
 public 
  
 void 
  
 init 
 () 
  
 throws 
  
 ServletException 
  
 { 
  
 // initiate GcsService 
  
 GcsService 
  
 gcsService 
  
 = 
  
 GcsServiceFactory 
 . 
 createGcsService 
 ( 
  
 new 
  
 RetryParams 
 . 
 Builder 
 () 
  
 . 
 initialRetryDelayMillis 
 ( 
 10 
 ) 
  
 . 
 retryMaxAttempts 
 ( 
 10 
 ) 
  
 . 
 totalRetryPeriodMillis 
 ( 
 15000 
 ) 
  
 . 
 build 
 ()); 
 }

    3. Handle the incoming Task Queue request, using the supplied file name to retrieve the image from Cloud Storage:

        protected 
  
 void 
  
 doPost 
 ( 
 HttpServletRequest 
  
 req 
 , 
  
 HttpServletResponse 
  
 resp 
 ) 
  
 throws 
  
 ServletException 
 , 
  
 IOException 
  
 { 
  
 String 
  
 filename 
  
 = 
  
 req 
 . 
 getParameter 
 ( 
 "filename" 
 ); 
  
 // Get the filename passed from the task requestor 
  
 GcsFilename 
  
 gcsFile 
  
 = 
  
 new 
  
 GcsFilename 
 ( 
 bucket 
 , 
  
 filename 
 ); 
  
 // Create a valid Cloud Storage filename 
  
 GcsInputChannel 
  
 readChannel 
  
 = 
  
 gcsService 
 . 
 openPrefetchingReadChannel 
 ( 
 gcsFile 
 , 
  
 0 
 , 
  
 BUFFER_SIZE 
 ); 
  
 // Read the file from Cloud Storage

    4. Use the ImagesService object to do the image resizing:

        // Get an instance of the ImagesService we can use to transform images. 
 ImagesService 
  
 imagesService 
  
 = 
  
 ImagesServiceFactory 
 . 
 getImagesService 
 (); 
 // Make an image directly from a byte array, and transform it. 
 Image 
  
 image 
  
 = 
  
 ImagesServiceFactory 
 . 
 makeImage 
 ( 
 IOUtils 
 . 
 toByteArray 
 ( 
 Channels 
 . 
 newInputStream 
 ( 
 readChannel 
 ))); 
 Transform 
  
 resize 
  
 = 
  
 ImagesServiceFactory 
 . 
 makeResize 
 ( 
 100 
 , 
  
 50 
 ); 
  
 // resize image to 100x50 
 Image 
  
 resizedImage 
  
 = 
  
 imagesService 
 . 
 applyTransform 
 ( 
 resize 
 , 
  
 image 
 ); 
 // Write the transformed image back to a Cloud Storage object. 
 gcsService 
 . 
 createOrReplace 
 ( 
  
 new 
  
 GcsFilename 
 ( 
 bucket 
 , 
  
 "thumbnail_" 
  
 + 
  
 filename 
 ), 
  
 new 
  
 GcsFileOptions 
 . 
 Builder 
 (). 
 acl 
 ( 
 "public-read" 
 ). 
 build 
 (), 
  
 ByteBuffer 
 . 
 wrap 
 ( 
 resizedImage 
 . 
 getImageData 
 ()));

      The above snippet uses the Image API makeResize() method to resize the image to a thumbnail. To do this, it reads the image from Cloud Storage into an InputChannel and convert it to a ByteArray using IOUtils.toByteArray() .

      After applying the transformation, the new image has the string thumbnail_ appended to its filename, permission set to be publicly readable and written to Cloud Storage.

    Securing task handler URLs

    You should secure tasks that perform sensitive operations such as modifying data so external users cannot call it directly. You can do this by restricting task access to App Engine administrators, which prevents users from accessing task URLs . Note that this restriction does not apply to task requests coming from your App Engine app.

    In the current example, the task handlers have URLs in the /tasks/ folder. To restrict access to the /tasks/ folder to App Engine administrators, add the following to the project's web.xml .

     < security 
 - 
 constraint 
>  
< web 
 - 
 resource 
 - 
 collection 
>  
< web 
 - 
 resource 
 - 
 name>tasks 
< / 
 web 
 - 
 resource 
 - 
 name 
>  
< url 
 - 
 pattern 
> / 
 tasks 
 /*</ 
 url 
 - 
 pattern 
>  
< / 
 web 
 - 
 resource 
 - 
 collection 
>  
< auth 
 - 
 constraint 
>  
< role 
 - 
 name>admin 
< / 
 role 
 - 
 name 
>  
< / 
 auth 
 - 
 constraint 
>
< / 
 security 
 - 
 constraint 
>

    Removing a single task from a queue

    To remove a single task from a queue, use deleteTask() :

      private 
  
 void 
  
 removeTask 
 ( 
 Queue 
  
 queue 
 , 
  
 String 
  
 taskName 
 ) 
  
 { 
  
 queue 
 . 
 deleteTask 
 ( 
 taskName 
 ); 
  
 // remove a task from a queue 
 }

    Removing all tasks from a queue

    To remove all tasks from a queue, use purge() . The purge can take up to a minute to remove all tasks in the queue.

      private 
  
 void 
  
 purgeQueue 
 ( 
 Queue 
  
 queue 
 ) 
  
 { 
  
 queue 
 . 
 purge 
 (); 
  
 // remove all tasks from a queue 
 }

    It can take a minute to remove all tasks from a queue, so you should wait a few seconds before adding new tasks to the queue.

    Deleting a task queue

    To delete a task queue, remove the entry from the project's queue.xml file and redeploy.

    Deploying to App Engine

    You can deploy your app to App Engine using Maven.

    Go to the root directory of your project and type:

    mvn package appengine:deploy -Dapp.deploy.projectId= PROJECT_ID

    Replace PROJECT_ID with the ID of your Google Cloud project. If your pom.xml file already specifies your project ID , you don't need to include the -Dapp.deploy.projectId property in the command you run.

    After Maven deploys your app, open a web browser tab automatically at your new app by typing:

      gcloud 
  
 app 
  
 browse

    What's next

    This guide shows how to use a task queue to create an image thumbnail and store it in Cloud Storage. It can be used with other storage services such as Cloud Datastore or Cloud SQL .

    Previous
    Using Cloud SQL
    Create a Mobile Website
    View Site in Mobile | Classic
    Share by: