Add Recommendations to your app with TensorFlow Lite and Firebase - Android Codelab

1. Overview

Welcome to the Recommendations with TensorFlow Lite and Firebase codelab. In this codelab you'll learn how to use TensorFlow Lite and Firebase to deploy a recommendation model to your app. This codelab is based on this TensorFlow Lite example .

Recommendations allow apps to use machine learning to intelligently serve the most relevant content for each user. They take into account past user behavior to suggest app's content the user might like to interact with in the future by using a model trained on the aggregate behavior of a large number of other users.

This tutorial shows how to obtain data from your app's users with Firebase Analytics, build a machine learning model for recommendations from that data, and then use that model in an Android app to run inference and obtain recommendations. In particular, our recommendations will suggest which movies a user would most likely watch given the list of movies the user has liked previously.

What you'll learn

• Integrate Firebase Analytics into an android app to collect user behavior data
• Export that data into Google Big Query
• Pre-process the data and train a TF Lite recommendations model
• Deploy the TF Lite model to Firebase ML and access it from your app
• Run on device inference using the model to suggest recommendations to users

What you'll need

• Latest Android Studio version.
• Sample code.
• A test device with Android 7+ and Google Play services 9.8 or later, or an Emulator with Google Play services 9.8 or later
• If using a device, a connection cable.

2. Get the sample code

Clone the GitHub repository from the command line.

$ git clone https://github.com/FirebaseExtended/codelab-contentrecommendation-android.git

3. Import the starter app

From Android Studio, select the codelab-recommendations-android directory () from the sample code download ( File> Open> .../codelab-recommendations-android/start).

You should now have the start project open in Android Studio.

4. Create Firebase console project

Create a new project

1. Go to the Firebase console .
2. Select Add project(or Create a projectif it's the first one).
3. Select or enter a Project name and click Continue.
4. Ensure that "Enable Google Analytics for this project" is enabled.
5. Follow the remaining setup steps in the Firebase console, then click Create project (or Add Firebase, if you're using an existing Google project).

5. Add Firebase

1. From the overview screen of your new project, click the Android icon to launch the setup workflow.
2. Enter the codelab's package name: com.google.firebase.codelabs.recommendations
3. Select Register app.

Add google-services.json file to your app

After adding the package name and selecting Register, click Download google-services.jsonto obtain your Firebase Android config file then copy the google-services.json file into the app directory in your project. After the file is downloaded you can Skipthe next steps shown in the console (they've already been done for you in the build-android-start project).

Add google-services plugin to your app

The google-services plugin uses the google-services.json file to configure your application to use Firebase. The following lines should already be added to the build.gradle.kts files in the project (check to confirm):

app/build.grade.kts

 plugins {
    id("com.google.gms.google-services")
} 

build.grade.kts

 plugins {
    id("com.google.gms.google-services") version "4.3.15" apply false
} 

Sync your project with gradle files

To be sure that all dependencies are available to your app, you should sync your project with gradle files at this point. Select File > Sync Project with Gradle Filesfrom the Android Studio toolbar.

6. Run the starter app

Now that you have imported the project into Android Studio and configured the google-services plugin with your JSON file, you are ready to run the app for the first time. Connect your Android device, and click Run()in the Android Studio toolbar.

The app should launch on your device. At this point, you can see a functioning application that shows a tab with a list of movies, a Liked movies tab, and a Recommendations tab. You can click on a movie in the list of movies to add it to your liked list. After completing the remaining steps of the codelab, we will be able to generate movie recommendations in the Recommendations tab.

7. Add Firebase Analytics to the app

In this step, you will add Firebase Analytics to the app to log user behavior data (in this case, which movies a user likes). This data will be used in aggregate in future steps to train the recommendations model.

Add Firebase Bill of Materials and Analytics dependency

The following dependencies are necessary to add Firebase Analytics to your app. They should already be included in the app/build.gradle.kts file (verify).

app/build.grade.kts

  implementation 
 ( 
 platform 
 ( 
 "com.google.firebase:firebase-bom:32.0.0" 
 )) 
 implementation 
 ( 
 "com.google.firebase:firebase-analytics-ktx" 
 ) 
 

Set up Firebase Analytics in the app

The LikedMoviesViewModelcontains functions to store the movies the user likes. Every time the user likes a new movie, we want to also send off an analytics log event to record that like.

Add the onMovieLiked function with the code below to register an analytics event when the user clicks like on a movie.

LikedMoviesViewModel.kt

  import 
  
 com.google.firebase.analytics.FirebaseAnalytics 
 import 
  
 com.google.firebase.analytics.ktx.analytics 
 import 
  
 com.google.firebase.analytics.ktx.logEvent 
 import 
  
 com.google.firebase.ktx.Firebase 
 class 
  
 LikedMoviesViewModel 
 internal 
 constructor 
 ( 
 application 
 : 
 Application 
 ) 
 : 
 AndroidViewModel 
 ( 
 application 
 ) 
 { 
 ... 
 fun 
 onMovieLiked 
 ( 
 movie 
 : 
 Movie 
 ) 
 { 
 movies 
 . 
 setLike 
 ( 
 movie 
 , 
 true 
 ) 
 logAnalyticsEvent 
 ( 
 movie 
 . 
 id 
 . 
 toString 
 ()) 
 } 
 } 
 

Add the following field and function to log an Analytics event when a movie is added to the user's Liked list.

LikedMoviesViewModel.kt

  import 
  
 com.google.firebase.analytics.FirebaseAnalytics 
 import 
  
 com.google.firebase.analytics.ktx.analytics 
 import 
  
 com.google.firebase.analytics.ktx.logEvent 
 import 
  
 com.google.firebase.ktx.Firebase 
 class 
  
 LikedMoviesViewModel 
 internal 
 constructor 
 ( 
 application 
 : 
 Application 
 ) 
 : 
 AndroidViewModel 
 ( 
 application 
 ) 
 { 
 ... 
 private 
 val 
 firebaseAnalytics 
 = 
 Firebase 
 . 
 analytics 
 ... 
 /** 
 * 
 Logs 
 an 
 event 
 in 
 Firebase 
 Analytics 
 that 
 is 
 used 
 in 
 aggregate 
 to 
 train 
 the 
 recommendations 
 * 
 model 
 . 
 */ 
 private 
 fun 
 logAnalyticsEvent 
 ( 
 id 
 : 
 String 
 ) 
 { 
 firebaseAnalytics 
 . 
 logEvent 
 ( 
 FirebaseAnalytics 
 . 
 Event 
 . 
 SELECT_ITEM 
 ) 
 { 
 param 
 ( 
 FirebaseAnalytics 
 . 
 Param 
 . 
 ITEM_ID 
 , 
 id 
 ) 
 } 
 } 
 

8. Test your Analytics integration

In this step, we will generate Analytics events in the app and verify that they are being sent to the Firebase Console.

Enable Analytics Debug Logging

Firebase Analytics is designed to maximize user battery life and will batch events on device and only send them to Firebase occasionally. For debugging purposes, we can disable this behavior to see events as they are logged in real time by running the following command in the shell.

Terminal

 adb shell setprop debug.firebase.analytics.app com.google.firebase.codelabs.recommendations 

Verify Analytics events are generated

1. In Android studio, open the Logcat window to examine logging from your app.
2. Set the Logcat filter to the string "Logging event".
3. Verify that "select_item" Analytics events are emitted every time you like a movie in the app.

At this point, you have successfully integrated Firebase Analytics into your app. As users use your app and like movies, their likes will be logged in aggregate. We will use this aggregate data in the rest of this codelab to train our recommendations model. The following is an optional step to see the same Analytics events you saw in Logcat also stream in to the Firebase console. Feel free to skip to the next page.

Optional: Confirm Analytics events in Firebase Console

1. Go to the Firebase console .
2. Select DebugViewunder Analytics
3. In Android Studio, select Runto launch the app and add some movies to your Liked list.
4. In the Firebase console's DebugView, verify that these events are being logged as you add movies in the app.

9. Export Analytics data to Big Query

Big Query is a Google Cloud product that allows you to examine and process large amounts of data. In this step, you will connect your Firebase Console project to Big Query so that the Analytics data generated by your app is automatically exported to Big Query.

Enable Big Query export

1. Go to the Firebase console .
2. Select the Settings gear icon next to Project Overview, and then select Project settings
3. Select the Integrationstab.
4. Select Link(or Manage) inside the BigQueryblock.
5. Select Nextin the About Linking Firebase to BigQuerystep.
6. Under the Configure integrationsection, click the switch to enable sending Google Analytics data and select Link to BigQuery.

You have now enabled your Firebase console project to automatically send Firebase Analytics event data to Big Query. This happens automatically without any further interaction, however, the first export that creates the analytics dataset in BigQuery may not happen for 24 hours. After the dataset is created, Firebase continually exports new Analytics events to Big Query into the intraday table, and groups events from past days in the events table.

Training a recommendations model requires a lot of data. Since we don't already have an app generating large amounts of data, in the next step we will import a sample dataset into BigQuery to use for the rest of this tutorial.

10. Use BigQuery to obtain model training data

Now that we have connected our Firebase Console to export to BigQuery, our app analytics event data will automatically show up in the BigQuery console after some time. To get some initial data for the purposes of this tutorial, in this step we will import an existing sample dataset into your BigQuery console to use to train our recommendations model.

Import sample dataset into BigQuery

1. Go to the BigQuery dashboard in the Google cloud console.
2. Select your project name in the menu.
3. Select your project name in the bottom of the BigQuery left navigation to see details.
4. Select Create datasetto open the dataset creation panel.
5. Enter ‘firebase_recommendations_dataset' for the Dataset IDand select Create dataset.
6. The new dataset will show up in the left menu under the project name. Click it.
7. Select Create tableto open the table creation panel.
8. For Create table fromselect ‘Google Cloud Storage'.
9. In the Select file from GCS bucketfield, enter ‘gs://firebase-recommendations/recommendations-test/formatted_data_filtered.txt'.
10. Select ‘JSONL' in the File formatdrop down.
11. Enter ‘recommendations_table' for the Table name.
12. Check the box under Schema > Auto detect > Schema and input parameters
13. Select Create table

Explore sample dataset

At this point, you can optionally explore the schema and preview this dataset.

1. Select firebase-recommendations-datasetin the left menu to expand the tables it contains.
2. Select the recommendations-tabletable to view the table schema.
3. Select Previewto see the actual Analytics event data this table contains.

Create service account credentials

Now, we will create service account credentials in our Google Cloud console project that we can use in the Colab environment in the following step to access and load our BigQuery data.

1. Make sure that billing is enabled for your Google Cloud project.
2. Enable the BigQuery and BigQuery Storage API APIs. < click here >
3. Go to the Create Service Account Key page .
4. From the Service accountlist, select New service account.
5. In the Service account namefield, enter a name.
6. From the Rolelist, select Project> Owner.
7. Click Create. A JSON file that contains your key downloads to your computer.

In the next step, we will use Google Colab to preprocess this data and train our recommendations model.

11. Preprocess data and train recommendations model

In this step, we will use a Colab notebook to perform the following steps:

1. import the BigQuery data into the Colab notebook
2. preprocess the data to prepare it for model training
3. train the recommendations model on the analytics data
4. export the model as a TF lite model
5. deploy the model to the Firebase Console so we can use it in our app

Before we launch the Colab training notebook, we will first enable the Firebase Model Management API so Colab can deploy the trained model to our Firebase console.

Enable Firebase Model Management API

Create a bucket to store your ML models

In your Firebase Console, go to Storage and click Get started.

Follow the dialogue to get your bucket set up.

Enable Firebase ML API

Go to Firebase ML API page on Google Cloud Console and click Enable.

Use Colab notebook to train and deploy the model

Open the colab notebook using the following link and complete the steps within. After finishing the steps in the Colab notebook, you will have a TF lite model file deployed to the Firebase console that we can sync down to our app.

Open in Colab

12. Download the model in your app

In this step, we'll modify our app to download the model we just trained from Firebase Machine Learning.

Add Firebase ML dependency

The following dependency is needed in order to use Firebase Machine Learning models in your app. It should already be added (verify).

app/build.grade.kts

  implementation 
 ( 
 "com.google.firebase:firebase-ml-modeldownloader:24.1.2" 
 ) 
 

Download the model with Firebase Model Manager API

Copy the code below into RecommendationClient.ktto set up the conditions under which model download occurs and create a download task to sync the remote model to our app.

RecommendationClient.kt

   
 private 
  
 fun 
  
 downloadModel 
 ( 
 modelName 
 : 
  
 String 
 ) 
  
 { 
  
 val 
  
 conditions 
  
 = 
  
 CustomModelDownloadConditions 
 . 
 Builder 
 () 
  
 . 
 requireWifi 
 () 
  
 . 
 build 
 () 
  
 FirebaseModelDownloader 
 . 
 getInstance 
 () 
  
 . 
 getModel 
 ( 
 modelName 
 , 
  
 DownloadType 
 . 
 LOCAL_MODEL 
 , 
  
 conditions 
 ) 
  
 . 
 addOnCompleteListener 
  
 { 
  
 if 
  
 ( 
 ! 
 it 
 . 
 isSuccessful 
 ) 
  
 { 
  
 showToast 
 ( 
 context 
 , 
  
 "Failed to get model file." 
 ) 
  
 } 
  
 else 
  
 { 
  
 showToast 
 ( 
 context 
 , 
  
 "Downloaded remote model: $modelName" 
 ) 
  
 GlobalScope 
 . 
 launch 
  
 { 
  
 initializeInterpreter 
 ( 
 it 
 . 
 result 
 ) 
  
 } 
  
 } 
  
 } 
  
 . 
 addOnFailureListener 
  
 { 
  
 showToast 
 ( 
 context 
 , 
  
 "Model download failed for recommendations, please check your connection." 
 ) 
  
 } 
  
 } 
 

13. Integrate the Tensorflow Lite recommendation model in your app

Tensorflow Lite runtime will let you use your model in the app to generate recommendations. In the previous step we initialized a TFlite interpreter with the model file we downloaded. In this step, we'll first load a dictionary and labels to accompany our model in the inference step, then we'll add pre-processing to generate the inputs to our model and post-processing where we will extract the results from our inference.

Load Dictionary and Labels

The labels used to generate the recommendation candidates by the recommendations model are listed in the file sorted_movie_vocab.jsonin the res/assets folder. Copy the following code to load these candidates.

RecommendationClient.kt

   
 /** 
  
 Load 
  
 recommendation 
  
 candidate 
  
 list 
 . 
  
 */ 
  
 private 
  
 suspend 
  
 fun 
  
 loadCandidateList 
 () 
  
 { 
  
 return 
  
 withContext 
 ( 
 Dispatchers 
 . 
 IO 
 ) 
  
 { 
  
 val 
  
 collection 
  
 = 
  
 MovieRepository 
 . 
 getInstance 
 ( 
 context 
 ) 
 . 
 getContent 
 () 
  
 for 
  
 ( 
 item 
  
 in 
  
 collection 
 ) 
  
 { 
  
 candidates 
 [ 
 item 
 . 
 id 
 ] 
  
 = 
  
 item 
  
 } 
  
 Log 
 . 
 v 
 ( 
 TAG 
 , 
  
 "Candidate list loaded." 
 ) 
  
 } 
  
 } 
 

Implement Pre-processing

In the pre-processing step, we change the form of the input data to match what our model expects. Here, we pad the input length with a placeholder value if we have not generated a lot of user likes already. Copy the code below:

RecommendationClient.kt

   
 /** Given a list of selected items, preprocess to get tflite input.  */ 
  
 @Synchronized 
  
 private 
  
 suspend 
  
 fun 
  
 preprocess 
 ( 
 selectedMovies 
 : 
  
 List<Movie> 
 ) 
 : 
  
 IntArray 
  
 { 
  
 return 
  
 withContext 
 ( 
 Dispatchers 
 . 
 Default 
 ) 
  
 { 
  
 val 
  
 inputContext 
  
 = 
  
 IntArray 
 ( 
 config 
 . 
 inputLength 
 ) 
  
 for 
  
 ( 
 i 
  
 in 
  
 0 
  
 until 
  
 config 
 . 
 inputLength 
 ) 
  
 { 
  
 if 
  
 ( 
 i 
 < 
 selectedMovies 
 . 
 size 
 ) 
  
 { 
  
 val 
  
 ( 
 id 
 ) 
  
 = 
  
 selectedMovies 
 [ 
 i 
 ] 
  
 inputContext 
 [ 
 i 
 ] 
  
 = 
  
 id 
  
 } 
  
 else 
  
 { 
  
 // 
  
 Padding 
  
 input 
 . 
  
 inputContext 
 [ 
 i 
 ] 
  
 = 
  
 config 
 . 
 pad 
  
 } 
  
 } 
  
 inputContext 
  
 } 
  
 } 
 

Run interpreter to generate recommendations

Here we use the model we downloaded in a previous step to run inference on our pre-processed input. We set the type of input and output for our model and run inference to generate our movie recommendations. Copy the following code into your app.

RecommendationClient.kt

   
/**  
Given  
a  
list  
of  
selected  
items,  
and  
returns  
the  
recommendation  
results.  
*/  
@Synchronized  
suspend  
fun  
recommend(selectedMovies:  
List<Movie>):  
List<Result>  
{  
return  
withContext(Dispatchers.Default)  
{  
val  
inputs  
=  
arrayOf<Any>(preprocess(selectedMovies))  
//  
Run  
inference.  
val  
outputIds  
=  
IntArray(config.outputLength)  
val  
confidences  
=  
FloatArray(config.outputLength)  
val  
outputs:  
MutableMap<Int,  
Any>  
=  
HashMap()  
outputs[config.outputIdsIndex]  
=  
outputIds  
outputs[config.outputScoresIndex]  
=  
confidences  
tflite?.let  
{  
it.runForMultipleInputsOutputs(inputs,  
outputs)  
postprocess(outputIds,  
confidences,  
selectedMovies)  
}  
?:  
run  
{  
Log.e(TAG,  
"No  
tflite  
interpreter  
loaded")  
emptyList()  
}  
}  
} 

Implement Post-processing

Finally, in this step we post-process the output from our model, selecting the results with the highest confidence and removing contained values (movies the user has already liked). Copy the following code into your app.

RecommendationClient.kt

   
 /** Postprocess to gets results from tflite inference.  */ 
  
 @Synchronized 
  
 private 
  
 suspend 
  
 fun 
  
 postprocess 
 ( 
  
 outputIds 
 : 
  
 IntArray 
 , 
  
 confidences 
 : 
  
 FloatArray 
 , 
  
 selectedMovies 
 : 
  
 List<Movie> 
  
 ) 
 : 
  
 List<Result> 
  
 { 
  
 return 
  
 withContext 
 ( 
 Dispatchers 
 . 
 Default 
 ) 
  
 { 
  
 val 
  
 results 
  
 = 
  
 ArrayList<Result> 
 () 
  
 // 
  
 Add 
  
 recommendation 
  
 results 
 . 
  
 Filter 
  
 null 
  
 or 
  
 contained 
  
 items 
 . 
  
 for 
  
 ( 
 i 
  
 in 
  
 outputIds 
 . 
 indices 
 ) 
  
 { 
  
 if 
  
 ( 
 results 
 . 
 size 
  
> = 
  
 config 
 . 
 topK 
 ) 
  
 { 
  
 Log 
 . 
 v 
 ( 
 TAG 
 , 
  
 String 
 . 
 format 
 ( 
 "Selected top K: %d. Ignore the rest." 
 , 
  
 config 
 . 
 topK 
 )) 
  
 break 
  
 } 
  
 val 
  
 id 
  
 = 
  
 outputIds 
 [ 
 i 
 ] 
  
 val 
  
 item 
  
 = 
  
 candidates 
 [ 
 id 
 ] 
  
 if 
  
 ( 
 item 
  
 == 
  
 null 
 ) 
  
 { 
  
 Log 
 . 
 v 
 ( 
 TAG 
 , 
  
 String 
 . 
 format 
 ( 
 "Inference output[%d]. Id: %s is null" 
 , 
  
 i 
 , 
  
 id 
 )) 
  
 continue 
  
 } 
  
 if 
  
 ( 
 selectedMovies 
 . 
 contains 
 ( 
 item 
 )) 
  
 { 
  
 Log 
 . 
 v 
 ( 
 TAG 
 , 
  
 String 
 . 
 format 
 ( 
 "Inference output[%d]. Id: %s is contained" 
 , 
  
 i 
 , 
  
 id 
 )) 
  
 continue 
  
 } 
  
 val 
  
 result 
  
 = 
  
 Result 
 ( 
  
 id 
 , 
  
 item 
 , 
  
 confidences 
 [ 
 i 
 ] 
  
 ) 
  
 results 
 . 
 add 
 ( 
 result 
 ) 
  
 Log 
 . 
 v 
 ( 
 TAG 
 , 
  
 String 
 . 
 format 
 ( 
 "Inference output[%d]. Result: %s" 
 , 
  
 i 
 , 
  
 result 
 )) 
  
 } 
  
 results 
  
 } 
  
 } 
 

Test your app!

Re-run your app. As you select a few movies, it should automatically download the new model and start generating recommendations!

14. Congratulations!

You have built a recommendations feature into your app using TensorFlow Lite and Firebase. Note that the techniques and pipeline shown in this codelab can be generalized and used to serve other types of recommendations as well.

What we've covered

• Firebase ML
• Firebase Analytics
• Export analytics events to BigQuery
• Preprocess analytics events
• Train recommendations TensorFlow model
• Export model and deploy to Firebase Console
• Serve movie recommendations in an app

Next Steps

• Implement Firebase ML recommendations in your app.

Learn More

• Firebase.google.com

Have a Question?

• Stackoverflow ‘firebase-machine-learning'

Report Issues

• Github Issues