In this colab notebook, you can learn how to use theTensorFlow Lite Model Makerlibrary to create a TFLite Searcher model. You can use a text Searcher model to build Semantic Search or Smart Reply for your app. This type of model lets you take a text query and search for the most related entries in a text dataset, such as a database of web pages. The model returns a list of the smallest distance scoring entries in the dataset, including metadata you specify, such as URL, page title, or other text entry identifiers. After building this, you can deploy it onto devices (e.g. Android) usingTask Library Searcher APIto run inference with just a few lines of code.
This tutorial leverages CNN/DailyMail dataset as an instance to create the TFLite Searcher model. You can try with your own dataset with the compatible input comma separated value (CSV) format.
Text search using Scalable Nearest Neighbor
This tutorial uses the publicly available CNN/DailyMail non-anonymized summarization dataset, which was produced from theGitHub repo. This dataset contains over 300k news articles, which makes it a good dataset to build the Searcher model, and return various related news during model inference for a text query.
The text Searcher model in this example uses aScaNN(Scalable Nearest Neighbors) index file that can search for similar items from a predefined database. ScaNN achieves state-of-the-art performance for efficient vector similarity search at scale.
Highlights and urls in this dataset are used in this colab to create the model:
Highlights are the text for generating the embedding feature vectors and then used for search.
Urls are the returned result shown to users after searching the related highlights.
This tutorial saves these data into the CSV file and then uses the CSV file to build the model. Here are several examples from the dataset.
Highlights
Urls
Hawaiian Airlines again lands at No. 1 in on-time performance. The Airline Quality Rankings Report looks at the 14 largest U.S. airlines. ExpressJet and American Airlines had the worst on-time performance. Virgin America had the best baggage handling; Southwest had lowest complaint rate.
European football's governing body reveals list of countries bidding to host 2020 finals. The 60th anniversary edition of the finals will be hosted by 13 countries. Thirty-two countries are considering bids to host 2020 matches. UEFA will announce host cities on September 25.
Once octopus-hunter Dylan Mayer has now also signed a petition of 5,000 divers banning their hunt at Seacrest Park. Decision by Washington Department of Fish and Wildlife could take months.
Galaxy was observed 420 million years after the Big Bang. found by NASA’s Hubble Space Telescope, Spitzer Space Telescope, and one of nature’s own natural 'zoom lenses' in space.
This tutorial uses the dataset CNN / Daily Mail summarization dataset from theGitHub repo.
First, download the text and urls of cnn and dailymail and unzip them. If it
failed to download from google drive, please wait a few minutes to try it again or download it manually and then upload it to the colab.
Then, save the data into the CSV file that can be loaded intotflite_model_makerlibrary. The code is based on the logic used to load this data intensorflow_datasets. We can't usetensorflow_datasetdirectly since it doesn't contain urls which are used in this colab.
Since it takes a long time to process the data into embedding feature vectors
for the whole dataset. Only first 5% stories of CNN and Daily Mail dataset are
selected by default for demo purpose. You can adjust the
fraction or try with the pre-built TFLitemodelwith 50% stories of CNN and Daily Mail dataset to search as well.
Save the highlights and urls to the CSV file
CNN_FRACTION=0.05DAILYMAIL_FRACTION=0.05importcsvimporthashlibimportosimporttensorflowastfdm_single_close_quote=u"\u2019"# unicodedm_double_close_quote=u"\u201d"END_TOKENS=[".","!","?","...","'","`",'"',dm_single_close_quote,dm_double_close_quote,")"]# acceptable ways to end a sentencedefread_file(file_path):"""Reads lines in the file."""lines=[]withtf.io.gfile.GFile(file_path,"r")asf:forlineinf:lines.append(line.strip())returnlinesdefurl_hash(url):"""Gets the hash value of the url."""h=hashlib.sha1()url=url.encode("utf-8")h.update(url)returnh.hexdigest()defget_url_hashes_dict(urls_path):"""Gets hashes dict that maps the hash value to the original url in file."""urls=read_file(urls_path)return{url_hash(url):url[url.find("id_/")+4:]forurlinurls}deffind_files(folder,url_dict):"""Finds files corresponding to the urls in the folder."""all_files=tf.io.gfile.listdir(folder)ret_files=[]forfileinall_files:# Gets the file name without extension.filename=os.path.splitext(os.path.basename(file))[0]iffilenameinurl_dict:ret_files.append(os.path.join(folder,file))returnret_filesdeffix_missing_period(line):"""Adds a period to a line that is missing a period."""if"@highlight"inline:returnlineifnotline:returnlineifline[-1]inEND_TOKENS:returnlinereturnline+"."defget_highlights(story_file):"""Gets highlights from a story file path."""lines=read_file(story_file)# Put periods on the ends of lines that are missing them# (this is a problem in the dataset because many image captions don't end in# periods; consequently they end up in the body of the article as run-on# sentences)lines=[fix_missing_period(line)forlineinlines]# Separate out article and abstract sentenceshighlight_list=[]next_is_highlight=Falseforlineinlines:ifnotline:continue# empty lineelifline.startswith("@highlight"):next_is_highlight=Trueelifnext_is_highlight:highlight_list.append(line)# Make highlights into a single string.highlights="\n".join(highlight_list)returnhighlightsurl_hashes_dict=get_url_hashes_dict("all_train.txt")cnn_files=find_files("cnn/stories",url_hashes_dict)dailymail_files=find_files("dailymail/stories",url_hashes_dict)# The size to be selected.cnn_size=int(CNN_FRACTION*len(cnn_files))dailymail_size=int(DAILYMAIL_FRACTION*len(dailymail_files))print("CNN size:%d"%cnn_size)print("Daily Mail size:%d"%dailymail_size)withopen("cnn_dailymail.csv","w")ascsvfile:writer=csv.DictWriter(csvfile,fieldnames=["highlights","urls"])writer.writeheader()forfileincnn_files[:cnn_size]+dailymail_files[:dailymail_size]:highlights=get_highlights(file)# Gets the filename which is the hash value of the url.filename=os.path.splitext(os.path.basename(file))[0]url=url_hashes_dict[filename]writer.writerow({"highlights":highlights,"urls":url})
Build the text Searcher model
Create a text Searcher model by loading a dataset, creating a model with the data and exporting the TFLite model.
Step 1. Load the dataset
Model Maker takes the text dataset and the corresponding metadata of each text string (such as urls in this example) in the CSV format. It embeds the text strings into feature vectors using the user-specified embedder model.
In this demo, we build the Searcher model usingUniversal Sentence Encoder, a state-of-the-art sentence embedding model which is already retrained fromcolab. The model is optimized for on-device inference performance, and only takes 6ms to embed a query string (measured on Pixel 6). Alternatively, you can usethisquantized version, which is smaller but takes 38ms for each embedding.
Create asearcher.TextDataLoaderinstance and usedata_loader.load_from_csvmethod to load the dataset. It takes ~10 minutes for this
step since it generates the embedding feature vector for each text one by one. You can try to upload your own CSV file and load it to build the customized model as well.
Specify the name of text column and metadata column in the CSV file.
Text is used to generate the embedding feature vectors.
Metadata is the content to be shown when you search the certain text.
Here are the first 4 lines of the CNN-DailyMail CSV file generated above.
highlights
urls
Syrian official: Obama climbed to the top of the tree, doesn't know how to get down. Obama sends a letter to the heads of the House and Senate. Obama to seek congressional approval on military action against Syria. Aim is to determine whether CW were used, not by whom, says U.N. spokesman.
Usain Bolt wins third gold of world championship. Anchors Jamaica to 4x100m relay victory. Eighth gold at the championships for Bolt. Jamaica double up in women's 4x100m relay.
The employee in agency's Kansas City office is among hundreds of "virtual" workers. The employee's travel to and from the mainland U.S. last year cost more than $24,000. The telecommuting program, like all GSA practices, is under review.
NEW: A Canadian doctor says she was part of a team examining Harry Burkhart in 2010. NEW: Diagnosis: "autism, severe anxiety, post-traumatic stress disorder and depression" Burkhart is also suspected in a German arson probe, officials say. Prosecutors believe the German national set a string of fires in Los Angeles.
For image use cases, you can create asearcher.ImageDataLoaderinstance and then usedata_loader.load_from_folderto load images from the folder. Thesearcher.ImageDataLoaderinstance needs to be created by a TFLite embedder model because it will be leveraged to encode queries to feature vectors and be exported with the TFLite Searcher model. For instance:
In the above example, we define the following options:
distance_measure: we use "dot_product" to measure the distance between two embedding vectors. Note that we actually compute thenegativedot product value to preserve the notion that "smaller is closer".
tree: the dataset is divided the dataset into 140 partitions (roughly the square root of the data size), and 4 of them are searched during retrieval, which is roughly 3% of the dataset.
score_ah: we quantize the float embeddings to int8 values with the same dimension to save space.
You can test the exported TFLite model using custom query text. To query text using the Searcher model, initialize the model and run a search with text phrase, as follows:
fromtflite_support.taskimporttext# Initializes a TextSearcher object.searcher=text.TextSearcher.create_from_file("searcher.tflite")# Searches the input query.results=searcher.search("The Airline Quality Rankings Report looks at the 14 largest U.S. airlines.")print(results)
See theTask Library documentationfor more information about how to integrate the model to various platforms.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2026-05-28 UTC."],[],[]]
