mp.tasks.vision.ImageSegmenter

    Class that performs image segmentation on images.

      mp 
 . 
 tasks 
 . 
 vision 
 . 
 ImageSegmenter 
 ( 
 graph_config 
 , 
 running_mode 
 , 
 packet_callback 
 ) 
 -> 
 None

    The API expects a TFLite model with mandatory TFLite Model Metadata.

    (kTfLiteUInt8/kTfLiteFloat32)
    • image input of size [batch x height x width x channels] .
    • batch inference is not supported ( batch is required to be 1).
    • RGB and greyscale inputs are supported ( channels is required to be 1 or 3).
    • if type is kTfLiteFloat32, NormalizationOptions are required to be attached to the metadata for input normalization.

    (kTfLiteUInt8/kTfLiteFloat32)
    • list of segmented masks.
    • if output_category_mask is True, uint8 Image, Image vector of size 1.
    • if output_confidence_masks is True, float32 Image list of size channels .
    • batch is always 1

    An example of such model can be found at: https://tfhub.dev/tensorflow/lite-model/deeplabv3/1/metadata/2

    labels
    Get the category label list the ImageSegmenter can recognize.

    For CATEGORY_MASK type, the index in the category mask corresponds to the category in the label list. For CONFIDENCE_MASK type, the output mask list at index corresponds to the category in the label list.

    If there is no label map provided in the model file, empty label list is returned.

    Methods

    close

    View source 

      close 
 () 
 -> 
 None

    Shuts down the mediapipe vision task instance.

    Raises

    RuntimeError
    If the mediapipe vision task failed to close.

    convert_to_normalized_rect

    View source 

      convert_to_normalized_rect 
 ( 
 options 
 : 
   mp 
 . 
 tasks 
 . 
 vision 
 . 
 holistic_landmarker 
 . 
 image_processing_options_module 
 . 
 ImageProcessingOptions 
 
 
 , 
 image 
 : 
   mp 
 . 
 Image 
 
 
 , 
 roi_allowed 
 : 
 bool 
 = 
 True 
 ) 
 -> 
   mp 
 . 
 tasks 
 . 
 components 
 . 
 containers 
 . 
 NormalizedRect

    Converts from ImageProcessingOptions to NormalizedRect, performing sanity checks on-the-fly.

    If the input ImageProcessingOptions is not present, returns a default NormalizedRect covering the whole image with rotation set to 0. If 'roi_allowed' is false, an error will be returned if the input ImageProcessingOptions has its 'region_of_interest' field set.

    Args

    options
    Options for image processing.
    image
    The image to process.
    roi_allowed
    Indicates if the region_of_interest field is allowed to be set. By default, it's set to True.

    Returns
    A normalized rect proto that represents the image processing options.

    create_from_model_path

    View source 

      @classmethod 
 
  create_from_model_path 
 ( 
 model_path 
 : 
 str 
 ) 
 -> 
 'ImageSegmenter'

    Creates an ImageSegmenter object from a TensorFlow Lite model and the default ImageSegmenterOptions .

    Note that the created ImageSegmenter instance is in image mode, for performing image segmentation on single image inputs.

    Args

    model_path
    Path to the model.

    Returns
    ImageSegmenter object that's created from the model file and the default ImageSegmenterOptions .

    Raises

    ValueError
    If failed to create ImageSegmenter object from the provided file such as invalid file path.
    RuntimeError
    If other types of error occurred.

    create_from_options

    View source 

      @classmethod 
 
  create_from_options 
 ( 
 options 
 : 
   mp 
 . 
 tasks 
 . 
 vision 
 . 
 ImageSegmenterOptions 
 
 
 ) 
 -> 
 'ImageSegmenter'

    Creates the ImageSegmenter object from image segmenter options.

    Args

    options
    Options for the image segmenter task.

    Returns
    ImageSegmenter object that's created from options .

    Raises

    ValueError
    If failed to create ImageSegmenter object from ImageSegmenterOptions such as missing the model.
    RuntimeError
    If other types of error occurred.

    get_graph_config

    View source 

      get_graph_config 
 () 
 -> 
   mp 
 . 
 calculators 
 . 
 core 
 . 
 constant_side_packet_calculator_pb2 
 . 
 mediapipe_dot_framework_dot_calculator__pb2 
 . 
 CalculatorGraphConfig

    Returns the canonicalized CalculatorGraphConfig of the underlying graph.

    segment

    View source 

      segment 
 ( 
 image 
 : 
   mp 
 . 
 Image 
 
 
 , 
 image_processing_options 
 : 
 Optional 
 [ 
   mp 
 . 
 tasks 
 . 
 vision 
 . 
 holistic_landmarker 
 . 
 image_processing_options_module 
 . 
 ImageProcessingOptions 
 
 
 ] 
 = 
 None 
 ) 
 -> 
 ImageSegmenterResult

    Performs the actual segmentation task on the provided MediaPipe Image.

    Args

    image
    MediaPipe Image.
    image_processing_options
    Options for image processing.

    Returns
    If the output_type is CATEGORY_MASK, the returned vector of images is per-category segmented image mask. If the output_type is CONFIDENCE_MASK, the returned vector of images contains only one confidence image mask. A segmentation result object that contains a list of segmentation masks as images.

    Raises

    ValueError
    If any of the input arguments is invalid.
    RuntimeError
    If image segmentation failed to run.

    segment_async

    View source 

      segment_async 
 ( 
 image 
 : 
   mp 
 . 
 Image 
 
 
 , 
 timestamp_ms 
 : 
 int 
 , 
 image_processing_options 
 : 
 Optional 
 [ 
   mp 
 . 
 tasks 
 . 
 vision 
 . 
 holistic_landmarker 
 . 
 image_processing_options_module 
 . 
 ImageProcessingOptions 
 
 
 ] 
 = 
 None 
 ) 
 -> 
 None

    Sends live image data (an Image with a unique timestamp) to perform image segmentation.

    Only use this method when the ImageSegmenter is created with the live stream running mode. The input timestamps should be monotonically increasing for adjacent calls of this method. This method will return immediately after the input image is accepted. The results will be available via the result_callback provided in the ImageSegmenterOptions . The segment_async method is designed to process live stream data such as camera input. To lower the overall latency, image segmenter may drop the input images if needed. In other words, it's not guaranteed to have output per input image.

    The result_callback prvoides:

    • A segmentation result object that contains a list of segmentation masks as images.
    • The input image that the image segmenter runs on.
    • The input timestamp in milliseconds.

    Args

    image
    MediaPipe Image.
    timestamp_ms
    The timestamp of the input image in milliseconds.
    image_processing_options
    Options for image processing.

    Raises

    ValueError
    If the current input timestamp is smaller than what the image segmenter has already processed.

    segment_for_video

    View source 

      segment_for_video 
 ( 
 image 
 : 
   mp 
 . 
 Image 
 
 
 , 
 timestamp_ms 
 : 
 int 
 , 
 image_processing_options 
 : 
 Optional 
 [ 
   mp 
 . 
 tasks 
 . 
 vision 
 . 
 holistic_landmarker 
 . 
 image_processing_options_module 
 . 
 ImageProcessingOptions 
 
 
 ] 
 = 
 None 
 ) 
 -> 
 ImageSegmenterResult

    Performs segmentation on the provided video frames.

    Only use this method when the ImageSegmenter is created with the video running mode. It's required to provide the video frame's timestamp (in milliseconds) along with the video frame. The input timestamps should be monotonically increasing for adjacent calls of this method.

    Args

    image
    MediaPipe Image.
    timestamp_ms
    The timestamp of the input video frame in milliseconds.
    image_processing_options
    Options for image processing.

    Returns
    If the output_type is CATEGORY_MASK, the returned vector of images is per-category segmented image mask. If the output_type is CONFIDENCE_MASK, the returned vector of images contains only one confidence image mask. A segmentation result object that contains a list of segmentation masks as images.

    Raises

    ValueError
    If any of the input arguments is invalid.
    RuntimeError
    If image segmentation failed to run.

    __enter__

    View source 

      __enter__ 
 ()

    Return self upon entering the runtime context.

    __exit__

    View source 

      __exit__ 
 ( 
 unused_exc_type 
 , 
 unused_exc_value 
 , 
 unused_traceback 
 )

    Shuts down the mediapipe vision task instance on exit of the context manager.

    Raises

    RuntimeError
    If the mediapipe vision task failed to close.
    Design a Mobile Site
    View Site in Mobile | Classic
    Share by: