Video Class

class plotbee.video.Video(frames, tracks, config, start=0, stop=None, step=1)

Bases: object

__dict__ = mappingproxy({'__module__': 'plotbee.video', 'from_config': <classmethod object>, 'load': <classmethod object>, 'from_detections': <classmethod object>, '__init__': <function Video.__init__>, 'config': <property object>, 'frames': <property object>, 'tracks': <property object>, 'video_path': <property object>, 'video_name': <property object>, 'frame_image': <function Video.frame_image>, '_get_frame': <function Video._get_frame>, 'get_video_stream': <function Video.get_video_stream>, '__repr__': <function Video.__repr__>, '__len__': <function Video.__len__>, 'append': <function Video.append>, '__getitem__': <function Video.__getitem__>, 'export': <function Video.export>, 'clear_ids': <function Video.clear_ids>, 'non_max_supression': <function Video.non_max_supression>, 'hungarian_tracking': <function Video.hungarian_tracking>, 'sort_tracking': <function Video.sort_tracking>, 'track_clasification': <function Video.track_clasification>, 'tag_detection': <function Video.tag_detection>, 'events_counter': <function Video.events_counter>, 'export_bodies': <function Video.export_bodies>, 'json': <function Video.json>, 'save': <function Video.save>, 'clear_tags': <function Video.clear_tags>, 'load_tags': <function Video.load_tags>, 'load_video': <function Video.load_video>, 'tagged': <function Video.tagged>, 'get_frame_with_untracked_body': <function Video.get_frame_with_untracked_body>, 'export_tagged': <function Video.export_tagged>, 'export_pollen': <function Video.export_pollen>, 'process_pollen': <function Video.process_pollen>, '__dict__': <attribute '__dict__' of 'Video' objects>, '__weakref__': <attribute '__weakref__' of 'Video' objects>, '__doc__': None, '__annotations__': {}})
__getitem__(index)

Get frame by index

Parameters

index (int) – index of the requested frame

Returns

the requested frame

Return type

Frame

__init__(frames, tracks, config, start=0, stop=None, step=1)

Default Constructor

Parameters

  • frames (list[Frame]) – Frame object that belong to the video

  • tracks (dict[Track]) – Track of the video

  • config (dict) – config dictionary with path to the video

  • start (int, optional) – starting frame id, defaults to 0

  • stop (int, optional) – end frame id, defaults to None

  • step (int, optional) – step for the video, defaults to 1

__len__()

Get the number of frames

Returns

number of frames in the video

Return type

int

__module__ = 'plotbee.video'
__repr__()

Print video info

Returns

Show video info such as video filename, length and their detection

Return type

str

__weakref__

list of weak references to the object (if defined)

_get_frame(frame_id)

Get frame by frame id

Parameters

frame_id (int) – frame id

Returns

Frame object

Return type

Frame

append(video)

Concatenate a video at the end

Parameters

video (Video) – Video to concatenate

clear_ids()

Clear track ids of all bodies in the video.

clear_tags()

Clear tags for all body in the video.

property config

Get video config file

Returns

config file

Return type

dict

events_counter()

Count Events

Returns

frequecy per event

Return type

dict

export(folder, skeleton=True, bbox=True, tracks=True, events=False, min_parts=- 1, max_workers=5, idtext=False, fontScale=2.5, fontThickness=8)

Export video images with plotting options on folder.

Parameters

  • folder (str) – folder to save the frame images

  • skeleton (bool, optional) – if True images will show the predicted skeleton, defaults to True

  • bbox (bool, optional) – if True images will show the bounding boxes of the predicted bodies, defaults to True

  • tracks (bool, optional) – if True images will show the tracks of the predicted bodies, defaults to True

  • events (bool, optional) – if True draws frame’s detections events on the frame_image, defaults to False

  • min_parts (int, optional) – this functions only draws bodies that len(body) >= min_parts, defaults to -1

  • idtext (bool, optional) – if True shows track id on the image, defaults to False

  • fontScale (float, optional) – idtext font scale, defaults to 1.5

  • fontThickness (int, optional) – idtext fontsize, defaults to 3

  • thickness (int, optional:param events: , defaults to False) – bounding box line thickness, defaults to 7

  • max_workers (int, optional) – workers to process this task in parallel, defaults to 5

export_bodies(folder, width=None, height=None, cX=None, cY=None, workers=5)

Export bodies and body images into a JSON format and image folder

Parameters

  • folder (str) – folder to save the exported json file and images

  • width (int, optional) – width of the patch rectagle,, defaults to 160

  • height (int, optional) – height of the patch rectagle, defaults to 320

  • cX (int, optional) – x of the center point in the output rectangle, defaults to None

  • cY (int, optional) – y of the center point in the output rectangle, defaults to None

  • workers (int, optional) – amount of workers to process this task in parallel, defaults to 5

export_pollen(output_folder, limit=None, sorted_scores=False)

Export pollen detection dataset

Parameters

  • output_folder (str) – folder to save the dataset

  • limit (int, optional) – set a limit of detection to export, defaults to None

  • sorted_scores (bool, optional) – if True the pollen score will be sorted and take the the top limit//2 and the bottom limit//2 to export, defaults to False

Note

This function was design to provide aditional training data for pollen classification in active learning fashion.

export_tagged(output_folder, save_images=True)

Export all tagged bodies in a JSON file and optionally all the body images.

Parameters

  • output_folder (str) – folder to save the images and json file

  • save_images (bool, optional) – if True will export tagged bodies images, defaults to True

Tip

This function will serve to create a dataset of tagged bees.

frame_image(index)

Get frame image at index position

Parameters

index (int) – index of the request frame image

Returns

return image of the requested frame

Return type

numpy.ndarray

property frames

Get a list of frames

Returns

list of video frames

Return type

list[Frame]

classmethod from_config(config, load_image=False)

Video Constructor from config dict (this is used to import beepose format file)

Parameters

  • config (dict) – A dictionary containing the beepose files path - DETECTIONS_PATH path to detection file - VIDEO_PATH path to video - TRACK_PATH path to tracks optional - ID_TRACK_PATH path to the ids of tracks optional

  • load_image (bool, optional) – if True will load images from the video, defaults to False

Returns

Video object

Return type

Video

Warning

load_image = True is not recommended for large videos. This option will result in slower performance and high memory usage.

Warning

This is old, generally Video.load is used.

classmethod from_detections(detections, video_path=None, load_images=False)

Beepose dictionary constructor

Parameters

  • detections (dict) – Beepose dictionary with detections

  • video_path (str, optional) – video path, defaults to None

  • load_images (bool, optional) – if True images will load at loading time, defaults to False

Returns

Video Object

Return type

Video

Note

This function is mostly internal. This function runs on beepose implementation to generate plotbee output.

get_frame_with_untracked_body()

Get the first frame with an untracked body

Returns

First frame that at least has one detection without a tracking id

Return type

Frame

Note

This was implemented to help to annotate bodies for a track evaluation.

get_video_stream(start=None, stop=None, step=None)

Get video stream

Parameters

  • start (int, optional) – frame to start, defaults to None

  • stop (int, optional) – frame to stop, defaults to None

  • step (int, optional) – frame step, defaults to None

Returns

An “efficient” VideoCapture Wrapper

Return type

VideoCaptureWrapper

hungarian_tracking(predictive=True, **kwargs)

Hungarian Tracking

Parameters

predictive (bool, optional) – if True in case that a body was misdetected at frame t+1 then this will be filled with a virtual detection in order to continue with the track, defaults to True

json()

Export Video to JSON format

Returns

Video as JSON format

Return type

dict

classmethod load(json_path)

JSON Constructor

Parameters

json_path (str) – plotbee file path. Normally, with suffix “_skeleton.json”

Returns

Video Object

Return type

Video

load_tags(tags_file, virtual=False)

Load tags from Tag multi-file format

Parameters

  • tags_file (str) – path to the tag file to load

  • virtual (bool, optional) – if True will attach tags to virtual body, defaults to False

load_video(video_file)

Set the video path to load images

Parameters

video_file (str) – path to the video file

Note

load_video is critical for all plotting. To load the frame image it is important that Video knows the location to load the frame image.

non_max_supression(nms_overlap_fraction=0.6)

Non maximum supression will supress all the bodies detection that hava an overlap greater or equal to nms_overlap_fraction to remove potential detection duplicates.

Parameters

nms_overlap_fraction (float, optional) – amount of overlaping fraction to supress a detection, defaults to 0.6

process_pollen(model_path, weights=None, workers=4, gpus=['1', '0'], model_size=2048, scale=1.0)

Process polle

Parameters

  • model_path (str) – path to the pollen detection model

  • weights (str, optional) – path to the weights of the model, defaults to None

  • workers (int, optional) – amount of workers to process this task in parallel, defaults to 4

  • gpus (list, optional) – which GPU this task will use, defaults to [“1”, “0”]

  • model_size (int, optional) – size of the model, defaults to 2048

  • scale (float, optional) – upsampling (resize) scale of the body image (scale > 1.0 => zoom), defaults to 1.0

save(path)

Save video as JSON

Parameters

path (str) – path to save the video data

sort_tracking(bbox=200, nms_overlap_fraction=0.6)

SORT tracking

Parameters

  • bbox (int, optional) – the size of the tracking box, defaults to 200

  • nms_overlap_fraction (float, optional) – amount of overlaping fraction to supress a detection, defaults to 0.6

tag_detection(max_workers=5, frame_level=False)

Tag detection of the video

Parameters

  • max_workers (int, optional) – amount of worker to perform the task in parallel, defaults to 5

  • frame_level (bool, optional) – if True the tag will be detected at frame level and then match the close body to the detected tag. if False the tag detection will be perform to each body image on the video, defaults to False

tagged()

Get all tagged body.

Returns

list of body with tags detected

Return type

list[Body]

track_clasification(**kwargs)

Track Classification This will classify all the tracks in the video.

property tracks

Get tracks

Returns

video tracks

Return type

dict

property video_name

Get video name

Returns

video name

Return type

str

property video_path

Get video path

Returns

video path

Return type

str

plotbee.video.create_bodies_from_mapping(tracking_limb, tracking_part, mapping, tracks, id_tracks, frame)
plotbee.video.dist(a, b)
plotbee.video.divide_video(video, fname, N)
plotbee.video.find_bodyid(body_point, tracks, id_tracks, frame_id)
plotbee.video.find_connections(point, part, mappings)
plotbee.video.frames_from_detections(detections, tracks, id_tracks, tracking_limb=(1, 3), tracking_part=3, video_path=None, load_image=False)

Create Video from Ivan’s detections and tracks JSON objects

plotbee.video.get_mappings_by_limb(maps)
plotbee.video.get_video(video_path, start=0, end=inf, step=1)
plotbee.video.image_from_video(video_path, frameid)
plotbee.video.load_json_model(json_file)
plotbee.video.load_pollen_model(model_path)
plotbee.video.match_tags(frame, tag_list, virtual=False, th_dist=50)
plotbee.video.merge_videos(video_names)
plotbee.video.point_in_frame(track_id, id_tracks, frame_id)
plotbee.video.preprocess_input(image, rescale_factor=1)
plotbee.video.process_frame(image, frame, img_folder, file_format)
plotbee.video.process_frame_consumer(video_path, q, lock, pbar)
plotbee.video.process_pollen(video, model_path, model_weights, workers=4, gpus=['1', '0'], model_size=2048, scale=1.0)
plotbee.video.process_video(frames, video_path, start, end, img_folder, file_format, lock, pbar)
plotbee.video.tfv2_pollen_classifier(video_filename, model_path, weigths_path, gpu, gpu_fraction, model_size=2048, scale=1.0)