Multi-camera Mocap Run ###################### This guides you through the process of creating a mocap run when using multiple capture inputs. To create a mocap run you will need to create an instance of `MocapRun` and provision that through a specific compute class. However, you will need to go through a specific process to be able to do this: 1. :ref:`Define camera settings and capture devices ` 2. :ref:`Create a capture volume ` 1. You will need to :ref:`create capture inputs for this ` 2. This can be either an aruco volume or an actor volume 3. :ref:`Create multiple capture inputs ` 1. This will be the inputs used for the mocap run 4. [Optional] **NOT YET IMPLEMENTED** :ref:`Create actor skeletons for all actors in your capture inputs ` 5. :ref:`Create a mocap run ` 1. Define camera settings and capture devices --------------------------------------------- .. _multicam-define-camera-settings-and-capture-devices: You will need to define the camera settings and capture devices that you used .. code-block:: python from moveai_mocap.models.captures.camera_settings import CameraSettings from moveai_mocap.models.captures.capture_device import CaptureDevice camera_settings = CameraSettings( model="GoProHero7-FHD", resolution_x=1920, resolution_y=1080, focal_length_x=1038.37, focal_length_y=1029.15, principal_point_x=953.06, principal_point_y=719.21, distortion_coeffs=(0, 0, 0, 0, 0), ) # We had 4 capture devices each 90 degrees apart from each other. They # were all the same type of camera capture_device_0 = CaptureDevice( device_id="capture_device_0", camera_settings=camera_settings, ) capture_device_90 = CaptureDevice( device_id="capture_device_90", camera_settings=camera_settings, ) capture_device_180 = CaptureDevice( device_id="capture_device_180", camera_settings=camera_settings, ) capture_device_270 = CaptureDevice( device_id="capture_device_270", camera_settings=camera_settings, ) 2. Create a capture volume -------------------------- .. _multicam-create-capture-volume: Create capture inputs ^^^^^^^^^^^^^^^^^^^^^ .. _multicam-create-volume-capture-inputs: We then define the VideoCaptureInputs that our :ref:`capture devices ` created in order for us to create the volume. Ensure that only one actor is present on camera for these inputs. .. code-block:: python from moveai_mocap.models.captures.capture_input import VideoCaptureInput from moveai_mocap.models.resources.enums.resource_type import ResourceType from moveai_mocap.models.resources.enums.service import Service from moveai_mocap.models.resources.resource import Resource video_capture_input_0 = VideoCaptureInput( capture_device=capture_device_0, # The Resource defines that this file is stored on S3 and is an input_mp4 type input=Resource( resource_type=ResourceType.video_capture_input, uri="s3://my-bucket/path/to/capture/capture_device_0.mp4", service=Service.s3, ), ) video_capture_input_90 = VideoCaptureInput( capture_device=capture_device_90, # The Resource defines that this file is stored on S3 and is an input_mp4 type input=Resource( resource_type=ResourceType.video_capture_input, uri="s3://my-bucket/path/to/capture/capture_device_90.mp4", service=Service.s3, ), ) video_capture_input_180 = VideoCaptureInput( capture_device=capture_device_180, # The Resource defines that this file is stored on S3 and is an input_mp4 type input=Resource( resource_type=ResourceType.video_capture_input, uri="s3://my-bucket/path/to/capture/capture_device_180.mp4", service=Service.s3, ), ) video_capture_input_270 = VideoCaptureInput( capture_device=capture_device_270, # The Resource defines that this file is stored on S3 and is an input_mp4 type input=Resource( resource_type=ResourceType.video_capture_input, uri="s3://my-bucket/path/to/capture/capture_device_270.mp4", service=Service.s3, ), ) Create capture ^^^^^^^^^^^^^^ We then define our capture - which pulls all the video capture inputs together .. code-block:: python from moveai_mocap.models.captures.capture import Capture from moveai_mocap.models.captures.sync_methods.clap_audio import ClapAudioSync capture = Capture( inputs=[ video_capture_input_0, video_capture_input_90, video_capture_input_180, video_capture_input_270, ], # Other sync methods are available. If not provided the engine # assumes the videos are fully synced sync_method=ClapAudioSync(clap_window=(0.0, 2.0)), ) Create actor volume run ^^^^^^^^^^^^^^^^^^^^^^^ Once all that is done we can create our ActorVolumeRun .. code-block:: python from moveai_mocap.models.volumes.runs.actor import ActorVolumeRun from moveai_mocap.models.volumes.enums.area import Area from moveai_mocap.models.actors.actor import Actor from moveai_mocap.models.resources.enums.service import Service actor = Actor( name="actor_1", height=1.8, ) actor_volume_run = ActorVolumeRun( # Only one actor is required for creating an actor volume actor=actor, # different size areas are available area=Area.large, capture=capture, volume_service=Service.s3, volume_uri_prefix="s3://my-bucket/path/to/directory", ) Provision the engine ^^^^^^^^^^^^^^^^^^^^ Now that we’ve generated the python object for our actor volume we can provision an engine with it .. code-block:: python from moveai_mocap.compute.ec2 import EC2ComputeManager from moveai_mocap.models.notifiers.sns import SNSNotifier sns_notifier = SNSNotifier( endpoint="arn:aws:sns:", payload={"foo": "bar"}, ) compute = EC2ComputeManager( # configure these settings according to your AWS account iam_instance_profile_name="instance-profile-name", ami_id="ami-12345678", subnet_ids=["subnet-1"], security_group_ids=["sg-1"], notifier=sns_notifier, ) # Calling provision will launch an EC2 instance and generate the output resources # defined on `volume_uri_prefix` on the `actor_volume_run` compute.provision(actor_volume_run) 3. Create capture inputs ------------------------- .. _multicam-create-capture-inputs: Now that you have created your volume you can film as normal with multiple actors and create capture inputs. These capture inputs will be used to generate the mocap resources. .. code-block:: python from moveai_mocap.models.captures.capture_input import VideoCaptureInput from moveai_mocap.models.resources.enums.resource_type import ResourceType from moveai_mocap.models.resources.enums.service import Service from moveai_mocap.models.resources.resource import Resource video_capture_input_0 = VideoCaptureInput( capture_device=capture_device_0, # The Resource defines that this file is stored on S3 and is an input_mp4 type input=Resource( resource_type=ResourceTypeEnum.video_capture_input, uri="s3://my-bucket/path/to/capture/capture_device_0.mp4", service=Service.s3, ), ) video_capture_input_90 = VideoCaptureInput( capture_device=capture_device_90, # The Resource defines that this file is stored on S3 and is an input_mp4 type input=Resource( resource_type=ResourceTypeEnum.video_capture_input, uri="s3://my-bucket/path/to/capture/capture_device_90.mp4", service=Service.s3, ), ) video_capture_input_180 = VideoCaptureInput( capture_device=capture_device_180, # The Resource defines that this file is stored on S3 and is an input_mp4 type input=Resource( resource_type=ResourceTypeEnum.video_capture_input, uri="s3://my-bucket/path/to/capture/capture_device_180.mp4", service=Service.s3, ), ) video_capture_input_270 = VideoCaptureInput( capture_device=capture_device_270, # The Resource defines that this file is stored on S3 and is an input_mp4 type input=Resource( resource_type=ResourceTypeEnum.video_capture_input, uri="s3://my-bucket/path/to/capture/capture_device_270.mp4", service=Service.s3, ), ) Create capture ^^^^^^^^^^^^^^ We then define our capture - which pulls all the video capture inputs together .. code-block:: python from moveai_mocap.models.captures.capture import Capture from moveai_mocap.models.captures.sync_methods.clap_audio import ClapAudioSync capture = Capture( inputs=[ video_capture_input_0, video_capture_input_90, video_capture_input_180, video_capture_input_270, ], # Other sync methods are available. If not provided the engine # assumes the videos are fully synced sync_method=ClapAudioSync(clap_window=(0.0, 2.0)), ) 4. [OPTIONAL] Create actor skeleton ----------------------------------- .. _multicam-create-actor-skeleton: .. warning:: **Not yet implemented** You can optionally create a skeleton for one or more of the actors that are in your capture inputs. This will enable the engine to more accurately generate mocap resources .. code-block:: python from moveai_mocap.models.actors.runs.actor import ActorSkeletonRun from moveai_mocap.models.actors.actor import Actor from moveai_mocap.models.resources.enums.service import Service actor = Actor( name="actor_1", height=1.8, ) actor_skeleton_run = ActorSkeletonRun( capture=capture, volume_service=Service.s3, volume_uri_prefix="s3://my-bucket/path/to/directory", actor=actor, # Where the actor skeleton file should be saved to actor_skeleton_uri_prefix="s3://my-bucket/path/to/skeleton/directory", actor_skeleton_service=Service.s3 ) Provision the engine ^^^^^^^^^^^^^^^^^^^^ .. code-block:: python from moveai_mocap.compute.ec2 import EC2ComputeManager from moveai_mocap.models.notifiers.sns import SNSNotifier sns_notifier = SNSNotifier( endpoint="arn:aws:sns:", payload={"foo": "bar"}, ) compute = EC2ComputeManager( # configure these settings according to your AWS account iam_instance_profile_name="instance-profile-name", ami_id="ami-12345678", subnet_ids=["subnet-1"], security_group_ids=["sg-1"], notifier=sns_notifier, ) # Calling provision will launch an EC2 instance and generate the output resources # defined on `volume_uri_prefix` on the `actor_skeleton_run` compute.provision(actor_skeleton_run) 5. Create a mocap run --------------------- .. _multicam-create-mocap-run: .. warning:: **actor_mapping not yet implemented** We can now use all the data created in the previous steps to provision a mocap run .. code-block:: python from moveai_mocap.models.actors.actor_mapping import ActorMapping from moveai_mocap.models.motion.runs.mocap import MocapRun from moveai_mocap.models.resources.enums.service import Service from moveai_mocap.models.resources.resource import Resource from moveai_mocap.models.resources.enums.resource_type import ResourceType mocap_run = MocapRun( # The capture defined in step 3 capture=capture, # set the number of actors in the capture number_of_actors=2, # This mapping tells compute which actors are where on each video input # The actors here must have a skeleton. If the actors don't have skeletons # then you cannot provide actor mapping and have to use `number_of_actors` only actor_mapping=[ ActorMapping( actor=actor_1, # Where the actor is in the frame bounding_box=(0, 100, 0, 0), video_capture_input=video_capture_input, ), ActorMapping( actor=actor_2, # Where the actor is in the frame bounding_box=(0, 0, 0, 500), video_capture_input=video_capture_input, ), ], volume_service=Service.s3, volume_uri_prefix="s3://my-bucket/path/to/directory", # Where outputs should be saved. There are many types of output # resource outputs=[ Resource( resource_type=ResourceType.main_fbx, uri="s3://my-bucket/path/to/mocap/scene.fbx", service=Service.s3, ), Resource( resource_type=ResourceType.all_fbx, uri="s3://my-bucket/path/to/mocap/all_actors.zip", service=Service.s3, ) ], ) Provision the engine ^^^^^^^^^^^^^^^^^^^^ Then provision the engine .. code-block:: python from moveai_mocap.compute.ec2 import EC2ComputeManager from moveai_mocap.models.notifiers.sns import SNSNotifier sns_notifier = SNSNotifier( endpoint="arn:aws:sns:", payload={"foo": "bar"}, ) compute = EC2ComputeManager( # configure these settings according to your AWS account iam_instance_profile_name="instance-profile-name", ami_id="ami-12345678", subnet_ids=["subnet-1"], security_group_ids=["sg-1"], notifier=sns_notifier, ) # Calling provision will launch an EC2 instance and generate the output resources # defined on `outputs` on the `mocap_run` compute.provision(mocap_run)