MediaPipe Framework on iOS

Follow instructions below to build iOS example apps with MediaPipe Framework. To learn more about these example apps, start from, start from Hello World! on iOS.

Building iOS example apps

Prerequisite

  1. Install Mediapipe Framework following these instructions.

  2. Install Xcode, then install the Command Line Tools using:

    xcode-select --install

  3. Install Bazelisk.

    We recommend using Homebrew to get the latest versions.

    brew install bazelisk

  4. Set Python 3.7 as the default Python version and install the Python "six" library. This is needed for TensorFlow.

    pip3 install --user six

  5. Clone the MediaPipe repository.

    git clone https://github.com/google/mediapipe.git

Set up a bundle ID prefix

All iOS apps must have a bundle ID, and you must have a provisioning profile that lets you install an app with that ID onto your phone. To avoid clashes between different MediaPipe users, you need to configure a unique prefix for the bundle IDs of our iOS demo apps.

If you have a custom provisioning profile, see Custom provisioning below.

Otherwise, run this command to generate a unique prefix:

python3 mediapipe/examples/ios/link_local_profiles.py

Create an Xcode project

This lets you to edit and debug one of the example apps in Xcode. It also lets you to make use of automatic provisioning (see later section).

  1. We will use a tool called Tulsi for generating Xcode projects from Bazel build configurations.

    # cd out of the mediapipe directory, then:
git clone https://github.com/bazelbuild/tulsi.git
cd tulsi
# remove Xcode version from Tulsi's .bazelrc (see http://github.com/bazelbuild/tulsi#building-and-installing):
sed -i .orig '/xcode_version/d' .bazelrc
# build and run Tulsi:
sh build_and_run.sh

    This will install Tulsi.app inside the Applications directory in your home directory.

  2. Open mediapipe/Mediapipe.tulsiproj using the Tulsi app.

  3. Select the MediaPipe config in the Configs tab, then press the Generate button below. You will be asked for a location to save the Xcode project. Once the project is generated, it will be opened in Xcode.

    If you get an error about bundle IDs, see the previous section.

Set up provisioning

To install applications on an iOS device, you need a provisioning profile. There are two options:

  1. Automatic provisioning. This allows you to build and install an app to your personal device. The provisioning profile is managed by Xcode, and has to be updated often (it is valid for about a week).

  2. Custom provisioning. This uses a provisioning profile associated with an Apple developer account. These profiles have a longer validity period and can target multiple devices, but you need a paid developer account with Apple to obtain one.

Automatic provisioning

  1. Create an Xcode project for MediaPipe, as discussed earlier

  2. In the project navigator in the left sidebar, select the "Mediapipe" project.

  3. Select one of the application targets, for example, HandTrackingGpuApp.

  4. Select the "Signing & Capabilities" tab.

  5. Check "Automatically manage signing", and confirm the dialog.

  6. Select "Your Name (Personal Team)" in the Team dialog menu.

  7. This set-up needs to be done once for each application you want to install. Repeat steps 3-6 as needed.

This generates provisioning profiles for each app you have selected. Now you need to tell Bazel to use them. We have provided a script to make this easier.

  1. In the terminal, to the mediapipe directory where you cloned the repository.

  2. Run this command:

    python3 mediapipe/examples/ios/link_local_profiles.py

This will find and link the provisioning profile for all applications for which you have enabled automatic provisioning in Xcode.

Custom provisioning

  1. Obtain a provisioning profile from Apple.

  1. Symlink or copy your provisioning profile to mediapipe/mediapipe/provisioning_profile.mobileprovision.

    cd mediapipe
ln -s ~/Downloads/MyProvisioningProfile.mobileprovision mediapipe/provisioning_profile.mobileprovision
  1. Open mediapipe/examples/ios/bundle_id.bzl, and change the BUNDLE_ID_PREFIX to a prefix associated with your provisioning profile.

Build and run an app using Xcode

  1. Create the Xcode project, and make sure you have set up either automatic or custom provisioning.

  2. You can now select any of the Mediapipe Framework demos in the target menu, and build and run them as normal.

Build an app using the command line

  1. Make sure you have set up either automatic or custom provisioning.

  2. Using MediaPipe Hands for example, run:

    bazel build -c opt --config=ios_arm64 mediapipe/examples/ios/handtrackinggpu:HandTrackingGpuApp

    You may see a permission request from codesign in order to sign the app.

  3. In Xcode, open the Devices and Simulators window (command-shift-2).

  4. Make sure your device is connected. You will see a list of installed apps. Press the "+" button under the list, and select the .ipa file built by Bazel.

  5. You can now run the app on your device.