SwipeVideo is a startup based in Japan that makes a unique software that combines videos from many different angles into one video that a viewer can swipe through. They and the professor running the CEEO lab tasked me with the exploratory project of demonstrating how their technology can be used with LEGO and in classrooms.

The end creation and culmination of this project is shown below. Click and drag on the swipe video to change the angle

Note: The interactive video no longer works unfortunately

The short:

The SPIKE prime robots rotate around a center object and capture a continuous video of a robot conducting some action in the center 128 times
The user runs the video through an executable I created to split the video into 128 separate clips of the robot conducting that action from different angles
SwipeVideo’s software combines the clips into the finished video you see above

To start off, I wrote a short poem describing the goal of this project:

<aside> 🥅 Goal: Provide students with a quick and easy way To comprehensively capture and display Their robotics projects so that they Can very easily convey All that they may Create that day

</aside>

Assorted links:

How I do it

All of the existing users of SwipeVideo were going with the approach of one subject conducting an action once while filmed by many cameras (shown on the right). This approach can get quite expensive, however. For just 10 camera angles you need 10 phones and that is also not nearly enough angles for smooth switching between them.

I went the other direction and instead had one subject conduct the same action repeatedly while filmed by a single phone that moves to many angles around the subject. This approach is much cheaper (only requires one phone) yet it can capture over 10 times the number of camera angles so switching is very smooth.

What other users of SwipeVideo have done

This method of capturing the swipe video is enhanced by the subject: robots. Robots are capable of completing the same action over and over again without any discernable difference. This allows for the illusion that all of the camera angles were filmed at the same time, and ensures the action is smooth.

Capturing videos 📽️

There are two robots required for this setup

The robot in the center which is being filmed
The robot controlling the train

For this project, I used LEGO SPIKE Primes and Jupyter notebooks to wirelessly control both robots over Bluetooth with Python.

What the robots do ⚙️

Both robots wait until you press the right arrow button on both robots to start. This synchronizes their internal clocks
The center robot beeps at a specific frequency to signal that it's starting and then conducts an action while the train waits
Then the train robot moves to the next checkpoint along the track (it knows how far it goes by using a color sensor and counting the changes between grey and green.
And repeats a set number of times 🔄

https://lh4.googleusercontent.com/3c5PhvUeUBq3aePCyq0IP9mQHjQgGsOCSSk2mQ4inGqYcAEZAN6McIyjxP5di9CsHEJoXnnCFg0yRjlT_j1Ybiip3KpHvEDDucFBf2JgPqJ90E_jQariSnD1YdGgNN8zzIfV0vnsSO8

LEGO phone holder I designed

Once I've recorded the center robot with the train setup described above, I am left with a single long video which can be two to twenty minutes long depending on the number of angles it captures and the length of each robot action.

Next, I run that video through the Python Parser Program ⤵️

Coding setup

Python Parser Program 🎞️

Separates individual camera angles from the original video
It uses the beeps that the center robot makes to signal the beginning of the video
Loops the videos if necessary
Can overlay a screen capture of the code running on one half of the swipe video

Finally, I upload all the clips to SwipeVideo which combines them into the final swipe video you saw above.

Why

The whole goal of this project (as described by the poem) is to give young makers an easier way to more fully document their projects.

Normally when documenting a robotics project, a single video is just too one-dimensional to give an audience a complete understanding of the project, but with SwipeVideo and the setup I created, a maker can capture all angles of their robotics project easily and efficiently.

Added Features

The program uses a Fast Fourier Transform to ensure it only picks up beeps of a specified frequency. This ensures the system works even in a noisy classroom environment.

It is also fault tolerant in that, if it picks up fewer beeps than expected, it will automatically adjust the magnitude threshold until it reads correctly spaced beeps.

Going further

After building the train, formalizing all the Python programs, and writing an instruction guide on the entire process of making a swipe video, I ran a hackathon to test to ease of use with the other interns at the CEEO.

It went quite well, and everyone succeeded in making their own swipe video (with a bit of my help as well).

For this hackathon I also had some people make swipe videos by hand, not using the train. They would move their phone to a new position every time the robot completed its action.

This was obviously not as smooth as the train, but it allowed for more creativity in the shots. You can get up close for one to examine the intricacies of a mechanism and then get a broad all-encompassing shot to show the context.

Next, to continue to test the setup and give kids a chance to use it to document their projects we are planning to move the setup to a makerspace in Littleton NH.

Limitations

This setup is certainly not without its limitations. The main ones concern the robot and the constraints and how it has to be built and programmed.

The most important feature of this robot is that it is capable of doing exactly the same action repeatedly - over 100 times. This means that the robot cannot require external input in the form of human interaction or external sensors.

This can be limiting, but also emphasizes the goals of many practical robots. They need to be built and coded to a standard that they can complete the same action perfectly over and over again.

<aside> 📌 I worked on this project while working at the Tufts CEEO (Center for Engineering Education and Outreach) the summer before my sophomore year at Tufts. See my resume for more.

</aside>

<aside> <img src="/icons/home_blue.svg" alt="/icons/home_blue.svg" width="40px" /> Home

</aside>

<aside> <img src="/icons/document_green.svg" alt="/icons/document_green.svg" width="40px" /> Resume

</aside>