This post was written by Zlata Barshteyn, a former intern here at Knewton.
Every few months at Knewton, we have something called Hack Day, where team members take a day off from regular work tasks to tackle projects they’ve been itching to develop. These “hacks” typically fall into one of four categories: product, business, culture, or performance.
For my Hack Day project, I worked with my mentor on building a LEGO Mindstorms NXT robot that could serve as a minion around the office. We programmed it to drive around and avoid colliding with obstacles, with help from an infrared sensor. Mounting an Android phone on the robot and having the robot communicate with the phone expanded its capabilities, as the phone had a camera and access to the Web, which the robot did not.
Equipped with a camera and Internet access, the robot was pretty powerful. It could provide virtual tours of the Knewton office, driving around and broadcasting its camera feed to a web page that could be accessed remotely. Viewers could even drive the robot themselves through built-in controls on the page, making the tours extremely flexible and open-ended. With text-to-speech capabilities, the robot could say each of the Knewton team’s SVN commit messages as they came in, and say a phrase typed in by a viewer on the web page. With enough time and Lego bricks, the robot could even be trained to fetch snacks from the kitchen and deliver them personally.
To get the robot to communicate with the phone, we had two options: write something from scratch to send and receive data over Bluetooth, or use the Cellbots app or Python library. The Cellbots team provides software and code libraries to facilitate communication between Android phones and various types of robots, such as the NXT. Using the app would provide many features and controls from the get-go, but we wanted to write and customize code ourselves, so we went with the Python library.
Running Python on Android
Now, the question is, how would Python code run on an Android phone? After all, Android apps are written in Java. That’s where Scripting Layer for Android, a handy Android app, comes in. Once you install it on your phone, you can run files written in a multitude of scripting languages (such as Python) by putting them into the app’s directory on the phone, then opening the app and selecting them. It even comes with “Facades”, which are basically APIs that give you simplified control over things such as Bluetooth and the camera in the scripting language of your choice.
This was our reasoning: If we could run Python scripts on the phone, then we could have the phone host a web server, fetch a video feed using the SL4A Webcam Facade, and display it on the server. Then, whoever wished could navigate to the server’s address and watch a live feed of what the robot “saw” through the Android phone’s camera. Controls could then be placed on the page, allowing someone to turn off the robot’s autonomous navigation and drive it around the office themselves, or even have it speak a given phrase, using the SL4A Text-to-Speech Facade.
Setting Up a Web Server on Your Phone
The first step is to find the IP address of the phone, in order to access the web server from another computer. Check out this blog post (scroll down to “Gleaning your IP address”). Do you want a web page that only lists the contents of some directory on your phone or SD card? If so, you can just use Python’s built-in SimpleHTTPServer, which is the server demonstrated in the previously mentioned blog post. As you can see, it only requires 4 lines of code:
from os import chdir
To run the server and view the page, do the following:
1. Save this code in the SL4A directory
2. Open the SL4A app on your phone
3. Run the code you just wrote
4. Open up a web browser on your computer
5. Navigate to the IP address you saved earlier, port 8000
(so if the IP address was 192.168.0.2, you’d go to http://192.168.0.2:8000)
You will then see a listing of the contents of /sdcard/. As you can see, this is a fairly limited web server, and is only useful for, say, browsing your photos. It definitely wouldn’t be enough for what we wanted to do with the robot (embed a video feed and controls). For that, we’d need a more sophisticated web server.
Luckily, using SL4A means that anything you can do with Python on your computer, you can do on your phone. That means that you can even run CherryPy on Android with no hassles, or any other Python web server, really. Once that’s done, you can use the SL4A Facades in your server code, meaning you can use the Webcam Facade to stream a video feed from your phone’s camera, the Text-to-Speech Facade to have the robot say out loud a phrase the user typed in, or any of the other Facades in a variety of creative ways to have your very own office robot/minion.
The bottom line is, if you love developing in Python or any other scripting language, you don’t have to suddenly stop just because you’re working on the Android platform, thanks to Scripting Layer for Android.
Posted in Knerds