The goal of this project is to make a system which plays pinball autonomously. This requires keeping track of the ball and activating the flippers of the pinball machine, when the ball is in position.
The original reason for building the pinball playing robot was to provide a realistic test-case for real-time programming technology, and in particular for real-time scheduling algorithms and real-time executives. The assumption was that we could build a system similar in structure and complexity to a radar tracking and defense system, but which could be developed in a university lab. The role of the video camera would be similar the role of a radar, and role of the ball would be similar to the role of attacking aircraft.
This project made use of the following equipment and software:
How these are connected is illustrated in Figure 1. The video frame captured by the video camera goes to the FG-100-V image processor. The FG-100-V is accessed from the MVME 133A-20 processor across the VME bus. The MVME133A-20 is programmed using a cross-compiler (the Verdix VADS 5.7 Sun-3 Unix to Motorola 68000 Family cross-compiler). The FG-100-V does real-time image processing on the digital frame data for tracking of the ball. Each frame of data is displayed to the video monitor in order to allow a human to observe the movement of the ball.
Figure 1: The Composition of the Hardware System
The low level packages had been set up previously ([6]) so that the work for this project reduced to programing the real-time pinball playing function. The system needs to locate the flippers, since the camera can be installed in different positions. The system then needs to detect when the ball comes near the flippers so that it can use the flippers to hit the ball. It reads each frame of data and compares it to the previous one so that it can recognize the ball by its movement. The system locates the ball, estimates the coordinates of its center, and draws a red cross on that position. Displaying the red cross at the estimated position of the ball plays an important role in allowing an observer to verify that the computer is tracking the pinball. Finally, if the ball is detected in a position to hit, the system activates the flippers to hit the ball.
Video signals are analog signals that are acquired in a fixed format. A video signal is composed of a sequence of images that are transmitted to a viewing monitor at a rate of 30 images (or frames) per second. Accordingly, tracking of the ball and activating the flippers should be done once in each frame time (1/30-sec), or more strictly speaking, half of each frame time (1/60-sec). A frame is made up of two horizontal scan line fields. The even field consists of all of the even-numbered scan lines in the frame, beginning with line zero (the top-most line). The odd field contains all of the odd-numbered scan lines in a frame, beginning with line one. When in full-resolution mode, the FG-100-V acquires and stores an image so that the lines from a pair of successive odd and even fields are merged properly in frame memory. Once the video processor has been turned on, it generates a even and odd vertical retrace interrupts, which are the interrupts that occur when the camera traces back to the origin point (leftmost top) of the frame. In other words, FG-100-V grabs 30 frames per second, and each frame consists of the two (even and odd) sets of frame data. Consequently the main function of the system, which is to keep track of the moving ball and activate the flippers, must be processed in one-sixtieth of a second. Therefore, this system is most naturally organized as a single periodic task, with a period of one sixtieth of a second.