Index of /~baker/devices/projects/pendawchi

	Name	Last modified	Size	Description
	Parent Directory		-
	Makefile	13-Jul-2008 13:09	391
	diff	13-Jul-2008 13:16	175K
	hrt.base.c	13-Jul-2008 13:16	108K
	hrt.c	13-Jul-2008 13:09	87K
	hrt.h	13-Jul-2008 13:09	2.6K
	hrt_load	13-Jul-2008 13:09	1.1K
	hrt_unload	13-Jul-2008 13:09	253
	pendawchi.tgz	17-Jul-2008 13:01	154K
	src/	13-Jul-2008 15:15	-
	test/	13-Jul-2008 13:09	-

• Overview
• A brief log of the project
• What we have accomplished
• Implementation keynotes
• Future works
• Workload distribution
• Driver source code
• Version list
• Others

Overview

The baseline PixelSmart/HRT driver

A brief log of the project

• We talked with professor Baker about the potential projects.
• Two topics - Linux kernel real-time scheduler and PixelSmart/HRT driver were given by the professor.
• Considering the shortage of time in the summer semester and the implementation complexity of the real-time scheduler, we decided to choose the frame grabber driver as our final course project.

• In this week, we tried to understand the exiting driver code. By the end of the week, we could understand the general idea of the driver code and find some errors of the baseline driver.
• We also tried to improve the performance of the driver, however, it turned out that the device-to-kernel data copy time consumes lots of time, which becomes the performance bottleneck of the driver.

• Interrupt handler, kernel timer handler, and tasklet handler were investigated.
• Learn the IOCTL operations specified in the V4L2 API.
• Decide to implement user pointers type of streaming I/O method;

• We read the source code of bttv video driver (bttv-driver.c) and a virtual video driver (vivi.c).
• Decide to use the kernel provided video buffer management interface to implement various I/O methods: read, streaming I/O via memory mapping, and streaming I/O via user pointers.
• Understand the implementation mechanism of IOCTL in the kernel 2.6.25+ .
• Ready to write a new PixelSmart/HRT frame grabber driver.

• Implementing read, ioctl, poll, tasklet handler, timer handler, interrupt handler with video buffer manipulation functions provided in the kernel code.
• Implementing IOCTL operations in a standard way.
• Taking bbtv driver and virtual video driver as reference implementations.

• Debugging the new driver.
• Writing a testing programs.
• Repeatedly testing and improving the new driver.

What we have accomplished

• Greatly improving the driver performance by doubling the data capture rate.
• Modifying the file handler data structure in the driver to support multi-open to a device. For each device open operation a file handler will be created and device can have multiple file handlers.
• Implementing 4 video buffer queue functions: buffer_setup, buffer_prepare, buffer_queue,buffer_release required by the video buffer management mechanism in the kernel source code.
• Replacing I/O method related IOCTL operations using system video buffer queue interfaces. The IOCTL operations are listed as the followings:
• VIDIOC_REQBUFS, VIDIOC_QUERYBUF, VIDIOC_QBUF, VIDIOC_DQBUF, VIDIOC_STREAMON, VIDIOC_STREAMOFF
• Implementing the IOCTL function in by using the following functions and data structures:

 	
	struct video_device  hrt_video_template
	{
	         .fops = &hrt_fops,
	         .vidioc_querycap = hrt_querycap, // each ioctl operation is implemented as a sperate function
	         .vidioc_enum_fmt_cap = hrt_enum_fmt_cap,
	          . . .        
	};

• Implementing mmap, read, poll, and tasklet handler using kernel video buffer queue functions.
• Realizing a correct use of mutex in the new driver.
• Writing a new driver test program.

Implementation keynotes

In the kernel source code the struct videobuf_buffer and videobuf_queue are defined in videobuf-core.h to manage video buffer. As we can see from videobuf_buffer, two list_head type members queue and stream are embedded in it, which are used to link video_buffer to 2 lists: one list called queue links all empty video buffer used in streaming I/O mode; the other list called stream is used in videobuf_queue to link all video buffers created in streaming I/O mode. Typically, in the video driver (whatever the video type is) a device specific video buffer is created and the first member of this specific video buffer is of videobuf_buffer type. Since file handler is introduced in the video device driver, whenever the driver perform an open operation an instance of file handler structure will created, in which it contains a member of videobuf_queue type. Not all I/O methods will use videobuf_queue, read operation is such a case, the file handler only need to provide a videobuf_buffer pointer in the read case.

The device will eventually fill the video buffers either created via memory mapping or passed through user pointers or read operations. Therefore, in the device driver the device specific structure will contain a member of list_head type that links all the available free video buffers through queue list as we mentioned in the above paragraph. Once the data generated by the device finish dumping into a video buffer, the filled buffer will be removed from the queue list, but still in the stream list and driver will also wakeup corresponding sleep processes waiting for data. Once user applications get the data, this buffer will be finally removed from the stream list.

struct videobuf_buffer {
        unsigned int            i;
        u32                     magic;

        /* info about the buffer */
        unsigned int            width;
        unsigned int            height;
        unsigned int            bytesperline; /* use only if != 0 */
        unsigned long           size;
        unsigned int            input;
        enum v4l2_field         field;
        enum videobuf_state     state;
        struct list_head        stream;  /* QBUF/DQBUF list */

        /* touched by irq handler */
        struct list_head        queue;
        wait_queue_head_t       done;
        unsigned int            field_count;
        struct timeval          ts;

        /* Memory type */
        enum v4l2_memory        memory;

        /* buffer size */
        size_t                  bsize;

        /* buffer offset (mmap + overlay) */
        size_t                  boff;

        /* buffer addr (userland ptr!) */
        unsigned long           baddr;

        /* for mmap'ed buffers */
        struct videobuf_mapping *map;

        /* Private pointer to allow specific methods to store their data */
        int                     privsize;
        void                    *priv;
};
struct videobuf_queue {
        struct mutex               vb_lock;
        spinlock_t                 *irqlock;
        void                       *dev; /* on pci, points to struct pci_dev */

        enum v4l2_buf_type         type;
        unsigned int               inputs; /* for V4L2_BUF_FLAG_INPUT */
        unsigned int               msize;
        enum v4l2_field            field;
        enum v4l2_field            last;   /* for field=V4L2_FIELD_ALTERNATE */
        struct videobuf_buffer     *bufs[VIDEO_MAX_FRAME];
        struct videobuf_queue_ops  *ops;
        struct videobuf_qtype_ops  *int_ops;

        unsigned int               streaming:1;
        unsigned int               reading:1;
        unsigned int               is_mmapped:1;

        /* capture via mmap() + ioctl(QBUF/DQBUF) */
        struct list_head           stream;

        /* capture via read() */
        unsigned int               read_off;
        struct videobuf_buffer     *read_buf;

        /* driver private data */
        void                       *priv_data;
};

Because the PixelSmart/HRT Frame Grabber cards used in the project do not support DMA operations, it is necessary to use no-DMA version video buffer support functions defined in videobuf-vmalloc.c. Another interesting issue raised by non-DMA support is the difficulty to directly use the user space based video buffers in user pointers streaming I/O mode. This is because the device could not directly copy data from device space into user space for the lack of user process context information. We solved this problem by creating a transitional video buffer in the kernel space and first copy device data to this transitional buffer, then copy data to the user provided buffer. The shortcoming of this solution is a potential performance loss, however, since the bottleneck of the driver performance is copying data from the device to kernel space buffer, this solution will not influence the performance too much and our testing results also confirm this conclusion.

In the old driver, the operation of copying data from device buffer to kernel buffer is accomplished by using memcpy and additional step is taken to empty device buffer by using memset. However, under the current kernel 2.6.25+, the memcpy is modified to transfer only one byte each time instead of four bytes in the older kernel. As a result, the performance of this device-to-kernel copying suffers from this modification. On the other side, we find out that the additional step in the old driver to empty the device buffer is unnecessary. Therefore, in the new driver implementation, we remove memset operations, which leads to a huge performance improvement (the actual performance gaining is given in "What we have accomplished" section).

Workload distribution

• Peng Chen and Chi Zhang are responsible for the standardization of video buffer management and driver optimizations.
• Dawei Su is responsible for the standardization of IOCTL operations.

Source code

• The source code of our new driver implementation can be found here.
• The source code of base line driver can be found here.
• The new test program can be found here.
• The old test program can be found here.
• A diff file between the new driver and the baseline driver can be found here.
• The whole source code tar ball can be found here.

Version history

• July 12th, 2008 - First draft by Chi Zhang
• July 14th, 2008 - Revision by Chi Zhang, Peng chen, Dawei su
• July 15th, 2008 - Final by Chi Zhang

Others

• If you have any questions about this project, you can contact us by writing to czhang@cs.fsu.edu.