Index of /~baker/devices/projects/ryajin

June 2,2006

We discussed with Dr. Kartik after class about background of anemone project and possibilities to improve the cache management policy.

June 4,2006

We had a further discussion about details on how to improve the cache performance. Base on our discussion, we proposed two policies.

1.       Logically separate cache into a smaller write buffer and a read cache. Sizes of them are configurable so that we can evaluate for optimum division.

2.       Replace the current LRU cache replacement policy using LRW-ended read cache replacement policy which we thought more suitable for network swapped storage.

In addition to above two base improvements, we planed to address the following issue if time allows.

1.       Solve page aging problem caused by our new cache replacement policy

2.       Dynamically adjust write buffer/read cache size based on workload

3.       Combine write buffer/read cache into a unified cache

We planed to start reading code and to mark places we should modify as well as places we should write from scratch.

June 5,2006

Reading code

June 6,2006

Reading code

June 8,2006

After some efforts reading anemone code and kernel swap daemon code, we both have questions and new ideas about our project.

On today¡¯s meeting, we begin to finalize design and make programming decisions. We decide to replace current LRU policy to MRW policy and we intend to use workqueue for scheduling network requests from write buffer transfer and for read cache prefetch because we think we need to use sleep for network transfer in these two situations.

We decide to create a circular buffer for write and a cache for read. Particularly, we discussed the implementation of write buffer. There are two ways, array and link list. Considering the memory fragmentation problem caused by frequently allocating/de-allocating pages and the complexity of pointers, we prefer using array to store pages for write. Although this will incur some copy operations from write buffer to read cache, copies will not be frequent because they happen only on initial cache population based on our cache management policy. Another benefit of using circular buffer is locking between reader and writer as show in scullpipe assignment.

Another issue is cache interface. In current implementation, ¡°cache¡± code actually behaves like a page swap manager, which first checks LRU cache. If cache hits, it returns page to upper level, else it issues a network request to fetch page from remote server and send it to upper level. Ideally, cache itself should not care about the network transfer. It just return page if there is one or return null if not so. We are planning to change cache_add and cache_retrieve function into page_put and page_get, inside which they look up cache first and then network and then local disk. And we can focus on cache part and another team (Jian & Kyle) can focus on network and local disk. This requires coordination with them so we may talk with them tomorrow.

We also talk a bit about using time ticking and not swapping cached page to remote server to avoid cache aging problem. We also think about the way to disable swap daemon read ahead (meaning less for anemone because anemone is conceptually a random device so sequential read/write can not gain any benefit).

June 9,2006 -

June 15, 2006

We run the first version with delayed write which schedule a write requests as soon as page write to buffer. After trials and errors, we finally build a stable version but the result is disappointing.

////////////////////////////////////////////////////////////////////////////////////
original
////////////////////////////////////////////////////////////////////////////////////
Outstanding_Reqs: 0
Read Goodput: 13 Mbit/s
Write Goodput: 13 Mbit/s
Retransmits: 0
Spaced: 0
Success: 241079
Packet drops: 0
Retrans drops: 0
Interface Drops: 0
Total Reads: 23442
Total Writes: 217637
Total Mapped Pages: 217634
Avg Read Latency: 4294966824 usec
Avg Write Latency 4294966817 usec
Control Message Bytes: 17844582 Broadcast msg count: 64 data msg count: 241079
Data Bytes : 1023144012   Ctrl/Data rate  2

Cache HIT/MISS
         HIT: 777
        READ: 24220

Running time: 128 secs (2 min)

Server Summary:
 Server 1) d2, avail_mem = 2091MB, 535460 pages, percent= 68, writes= 217637, reads=23442.
 Total Memory: 779006 pages, 3042 MB
 Total Available: 535460 pages, 2091 MB

////////////////////////////////////////////////////////////////////////////////////
Our's
////////////////////////////////////////////////////////////////////////////////////
Outstanding_Reqs: 0
Read Goodput: 52 Mbit/s
Write Goodput: 0 Mbit/s
Retransmits: 0
Spaced: 0
Success: 246119
Packet drops: 0
Retrans drops: 0
Interface Drops: 0
Total Reads: 27472
Total Writes: 218647
Total Mapped Pages: 218647
Avg Read Latency: 4294966818 usec
Avg Write Latency 4294966909 usec
Control Message Bytes: 18217542 Broadcast msg count: 64 data msg count: 246119
Data Bytes : 1044533772   Ctrl/Data rate  0

Cache HIT/MISS
         HIT: 364
        READ: 27797

Running time: 129 secs (2 min)

Server Summary:
 Server 1) d2, avail_mem = 2087MB, 534437 pages, percent= 68, writes= 218647, reads=27472.
 Total Memory: 779006 pages, 3042 MB
 Total Available: 534437 pages, 2087 MB

We find it is not good to build a algorithm just based on our imaging so we add some pieces of code to capture the real trace.

Huge amount of write data occupied the cache most of time so read hit rate suffers. We changed our algorithm to add prefetch when swapper requests to read some page in. In addition, we limit the cache space size used as write buffer.

New performance data: 
Outstanding_Reqs: 0
Read Goodput: 1396 Mbit/s
Write Goodput: 0 Mbit/s
Retransmits: 0
Spaced: 0
Success: 394500

Packet drops: 0
Retrans drops: 0
Interface Drops: 0
Total Reads: 174088
Total Writes: 220412
Total Mapped Pages: 220412
Avg Read Latency: 4294966817 usec
Avg Write Latency 4294966895 usec
Control Message Bytes: 29195664 Broadcast msg count: 36 data msg count: 394500
Data Bytes : 1674260664   Ctrl/Data rate  1
Cache HIT/MISS
         HIT: 33307
        READ: 33655
Running time: 73 secs (1 min)
Server Summary:
 Server 1) d2, avail_mem = 2010MB, 514692 pages, percent= 66, writes= 220412, reads=174088.

 Total Memory: 779006 pages, 3042 MB
 Total Available: 514692 pages, 2010 MB

cache stat graphs

cache management graphs

prefetch graphs