Date: Fri, 11 May 2001 00:35:14 -0700From: paul@pjrc.comTo: *************Subject: Re: Yeah yeah... more questions... :-)I'll probably copy portions of this explaination into some of the docs...>I am really getting frustrated with this code. I hate to be a wussy but are>you planning on porting the whole drivers file to C? I have been >staring at it for at least 3-4 hours today and I am really having a tough >time figuring it out. I looked at the file_cache routine to check how you Those routines will remain in assembly language, and there's reallyno need to do much with them, since they're already working.>were doing that.  I would really like to help out with the file caching >problem but other than advice I am not going to be much good to you. I >can't reverse engineer Assembler code. Rather than wasting your time >explaining stuff to me, I would rather be useful some other way. I would What I meant by a "caching strategy" has nothing to do with thecode that's inside file_cache and file_cache_work.  The cachingstrategy involves how and when the main program decides to callthese functions and what parameters are passed to file_cache.The driver API has been carefully designed to allow the mainprogram to have a lot of control over what gets cached and when.Please take a look at this web page:http://www.pjrc.com/tech/mp3/library.htmlBegin reading the section "File/Directory Reading".  The file_cache,file_cache_work, file_seek, and other functions are intended to givethe application complete control over what data gets cached... so thecaching strategy is completely under the control of the main program.Those functions provide you with a relatively high level of abstraction,so you can think of files as streams of bytes, more or less like usingthe functions in a standard C library.  My goal was to make the APIsimilar to using standard C library functions, but to give much morecontrol over the disk activity, which is what file_cache andfile_cache_work do for you.  Remember, the "read" functions don'tever cause disk activity... they either find the data in the cacheor return an error.  The drivers manage the cached sectors and clustersfor you, more or less like an operating system would do.Very briefly, after you've opened a file, you call file_cache toset up a transfer of bytes.  file_cache only sets up some parametersinside a structure associated with that file (yes, each of the 64possible open files by have a caching operation in progress).  Toactually cause something to be done towards to ultimate goal ofgetting the data read into the cache, you call file_cache_work.  Itwill return one of three things: still working, complete, or ranout of memory.  file_cache_work doesn't even really do the I/Ooperation... it writes into a request queue for the IDE driver andreturns back to the main program as quickly as possible.  As youknow, there's a lot of really complicated things these functionsdo... the idea is that you really don't need to worry about howthey accomplish their job, just like how most C programmers reallydon't dig inside fprintf, fgets, fopen, etc.Today there is a function called do_ide_loop(), which willeventually go away.  Calling this is what actually causes thedisk operations to take place.  Later all disk operations willbe interrupt driven and do_ide_loop will vanish.>really like to help out but unless there is a lot more commenting as to >what's going on or it's in C I am going to be pretty useless to you. Is >there something that I can work on that hasn't been implemented yet that I >can do on the side and you can include later? Or even write C code with The truth is you're a bit early in the game, as there's still a fewproblems in the SDCC-AS31 glue code, but I'm rapidly working thoseout.  Most of the functions work as documented.  There's also a coupleimportant functions that are still missing, the most important onebeing file_uncache, which will allow you to tell the drivers to freea portion of the file.Still, now is a good time to start coming up with a strategy.  Themain program will almost certainly get re-arranged quite a bit.Today it's very simple (and not very smart).  You can see there'sfive states, and the flow is always:S_READ_DIR -> S_CACHING -> S_PLAYING_AND_CACHING ->	S_PLAYING -> S_FINISH_PLAYINGThe S_FINISH_PLAYING state is when the program has sent all of thefile to the drivers for playback.  There is considerable bufferingin the playback driver, so the program has to wait until the driverhas actually played all the buffered data before closing the file.Now, as an example of improving the caching strategy, it wouldmake a lot of sense to call file_uncache on the part of file that'snot being buffered (hmm... is it possible to figure out what partisn't in the playback driver's buffer?)  After the call to file_uncache(damn, I really need to get that written), then there will be lotsof free memory again, so the drive could be started (ide_wakeup) andthe next file could be opened and some of it could be cached.In general, calling file_cache_work end up resulting in calls tomalloc_blocks, and calling file_close calls free_blocks for all theblocks that were allocated.  Someday soon file_uncache will exist,so you can cause free_blocks to be called on only a portion of thefile that you don't need anymore (and you need memory freed up).There's a balancing act between turning the drive on too soon anddraining batteries needlessly, and turning it on too late and nothaving data in memory to play.  This will ultimately require anAPI to some timer drivers.  It's on my to-do list, after fixingbugs and writing file_uncache.This is just one simple thing to do... you can have up to 64 filesopen at once (directories count as files).  You could open up andcache perhaps the first 100k of a few upcoming files, so when/ifthe user presses the next button, you've got some in memory andthey're just a file_seek and file_read/file_read_block away, insteadof making the user wait for the drive to spin up when they press abutton.The API is designed to give the main program complete control ofwhat portion of what files gets cached when.  Calls to file_cache_workcause small increments of the file to get loaded into the cache.It is legal to have many files open and have each one with a pendingfile_cache set of parameters established.  You get to decide when toget each one to read another chunk.  Perhaps 9 times out of 10 youcall file_cache_work for the file that you're playing right now, and1 of 10 times call it for portions upcoming files, in anticipationof the user pressing a button.>Sorry to be such a pain in the ass... Put me to work, just not something so >hard as the file cache... :o)Please please please read this web page:http://www.pjrc.com/tech/mp3/library.htmlSome parts are slightly out of date, so also refer to as31glue.hfor a list of all the API to the drivers.  Once the code is morestable, I'll update this page.  The API is still open and somemore calls will need to be added to accomplish some things, butit's important that you understand what the functions do and howto call them.  That's why I spent quite a bit of time to writethat page documenting the driver API.Paul