Author Topic: Usage of GFS_NwFread (you probably expected this...)  (Read 734 times)

ponut64

  • Full Member
  • ***
  • Posts: 175
  • Karma: +13/-0
    • View Profile
Re: Usage of GFS_NwFread (you probably expected this...)
« Reply #15 on: June 08, 2018, 02:26:41 am »
Funny I should go on double (now triple) posting like that. I found an arrangement with a GOTO statement that can pass over but safely complete the sluggish parts of the loop IF the function is continuously called. This is an important step in the right direction.

Code: [Select]
	Uint8 rd_frames = 0;
Uint8 curRdFrame = 0;

 //Experimental
void * ztFileStream(Sint8 * filename, void * startAddress, entity_t * model, Sint8 evenStep, Sint8 oddStep)
{
    memset_l((void*)startAddress,0x0000, (0xFFFE - ((Uint32)startAddress-(Uint32)LWRAM)));  //Not 100% necessary, since data can just be overwritten, but good for testing and see how much data a level takes
    void * currentAddress;
    currentAddress = startAddress;
//Data of read structure
    Sint32 fid = GFS_NameToId((Sint8*)filename);
GfsHn gfs;
Sint32 nsct;
Sint32 fsize;
//Accepted rate of reading per frame (33 ms)
//Note to self: If your data is already byte size information, do not annotate with sizeOf. Thanks.
Uint32 rt_step = 4096;
Uint8 rt_sector = rt_step / 2048;
//Data of read-in-process
Sint32 stat;
Uint32 rdsize;
//Model Data is the destination of data in the work area.
modelData_t bufModel;
//Destination address?
    void * ptr = &bufModel;
//Loading follows
gfs = GFS_Open(fid);
//Get sectors
//HEY! SBL DOCUMENTATION IS WRONG! THIRD ITEM NSCT IS GFS SECTOR COUNT. SECOND ITEM IS CD SECTOR SIZE.
GFS_GetFileSize(gfs, NULL, &nsct, NULL);
GFS_GetFileInfo(gfs, NULL, NULL, &fsize, NULL);
//How many frames are we reading?
if(rt_step < fsize){
rd_frames = (fsize + (rt_step - 1))/(rt_step);
}



jo_printf(0, 8, "(%i)", nsct);
jo_printf(10, 8, "(sectors)");
jo_printf(0, 9, "(%i)", fsize);
jo_printf(10, 9, "(file size)");


//Read from CD to CD buffer
//CONCEPT: NwFread specifies only a fetch from CD buffer. If no such read has been specified, it will carry it out itself.
//However, if you specify a read from CD, repeats of the fetch command will only fetch from the CD buffer and will not give errors for data remainders.
//Now I understand how the program on SBL Document 1 Page 18 works. It is incomplete; they assume the program is a system callback, but it works.


GFS_SetTmode(gfs, GFS_TMODE_CPU);
GFS_SetGmode(gfs, GFS_GMODE_ERASE);
GFS_SetReadPara(gfs, rt_step);
GFS_SetTransPara(gfs, rt_sector);
if(curRdFrame >= rd_frames){
goto RUNARC;
}
GFS_NwCdRead(gfs, fsize);
for( ; curRdFrame <= rd_frames ; ){
GFS_NwFread(gfs, rt_sector, (Uint32*)(currentAddress +(curRdFrame * rt_step)), rt_step);
do{
GFS_NwExecServer(gfs);
GFS_NwGetStat(gfs, &stat, &rdsize);
jo_printf(0, 15, "(%i)", curRdFrame);
jo_printf(4, 15, "(%i)", rd_frames);
}while (stat != GFS_SVR_COMPLETED && rdsize < rt_step);
curRdFrame++;
}
 jo_printf(0, 11, "(%i)", rdsize);
jo_printf(10, 11, "(fetched filesize)");

GFS_Close(gfs);

// Copy gfs to modelData_t bufmodel???
memcpy_l((Sint32*)ptr, (Sint32*)(currentAddress), sizeof(modelData_t));

    //ADDED
    model->pos[X]=bufModel.Origin[X]; model->pos[Y]=bufModel.Origin[Y]; model->pos[Z]=bufModel.Origin[Z];
    model->length[X]=bufModel.Length[X]; model->length[Y]=bufModel.Length[Y]; model->length[Z]=bufModel.Length[Z];
    model->nbMeshes=bufModel.TOTAL_MESH;

Uint16 first_texture = loadTextures(currentAddress, &bufModel);
Sint32 bytesOff = bufModel.TEXT_SIZE+(sizeof(modelData_t));
currentAddress = (void*)(currentAddress + bytesOff);
currentAddress = loadPDATA((void*)(currentAddress), model, &bufModel);
currentAddress = (void*)(currentAddress + (bufModel.PDATA_SIZE + sizeof(modelData_t)));
setTextures(first_texture, model, bufModel.TOTAL_MESH, true);

RUNARC:

jo_printf(0, 13, "(%i)", bufModel.TOTAL_MESH);
jo_printf(3, 13, "(%i)", bufModel.PDATA_SIZE);
jo_printf(10, 13, "(%i)", model->nbMeshes);

    return currentAddress;

}


ponut64

  • Full Member
  • ***
  • Posts: 175
  • Karma: +13/-0
    • View Profile
Re: Usage of GFS_NwFread (you probably expected this...)
« Reply #16 on: June 11, 2018, 11:57:55 am »
An update.

Unlike previous incantations here, this one somewhat accomplishes the goals I have set out for.
I mean it does read from the CD in a non-linear fashion and puts it in the work area.
There's two problems though.
- It is painfully slow. If you think it's bad in an emulator, try it on real hardware. This structure of code is just untenable. I need a different approach.
- The data is corrupted somehow. I would like to solve that before I moved on to something else, but there's clearly no reason to so I am going to move on to a different command structure.

I have also identified a few things, such as GFS commands being a significant contributor to slowdown, although the primary slow-down on real hardware is going to be any loading loop as the CD system is not lightning fast (obviously, its a CD).

Code: [Select]
volatile	Uint32 rd_frames = 0;
volatile Uint32 curRdFrame = 0;

bool file_reading;
bool TimeToSet = false;

 //Experimental
void * ztFileStream(Sint8 * filename, void * startAddress, entity_t * model, Sint8 evenStep, Sint8 oddStep)
{
    void * currentAddress;
    currentAddress = startAddress;

//Data of read structure
    Sint32 fid = GFS_NameToId((Sint8*)filename);
GfsHn gfsx;
Sint32 nsct;
Sint32 fsize;
//Accepted rate of reading per frame (33 ms)
//Note to self: If your data is already byte size information, do not annotate with sizeOf. Thanks.
Sint32 rt_step = 2048;
Uint8 rt_sector = rt_step / 2048;
    Uint8 synctime;
//Data of read-in-process
Sint32 stat;
Sint32 rdsize;
//Model Data is the destination of data in the work area.
modelData_t bufModelX;
//Destination address?
    void * ptr2 = &bufModelX;

if(gfsx == NULL && file_reading != true){
//Loading follows
gfsx = GFS_Open(fid);
//Get sectors
//HEY! SBL DOCUMENTATION IS WRONG! THIRD ITEM NSCT IS gfsx SECTOR COUNT. SECOND ITEM IS CD SECTOR SIZE.
GFS_GetFileSize(gfsx, NULL, &nsct, NULL);
GFS_GetFileInfo(gfsx, NULL, NULL, &fsize, NULL);
//How many frames are we reading?
if(rt_step < fsize){
rd_frames = (fsize + (rt_step - 1))/(rt_step);
}
}

//Skip the loop if it is finished
if(curRdFrame >= rd_frames){
if(TimeToSet != true){
  // Copy gfsx payload to modelData_t bufmodel
memcpy_l((Sint32*)ptr2, (Sint32*)(currentAddress), sizeof(modelData_t));
//ADDED
    model->pos[X]=bufModelX.Origin[X]; model->pos[Y]=bufModelX.Origin[Y]; model->pos[Z]=bufModelX.Origin[Z];
    model->length[X]=bufModelX.Length[X]; model->length[Y]=bufModelX.Length[Y]; model->length[Z]=bufModelX.Length[Z];
    model->nbMeshes=bufModelX.TOTAL_MESH;

Uint16 first_texture = loadTextures(currentAddress, &bufModelX);
Sint32 bytesOff = bufModelX.TEXT_SIZE+(sizeof(modelData_t));
currentAddress = (void*)(currentAddress + bytesOff);
currentAddress = loadPDATA((void*)(currentAddress), model, &bufModelX);
currentAddress = (void*)(currentAddress + (bufModelX.PDATA_SIZE + sizeof(modelData_t)));
setTextures(first_texture, model, bufModelX.TOTAL_MESH, false);
TimeToSet = true;
}

file_reading = true;
jo_printf(0, 9, "(skipped loop)");
goto SKIP;
}

jo_printf(0, 8, "(%i)", nsct);
jo_printf(10, 8, "(sectors)");
jo_printf(0, 9, "(%i)", fsize);
jo_printf(10, 9, "(file size)");

//Read from CD to CD buffer
//CONCEPT: NwFread specifies only a fetch from CD buffer. If no such read has been specified, it will carry it out itself.
//However, if you specify a read from CD, repeats of the fetch command will only fetch from the CD buffer and will not give errors for data remainders.
//Now I understand how the program on SBL Document 1 Page 18 works. It is incomplete; they assume the program is a system callback, but it works.

//TIP: Since TMODE_CPU is the SH1 of the CD system, there is no reason to use anything else, as that CPU is otherwise entirely idle.
GFS_SetTmode(gfsx, GFS_TMODE_CPU);
//TIP: Setting GFS_GMODE_RESIDENT is a guaranteed crash.
GFS_SetGmode(gfsx, GFS_GMODE_ERASE);
GFS_SetReadPara(gfsx, rt_step);
GFS_SetTransPara(gfsx, rt_sector);
//TIP: GFS_Nw_CdRead cannot be checked for completion. Sorry!
GFS_NwCdRead(gfsx, fsize);
PUSHF:
    synctime = jo_get_ticks();
jo_printf(0, 2, "x(%i)x", synctime);
GFS_NwFread(gfsx, rt_sector, (Uint32*)(currentAddress +(curRdFrame * rt_step)), rt_step);
READING:
GFS_NwExecOne(gfsx);
GFS_NwGetStat(gfsx, &stat, &rdsize);
jo_printf(0, 11, "(%i)", rdsize);
jo_printf(10, 11, "(fetched filesize)");
jo_printf(0, 15, "(%i)", curRdFrame);
jo_printf(4, 15, "(%i)", rd_frames);
if(stat != GFS_SVR_COMPLETED && rdsize < rt_step && synctime & 2){
goto READING;
}
//Remainder of file in comparison to rt_step [%] is handled, because we do not want to repeat the loop for the end of the data anyway. Do you understand?
//Technically, a file exactly divisible by rt_step will read-loop indefinitely. Watch out for that.
GFS_NwStop(gfsx);
if(rdsize == rt_step && synctime & 2 ){
curRdFrame = curRdFrame + 1;
goto PUSHF;
}
GFS_Close(gfsx);
SKIP:

jo_printf(0, 13, "(%i)", bufModelX.TOTAL_MESH);
jo_printf(3, 13, "(%i)", bufModelX.PDATA_SIZE);
jo_printf(10, 13, "(%i)", model->nbMeshes);

    return currentAddress;
}

ponut64

  • Full Member
  • ***
  • Posts: 175
  • Karma: +13/-0
    • View Profile
Re: Usage of GFS_NwFread (you probably expected this...)
« Reply #17 on: June 12, 2018, 06:45:47 pm »
After skimming the SBL documentation, it seems I had the wrong idea all along.

Quote
Immediate-Return  Read
Reads data from files by means of a request function and a server function. The
request issued by the request function is processed by the server function one pro-
cessing unit at a time.  The server function must be recalled repeatedly. By inserting
application processing into the server function call loop, execution of the application
can be continued until completion of the data read.

Quote
5.2  Immediate-Return Access
Immediate-return access is performed by using a request function and a server
function. The request function executes only request acceptance and immediately
returns. Actual access is performed by repeatedly calling the server function while
monitoring the completion status. The application can also be processed in the call
loop of the server function.


A file handle that has issued a request cannot issue another request until processing
of the first one is completed.

In essence, the "program", in other words (I am assuming) ALL SYSTEM CALLBACKS must be put inside the GFS_NwFread loop in order for it to perform whilst the rest of the program is uninterrupted. I assume you DO NOT put it inside the NwExecOne loop (because no sample code does anything inside of it besides check on it). In essence that's a major restructure of jo_main. I will try it someday.

I am still a little confused about how NwCdRead is used but it doesn't seem to cause crashes if or if not present.
« Last Edit: June 12, 2018, 06:48:41 pm by ponut64 »

XL2

  • Sr. Member
  • ****
  • Posts: 341
  • Karma: +72/-1
    • View Profile
Re: Usage of GFS_NwFread (you probably expected this...)
« Reply #18 on: June 12, 2018, 07:40:40 pm »
You don't need to change the Jo main implementation, just use a callback and just fetch a small amount of data for each loop.

ponut64

  • Full Member
  • ***
  • Posts: 175
  • Karma: +13/-0
    • View Profile
Re: Usage of GFS_NwFread (you probably expected this...)
« Reply #19 on: June 14, 2018, 12:50:21 pm »
Okay, that's what I thought I should do at first since that's how Jo engine does it.

Now I am at a similar problem to before: the read data is getting corrupted if I interrupt the NwFread loop (and thus run the program without interruption).
I do happen to know that the Saturn should be able to manage about 240 bytes fetched each frame, and if I use that for rt_step, it will just crash (im assuming because the data is corrupted).
I don't know why the data is getting corrupted...

Code: [Select]

typedef struct
{
bool file_done;
void * dstAddress;
entity_t * tmodel;
Sint8 * filename;
} request;

Uint32 rd_frames = 0;
Uint32 curRdFrame = 0;

Uint16 FilRqNm;

bool TimeToSet = false;
bool process_requested;

request requests[4];

void ztFileRequest(Sint8 * name, void * startAddress, entity_t * model, Uint16 rqNum)
{
if(requests[rqNum].file_done == true){
process_requested = false;
goto SKIP;
}
requests[rqNum].filename = name;
requests[rqNum].dstAddress = startAddress;
requests[rqNum].tmodel = model;
FilRqNm = rqNum;
process_requested = true;
jo_printf(0, 0, "(started)");
SKIP:
if(process_requested == false){
jo_printf(0, 0, "(-ended-)");
}
}

 //Experimental
void ztFileServer(void)
{
if(process_requested != true){
goto SKIP;
}
    void * currentAddress;
currentAddress = requests[FilRqNm].dstAddress;

//Data of read structure
    Sint32 fid = GFS_NameToId((Sint8*)requests[FilRqNm].filename);
GfsHn gfsx;
Sint32 nsct;
Sint32 fsize;
//Accepted rate of reading per frame (33 ms)
//Note to self: If your data is already byte size information, do not annotate with sizeOf. Thanks.
Sint32 rt_step = 2048;
Uint8 rt_sector = 1;
//Data of read-in-process
Sint32 stat;
Sint32 rdsize;
bool gfs_Active = false;
//Model Data is the destination of data in the work area.
modelData_t bufModelX;
//Destination address?
    void * ptr2 = &bufModelX;

if(gfsx == NULL && requests[FilRqNm].file_done != true){
//Loading follows
gfsx = GFS_Open(fid);
gfs_Active = true;
//Get sectors
//HEY! SBL DOCUMENTATION IS WRONG! THIRD ITEM NSCT IS GFS SECTOR COUNT. SECOND ITEM IS CD SECTOR SIZE.
GFS_GetFileSize(gfsx, NULL, &nsct, NULL);
GFS_GetFileInfo(gfsx, NULL, NULL, &fsize, NULL);
//How many frames are we reading?
if(rt_step < fsize){
rd_frames = (fsize + (rt_step - 1))/(rt_step);
}
}
//Skip the loop if it is finished
if(curRdFrame >= rd_frames){
if(requests[FilRqNm].file_done != true){
  // Copy gfsx payload to modelData_t bufmodel
memcpy_l((Sint32*)ptr2, (Sint32*)(currentAddress), sizeof(modelData_t));
//ADDED
    requests[FilRqNm].tmodel->pos[X]=bufModelX.Origin[X]; requests[FilRqNm].tmodel->pos[Y]=bufModelX.Origin[Y]; requests[FilRqNm].tmodel->pos[Z]=bufModelX.Origin[Z];
    requests[FilRqNm].tmodel->length[X]=bufModelX.Length[X]; requests[FilRqNm].tmodel->length[Y]=bufModelX.Length[Y]; requests[FilRqNm].tmodel->length[Z]=bufModelX.Length[Z];
    requests[FilRqNm].tmodel->nbMeshes=bufModelX.TOTAL_MESH;

Uint16 first_texture = loadTextures(currentAddress, &bufModelX);
Sint32 bytesOff = bufModelX.TEXT_SIZE+(sizeof(modelData_t));
currentAddress = (void*)(currentAddress + bytesOff);
currentAddress = loadPDATA((void*)(currentAddress), requests[FilRqNm].tmodel, &bufModelX);
currentAddress = (void*)(currentAddress + (bufModelX.PDATA_SIZE + sizeof(modelData_t)));
setTextures(first_texture, requests[FilRqNm].tmodel, bufModelX.TOTAL_MESH, false);
}
requests[FilRqNm].file_done = true;
jo_printf(0, 9, "(skipped loop)");
goto SKIP;
}

jo_printf(0, 8, "(%i)", nsct);
jo_printf(10, 8, "(sectors)");
jo_printf(0, 9, "(%i)", fsize);
jo_printf(10, 9, "(file size)");

//Read from CD to CD buffer
//CONCEPT: NwFread specifies only a fetch from CD buffer. If no such read has been specified, it will carry it out itself.
//However, if you specify a read from CD, repeats of the fetch command will only fetch from the CD buffer and will not give errors for data remainders.
//TIP: Since TMODE_CPU is the SH1 of the CD system, there is no reason to use anything else, as that CPU is otherwise entirely idle.
GFS_SetTmode(gfsx, GFS_TMODE_CPU);
//TIP: Setting GFS_GMODE_RESIDENT is a guaranteed crash.
GFS_SetGmode(gfsx, GFS_GMODE_ERASE);
GFS_SetReadPara(gfsx, rt_step);
GFS_SetTransPara(gfsx, rt_sector);
//TIP: GFS_Nw_CdRead cannot be checked for completion. Sorry!
GFS_NwCdRead(gfsx, fsize);
PUSHF:
GFS_NwFread(gfsx, rt_sector, (Uint32*)(currentAddress +(curRdFrame * rt_step)), rt_step);
do{
GFS_NwExecOne(gfsx);
GFS_NwGetStat(gfsx, &stat, &rdsize);
jo_printf(0, 11, "(%i)", rdsize);
jo_printf(10, 11, "(fetched filesize)");
}while(stat != GFS_SVR_COMPLETED && rdsize < rt_step);
//Remainder of file in comparison to rt_step [%] is handled, because we do not want to repeat the loop for the end of the data anyway. Do you understand?
//Technically, a file exactly divisible by rt_step will read-loop indefinitely. Watch out for that.
if(rdsize == rt_step){
// GFS_NwStop(gfsx);
curRdFrame = curRdFrame + 1;
// goto PUSHF;
}

if(gfs_Active == true){
GFS_Close(gfsx);
gfs_Active = false;
}

SKIP:

jo_printf(0, 15, "(%i)", curRdFrame);
jo_printf(4, 15, "(%i)", rd_frames);
jo_printf(0, 13, "(%i)", bufModelX.TOTAL_MESH);
jo_printf(3, 13, "(%i)", bufModelX.PDATA_SIZE);
jo_printf(10, 13, "(%i)", requests[FilRqNm].tmodel->nbMeshes);
}

ponut64

  • Full Member
  • ***
  • Posts: 175
  • Karma: +13/-0
    • View Profile
Re: Usage of GFS_NwFread (you probably expected this...)
« Reply #20 on: June 16, 2018, 05:34:17 pm »
Hold up.

With a loop like this...

Quote
   PUSHF:
   GFS_NwFread(gfsx, rt_sector, (Uint32*)(currentAddress +(curRdFrame * rt_step)), rt_step);
do{
      GFS_NwExecOne(gfsx);
      GFS_NwGetStat(gfsx, &stat, &rdsize);
      jo_printf(0, 11, "(%i)", rdsize);
      jo_printf(10, 11, "(fetched filesize)");
}while(stat != GFS_SVR_COMPLETED && rdsize < rt_step);
   //Remainder of file in comparison to rt_step [%] is handled, because we do not want to repeat the loop for the end of the data anyway. Do you understand?
   //Technically, a file exactly divisible by rt_step will read-loop indefinitely. Watch out for that.
   if(rdsize == rt_step){
//   GFS_NwStop(gfsx);
   curRdFrame = curRdFrame + 1;
//      goto PUSHF;
   }

When the program gets to NwFread the second time, it does not seem to register it differently than the first time.
I make this conclusion separate from NwExecOne (or Server) since the loop functions as intended if the goto PUSHF line is enabled, which involves majorly GFS_NwFread.
I also am able to make this conclusion because the program's rdsize doesn't properly get to the last sector size of the file.

I guess I am back to square one. GFS_NwFread is not behaving as expected. If I put it in a loop, the entire program halts whilst inside that loop, even if the function itself is a system callback.
If it is a part of the main program loop, it doesn't behave as expected; it doesn't seem to be picked up more than once.

I always thought at first this is related to GFS_Open and GFS_Close. There's a few problems with that...
If one or the other is missing from each frame, the program will freeze.

So let's look at the NwExecOne loop again. GFS_SVR_COMPLETED can't be causing a problem since this being a do...while loop, it will always hit the GFS_NwGetStat function first, which will set GFS_SVR_BUSY. rdsize being less than RT step.. again, one would think this couldn't cause any problems since rdsize is referenced and should be filled at zero in GFS_NwGetStat. And upon closer examination, rdsize becomes NULL after an incantation of the loop anyway.

So I would say I have identified the problem. It is between GFS_NwFread and GFS_NwExecOne/Server. My guess is NwFread being hit on the second frame is not being registered as a unique command so NwExecOne does not change its status. I can't skip to it on the next frame since GFS_Open needs to happen.

Now, of course, if I put the rest of the game code inside this loop (before goto PUSHF), it would work. I understand what SBL documentation means now. If I go by the documentation, this actually shouldn't work; they never set up an example of the loop being allowed to exit. My assumption is there is no other way for GFS_NwFread to work properly except between a single GFS Open and GFS Close. GFS cannot be closed or opened between incantations of it on the same file, which makes no logical sense but that really appears to be how it works.

My question becomes, is there any workaround...? Well I have been exploring a lot of potential ones, no dice so far...
Not even a unique GfsHn for each frame read works.
Jo engine "async" reading really does not appear to be async reading either. The program is still going to get "stuck" in the file access loop. In an emulator, it's very zippy. On real hardware however, I can't see it behaving as hoped.

I will get back at this tomorrow. Or maybe.. you know some time soon. Restructuring jo_main should be entirely unncessary for this. The loading function just needs to be the preeminent code of a game_frame or game_loop function, like I have in my code. Not that big of a deal now that I think about it.

/e: Just tested it real quick. It DOES work as intended if I put game_frame inside the reading loop. It goes mental (as in frame-rate goes REALLY high because I am sloppily calling the frame code twice), but it works :)
« Last Edit: June 16, 2018, 06:23:50 pm by ponut64 »

ponut64

  • Full Member
  • ***
  • Posts: 175
  • Karma: +13/-0
    • View Profile
Re: Usage of GFS_NwFread (you probably expected this...)
« Reply #21 on: June 17, 2018, 02:51:08 pm »
This thread has reached the conclusion -

https://youtu.be/bkaD8isNVz0

As noted in my comment, any remaining slowdown is due to directory change functions. I did enough debugging to know it isn't because of NwCdRead.
NwCdRead really just makes things faster, no reason not to use it (so far, anyway).

XL2

  • Sr. Member
  • ****
  • Posts: 341
  • Karma: +72/-1
    • View Profile
Re: Usage of GFS_NwFread (you probably expected this...)
« Reply #22 on: June 17, 2018, 03:55:20 pm »
How big is your model?
Also, I'm not 100% sure what it does.
Is it streaming animation data?
If so, why does it disappear?
I think it would be more interesting to test it with backgrounds (or palettes), you could have animated backgrounds.
Is it running at 60 fps?
So you can fetch 512 bytes per frame at 60 fps, I guess streaming visibility data would work fine.
But, first things first : getting midi music to work!

ponut64

  • Full Member
  • ***
  • Posts: 175
  • Karma: +13/-0
    • View Profile
Re: Usage of GFS_NwFread (you probably expected this...)
« Reply #23 on: June 17, 2018, 05:00:30 pm »
It's just streaming / "loading" an entire .ZTP / binary model file. An outdated .ZTP file from the 0.05 model loading demo you put up.

Yeah I know I am not very clear on it. It disappears because my code moves to displaying a different model (that is not loaded). If I loaded everything, nothing would disappear.
These models are about 12KB. All it is streaming is that model. There's no way that can be done for animations using entirely new models, it would just take too long.

Perhaps when I get to your new model loading demo, I could figure something else out but you have the memory usage trimmed down so well. I imagine using this for streaming meshes between levels that are built from smaller meshes. Let's say if you had two very similar levels, but with a few unique assets that can't fit into memory for one or the other, you could load that through a "transitional area" wherein neither of those meshes need to be displayed. An important thing about this is it does not error if the mesh is loaded whilst the game is "trying" to display it.
« Last Edit: June 17, 2018, 05:05:40 pm by ponut64 »

XL2

  • Sr. Member
  • ****
  • Posts: 341
  • Karma: +72/-1
    • View Profile
Re: Usage of GFS_NwFread (you probably expected this...)
« Reply #24 on: June 17, 2018, 05:03:33 pm »
Ok, so you reload the model each frame?
Yeah, for animation you can just use the newer version.
It can be improved and in fact I already made improvements for Sonic Z-Treme.
I might revisit the tool and demo in the near future.

ponut64

  • Full Member
  • ***
  • Posts: 175
  • Karma: +13/-0
    • View Profile
Re: Usage of GFS_NwFread (you probably expected this...)
« Reply #25 on: June 17, 2018, 05:09:22 pm »
Here, this is the loading structure.
*30 FPS, by the way. My calculation of roughly 240 bytes per frame must have been at 60 FPS. It is important to know this is an extrapolation based on CDDA quality. ... it's probably wrong.

Code: [Select]
void	load_test()
{

//Hesitant to add code here...
if(accelState > 0){
load_this++;
}
 jo_printf(20, 0, "(%i)", load_this);
if(load_this < 1){
//This mesh is 216 quads
currentAddress = ztFileRequest("BIN",(Sint8*)"BPONY.ZTP", currentAddress, &entities[0], 0);
}
//Left turn steps
if(load_this > 60 && load_this < 119){
currentAddress = ztFileRequest("LTURN", (Sint8*)"LTURN01.ZTP", currentAddress, &entities[11], 1);
}
if(load_this > 120 && load_this < 179){
currentAddress = ztFileRequest("LTURN", (Sint8*)"LTURN02.ZTP", currentAddress, &entities[12], 2);
}
if(load_this > 180 && load_this < 239){
currentAddress = ztFileRequest("LTURN", (Sint8*)"LTURN03.ZTP", currentAddress, &entities[13], 3);
}
if(load_this > 240 && load_this < 379){
currentAddress = ztFileRequest("LTURN", (Sint8*)"LTURN04.ZTP", currentAddress, &entities[14], 4);
}
if(load_this > 380 && load_this < 479){
currentAddress = ztFileRequest("LTURN", (Sint8*)"LTURN05.ZTP", currentAddress, &entities[15], 5);
}
if(load_this > 540 && load_this < 599){
currentAddress = ztFileRequest("LTURN", (Sint8*)"LTURN06.ZTP", currentAddress, &entities[16], 6);
}
if(load_this > 660 && load_this < 729){
currentAddress = ztFileRequest("LTURN", (Sint8*)"LTURN07.ZTP", currentAddress, &entities[17], 7);
}
if(load_this > 780 && load_this < 839){
currentAddress = ztFileRequest("LTURN", (Sint8*)"LTURN08.ZTP", currentAddress, &entities[18], 8);
}
if(load_this > 900 && load_this < 959){
currentAddress = ztFileRequest("LTURN", (Sint8*)"LTURN09.ZTP", currentAddress, &entities[19], 9);
}
if(load_this > 999 && load_this < 1100){
currentAddress = ztFileRequest("LTURN", (Sint8*)"LTURN10.ZTP", currentAddress, &entities[20], 10);
}

}

Very simple. In a given range of numbers, the model loading code is active. I don't really need to do it that way, but I do.

This is the file request function.

Code: [Select]
void * ztFileRequest(char * dirname, Sint8 * name, void * startAddress, entity_t * model, Uint16 rqNum)
{
FilRqNm = rqNum;
if(requests[FilRqNm].file_done == true){
process_requested = false;
jo_printf(0, 0, "(-ended-)");
goto SKIP;
}
if(process_requested == false){
rd_frames = 0;
curRdFrame = 0;
ztCDsetRoot();
}
ztCDsetDir(dirname);
requests[FilRqNm].filename = GFS_NameToId(name);
requests[FilRqNm].dstAddress = startAddress;
requests[FilRqNm].tmodel = model;
process_requested = true;
jo_printf(0, 0, "(started)");
SKIP:
return requests[FilRqNm].dstAddress;
}

And this explains why the model is sometimes not visible. This was my animation system for the old ZTPs using a unique mesh for each frame.

Code: [Select]
void	anim_entity_state(Uint16 start, Uint8 length, Sint8 sync, Bool loop, Bool reverse)
{
if(loop == true && reverse == false){
if(anim_target < length && sync > 0){
anim_target++;
}
if(anim_target == length){
anim_target -= length;
}
display_ztp(&entities[start+anim_target]);
}

if(loop == true && reverse == true){
if(anim_target < length && sync > 0){
anim_target++;
}
if(anim_target == length){
anim_target -= length;
}
display_ztp(&entities[start+length-anim_target]);
}

if(loop == false && reverse == false){
if(anim_target < length && sync > 0){
anim_target++;
}
if(anim_target == length){
display_ztp(&entities[start]);
}
display_ztp(&entities[start+anim_target]);
}

if(loop == false && reverse == true){
if(anim_target < length && sync > 0){
anim_target++;
}
if(anim_target == length){
display_ztp(&entities[start]);
}
display_ztp(&entities[start+length-anim_target]);
}

}

Code: [Select]
		if(accelState == 0 && rotState[X] == 0){
anim_target = 0;
display_ztp(&entities[0]);
}
if(accelState > 0){
anim_entity_state(21, 19, evenFrame, true, false);
} else if(accelState < 0){
anim_entity_state(21, 19, evenFrame, true, true);
} else if(rotState[X] < 0){
anim_entity_state(1, 9, evenFrame, true, false);
} else if(rotState[X] > 0){
anim_entity_state(11, 9, evenFrame, true, false);
}

I'm wordy but I like to leave little up to question.
Once the mesh is finished loading, no more loading is done with it.
It is important to note that if you wanted to use that entity slot for a different mesh, you can just load to that mesh address (&entitites[number]) again and it will be replaced.

And finally, the core loading loop. Note that there is a void function parameter in the function This is where you put the game loop.

Code: [Select]
 //Experimental
void ztFileServer(void(*sysframe)(void))
{
if(process_requested != true){
goto SKIP;
}
    void * workAddress;
workAddress = requests[FilRqNm].dstAddress;

//Data of read structure
    Sint32 fid = (Sint32)requests[FilRqNm].filename;
GfsHn gfsx;
Sint32 nzect;
Sint32 fsize;
//Accepted rate of reading per frame (33 ms)
//Note to self: If your data is already byte size information, do not annotate with sizeOf. Thanks.
Sint32 rt_step = 512;
Uint8 rt_sector = 1;
//Data of read-in-process
Sint32 stat;
Sint32 rdsize;
//Model Data is the destination of data in the work area.
modelData_t bufModelX;
//Destination address?
void * ptr2 = &bufModelX;
//Loading follows
gfsx = GFS_Open(fid);
//Get sectors
//HEY! SBL DOCUMENTATION IS WRONG! THIRD ITEM nzect IS GFS SECTOR COUNT. SECOND ITEM IS CD SECTOR SIZE.
GFS_GetFileSize(gfsx, NULL, &nzect, NULL);
GFS_GetFileInfo(gfsx, NULL, NULL, &fsize, NULL);
//How many frames are we reading?
if(rt_step < fsize){
rd_frames = (fsize + (rt_step - 1))/(rt_step);
}
//Skip the loop if it is finished
if(curRdFrame >= rd_frames){
if(requests[FilRqNm].file_done != true){
  // Copy gfsx payload to modelData_t bufmodel
slDMACopy(workAddress, ptr2, sizeof(modelData_t));
slDMAWait();
//ADDED
    requests[FilRqNm].tmodel->pos[X]=bufModelX.Origin[X];
requests[FilRqNm].tmodel->pos[Y]=bufModelX.Origin[Y];
requests[FilRqNm].tmodel->pos[Z]=bufModelX.Origin[Z];
    requests[FilRqNm].tmodel->length[X]=bufModelX.Length[X];
requests[FilRqNm].tmodel->length[Y]=bufModelX.Length[Y];
requests[FilRqNm].tmodel->length[Z]=bufModelX.Length[Z];
    requests[FilRqNm].tmodel->nbMeshes=bufModelX.TOTAL_MESH;

Uint16 first_texture = loadTextures(workAddress, &bufModelX);
Sint32 bytesOff = bufModelX.TEXT_SIZE+(sizeof(modelData_t));
workAddress = (workAddress + bytesOff);
workAddress = loadPDATA((workAddress), requests[FilRqNm].tmodel, &bufModelX);
workAddress = (workAddress + (bufModelX.PDATA_SIZE + sizeof(modelData_t)));
setTextures(first_texture, requests[FilRqNm].tmodel, bufModelX.TOTAL_MESH, true);
requests[FilRqNm].dstAddress = workAddress;
}
requests[FilRqNm].file_done = true;
jo_printf(0, 9, "(skipped loop)");
goto SKIP;
}

jo_printf(0, 8, "(%i)", nzect);
jo_printf(10, 8, "(sectors)");
jo_printf(0, 9, "(%i)", fsize);
jo_printf(10, 9, "(file size)");

//Read from CD to CD buffer
//CONCEPT: NwFread specifies only a fetch from CD buffer. If no such read has been specified, it will carry it out itself.
//However, if you specify a read from CD, repeats of the fetch command will only fetch from the CD buffer and will not give errors for data remainders.
GFS_SetReadPara(gfsx, rt_step);
GFS_SetTransPara(gfsx, rt_sector);
//TIP: GFS_Nw_CdRead cannot be checked for completion. Sorry!
GFS_NwCdRead(gfsx, fsize);
for( ; curRdFrame < rd_frames ; curRdFrame++){
GFS_NwFread(gfsx, rt_sector, (Uint32*)(workAddress +(curRdFrame * rt_step)), rt_step);
do{
GFS_NwExecOne(gfsx);
GFS_NwGetStat(gfsx, &stat, &rdsize);
jo_printf(0, 11, "(%i)", rdsize);
jo_printf(10, 11, "(fetched filesize)");
}while(stat != GFS_SVR_COMPLETED && rdsize < rt_step);
//Remainder of file in comparison to rt_step [%] is handled, because we do not want to repeat the loop for the end of the data anyway. Do you understand?
//Technically, a file exactly divisible by rt_step will read-loop indefinitely. Watch out for that.
sysframe();
slSynch();
jo_printf(0, 15, "(%i)", curRdFrame);
jo_printf(4, 15, "(%i)", rd_frames);
jo_printf(0, 13, "(%i)", bufModelX.TOTAL_MESH);
jo_printf(3, 13, "(%i)", bufModelX.PDATA_SIZE);
jo_printf(10, 13, "(%i)", requests[FilRqNm].tmodel->nbMeshes);
}

SKIP:
//It is important to ensure GFS is closed at all times outside the read loop.
GFS_Close(gfsx);

sysframe();
}
« Last Edit: June 17, 2018, 05:25:38 pm by ponut64 »

ponut64

  • Full Member
  • ***
  • Posts: 175
  • Karma: +13/-0
    • View Profile
Re: Usage of GFS_NwFread (you probably expected this...)
« Reply #26 on: June 26, 2018, 02:35:17 am »
Just to update you and the specifications,

clearly with the Saturn having an X2 speed CD-ROM drive, the system should be able to handle up to about 5 KB per frame (at 30 FPS) (with a good quality CD and laser?).
I must have had a bad burn that made testing at 2048 bytes per frame cause slow-down. I just did another burn and it handled 2048 bytes per frame (at 30 FPS) without a hitch.
So we can do more than I thought.

XL2

  • Sr. Member
  • ****
  • Posts: 341
  • Karma: +72/-1
    • View Profile
Re: Usage of GFS_NwFread (you probably expected this...)
« Reply #27 on: June 26, 2018, 04:41:59 am »
Yeah, 2 KB sounds great. You could easily stream visibility data with this, which would allow great graphics. But you lose CDDA, so you need sequence music or pcm music, so it's a tradeoff.

 

Sitemap 1 2 3 4 5 6 7 8 9 10 
SMF spam blocked by CleanTalk