Author Topic: Reading binary files with pointers instead of returned char stream?  (Read 212 times)

XL2

  • Jr. Member
  • **
  • Posts: 71
  • Karma: +4/-0
    • View Profile
I managed to build my custom binary file format and read it on the Saturn, so I can display a mesh loaded from a binary file on the CD.

My concern now is that I'm using the jo_fs_read_file, which returns a char stream, which takes memory while I'm loading everything to the right place.
It's not a huge problem while loading small images and sound files, but for a map taking 1 MB, it just won't be doable (unless I'm mistaken and got it all wrong).

Is there a way, in Jo Engine, to just use a pointer to read the file and return each time only what you need (like 8, 16, 24 or 32 bits from a start pointer)?
If not, can it be added in the next version?
Thanks!

mindslight

  • Administrator
  • Full Member
  • *****
  • Posts: 120
  • Karma: +0/-0
    • View Profile
    • Jo Engine
Re: Reading binary files with pointers instead of returned char stream?
« Reply #1 on: August 08, 2017, 01:37:22 pm »
Sure :)

XL2

  • Jr. Member
  • **
  • Posts: 71
  • Karma: +4/-0
    • View Profile
Re: Reading binary files with pointers instead of returned char stream?
« Reply #2 on: August 08, 2017, 10:52:11 pm »
Great! Thanks!

I know the topic is now in "Wish list", but I could need some help.
Bad news ; while the binary mesh loading works in Yabause, in SSF it's a real mess (but works with lower compatibility) while on real hardware it doesn't work at all (it crashes before even loading anything).
It's probably a mistake related to memory allocation or initalization, but I have no idea what's wrong at the moment.
I've attached the files, if anyone with better knowledge of memory allocation could look it up, I would really appreciate!
Thanks!

Code: [Select]
//Start address is 0x00200000

void * LoadBinaryMAP(void * startAddress)
{

 char * stream;
 void * currentAddress;
 int length;
 mesh_tot = 0;
 stream = jo_fs_read_file("MAPT.ZTM", &length);

register unsigned int idx = 0;
register unsigned short i=0;
register unsigned int ii=0;
register unsigned int iii=0;
register unsigned int nbpoint=0;
register unsigned int nbpolygon=0;

currentAddress = startAddress;

mesh_tot = jo_swap_endian_ushort(*((unsigned short *)(stream)));
idx+=2;

for (i=0; i<mesh_tot; i++)
{
    jo3dMesh[i] = currentAddress;
    nbpoint = jo_swap_endian_uint(*((unsigned int *)(stream+idx)));
    idx+=4;
    nbpolygon = jo_swap_endian_uint(*((unsigned int *)(stream + idx)));
    idx+=4;

    jo3dMesh[i]->data.pntbl = (POINT*)(jo3dMesh[i] + sizeof(unsigned int));
    for (JO_ZERO(ii); ii<nbpoint; ii++)
    {
        for (JO_ZERO(iii); iii<3; iii++)
        {
            jo3dMesh[i]->data.pntbl[ii][iii] =  jo_swap_endian_int(*((int *)(stream + idx)));
            idx+=4;
        }
    }
    jo3dMesh[i]->data.pltbl = (POLYGON*) (jo3dMesh[i]->data.pntbl + sizeof(POINT)*nbpoint);
    for (JO_ZERO(ii); ii<nbpolygon; ii++)
    {
        for (JO_ZERO(iii); iii<3; iii++)
        {
            jo3dMesh[i]->data.pltbl[ii].norm[iii] = jo_swap_endian_int(*((int *)(stream + idx)));
            idx+=4;
        }
        for (JO_ZERO(iii); iii<4; iii++)
        {
            jo3dMesh[i]->data.pltbl[ii].Vertices[iii]=jo_swap_endian_ushort(*((unsigned short*)(stream + idx)));
            idx+=2;
        }
    }
    jo3dMesh[i]->data.attbl = (ATTR*) (jo3dMesh[i]->data.pltbl + sizeof(POLYGON)*nbpolygon);

    for (JO_ZERO(ii); ii<nbpolygon; ii++)
    {
        jo3dMesh[i]->data.attbl[ii].texno = jo_swap_endian_ushort(*((unsigned short*)(stream + idx)));
        idx+=2;
        jo3dMesh[i]->data.attbl[ii].sort = SORT_MAX;
        jo3dMesh[i]->data.attbl[ii].flag = Dual_Plane;
        jo3dMesh[i]->data.attbl[ii].colno = No_Palet;
        jo3dMesh[i]->data.attbl[ii].gstb = No_Gouraud;
        jo3dMesh[i]->data.attbl[ii].dir = MESHoff, sprNoflip, UseLight;
    }

    jo3dMesh[i]->data.nbPoint = nbpoint;
    jo3dMesh[i]->data.nbPolygon = nbpolygon;

    currentAddress = (void*) (jo3dMesh[i] + sizeof(jo3dMesh[i]));
}
jo_free(stream);
jo_printf(0, 6, "current adress :            %d     ", currentAddress);
return currentAddress;
}
« Last Edit: August 10, 2017, 05:06:39 am by XL2 »

XL2

  • Jr. Member
  • **
  • Posts: 71
  • Karma: +4/-0
    • View Profile
Re: Reading binary files with pointers instead of returned char stream?
« Reply #3 on: August 09, 2017, 03:42:15 pm »
The good folks at SegaXtreme helped with it, and it was an issue with data alignment.
Seems to be fixed for now!
I will update the tool and FPS demo in the next few days to show map loading from the disc.

mindslight

  • Administrator
  • Full Member
  • *****
  • Posts: 120
  • Karma: +0/-0
    • View Profile
    • Jo Engine
Re: Reading binary files with pointers instead of returned char stream?
« Reply #4 on: August 09, 2017, 04:04:12 pm »
Great !

BTW, I'll add more filesystem feature in the next release

mindslight

  • Administrator
  • Full Member
  • *****
  • Posts: 120
  • Karma: +0/-0
    • View Profile
    • Jo Engine
Re: Reading binary files with pointers instead of returned char stream?
« Reply #5 on: August 12, 2017, 03:16:16 pm »
Hi XL2,

You will find below, the prototypes of the functions I have added in the next release:

Code: [Select]
/** @brief Open a file
 *  @param file Pointer to an allocated jo_file struct
 *  @param filename Filename (upper case and shorter as possible like "A.TXT")
 *  @return true if succeed
 */
bool            jo_fs_open(jo_file * const file, const char *const filename);

/** @brief Close a file
 *  @param file Pointer to an allocated and valid jo_file struct
 */
void            jo_fs_close(jo_file * const file);

/** @brief Read bytes from a file
 *  @param file Pointer to an allocated and valid jo_file struct
 *  @param buffer Pointer to an allocated buffer (length >= nbytes)
 *  @param nbytes number of bytes to read
 *  @return Number of bytes read (<= 0 means EOF)
 */
int             jo_fs_read_next_bytes(jo_file * const file, char *buffer, unsigned int nbytes);


Sample:

Code: [Select]
    jo_file     file;
    char        buf[2048];

    if (jo_fs_open(&file, "TEST.TXT"))
    {
        while (jo_fs_read_next_bytes(&file, buf, 2048) > 0)
            /*DO SOMETHING */;
        jo_fs_close(&file);
    }

XL2

  • Jr. Member
  • **
  • Posts: 71
  • Karma: +4/-0
    • View Profile
Re: Reading binary files with pointers instead of returned char stream?
« Reply #6 on: October 09, 2017, 04:17:54 am »
Would it be possible to add a function to read from an user-defined pointer?

Like, if I'm loading map 2 and it's using the same background as map 1, I want to be able to skip it and just keep it without loading it all over again.

So something like :
jo_fs_read_from_pointer(&file, buffer, StartingByte, Length);

Also, I've had a few failed read on real Saturn.
The Saturns are getting old, so I think it would be nice to implement a re-read feature in case of failure to read.
Like, in your read functions, if it fails, you try again.

Thanks!

mindslight

  • Administrator
  • Full Member
  • *****
  • Posts: 120
  • Karma: +0/-0
    • View Profile
    • Jo Engine
Re: Reading binary files with pointers instead of returned char stream?
« Reply #7 on: October 09, 2017, 08:37:31 am »
Noted  ;)

mindslight

  • Administrator
  • Full Member
  • *****
  • Posts: 120
  • Karma: +0/-0
    • View Profile
    • Jo Engine
Re: Reading binary files with pointers instead of returned char stream?
« Reply #8 on: October 11, 2017, 02:16:37 pm »
I made a new version (http://jo-engine.org/download/) with CD read-retry and a new function:

Code: [Select]
/** @brief Seek forward from current position of a file
 *  @param file Pointer to an allocated and valid jo_file struct
 *  @param nbytes number of bytes to skip
 *  @return true if succeed
 */
bool            jo_fs_seek_forward(jo_file * const file, unsigned int nbytes);

Now you can skip a part of a file and use jo_fs_read_next_bytes() after (just don't forget to shift the pointer : buffer + nbytes)

Example:

Code: [Select]
jo_file file;
char *file_contents;
...

jo_fs_open(&file, "FILE.TXT");
jo_fs_seek_forward(&file, 4096); /* I assume that 4096 is the size of the part to skip in the file and to preserve in the buffer */
jo_fs_read_next_bytes(&file, file_contents + 4096, 42);
jo_fs_close(&file);

« Last Edit: October 11, 2017, 02:22:01 pm by mindslight »

 

Sitemap 1 2 3 4 5 6 7 8 9 10 
SMF spam blocked by CleanTalk