Messages

61

Share your code / Re: Spreadsheet data analysis SS featured games UPDATED: 17-02-2018 68% Complete

« on: February 20, 2018, 08:46:19 pm »

Thank you Johannes!

I will make more update, but more slow I believe. Right now I think that It is a good database to knowledge the "state of art" in SS.

I want more forward also. I will do you proposals of functions to get to do with the SS what was not done and more.

More work for you and XL2! XD

Greetings!

62

Project announcement / Re: Sonic Z-Treme

« on: February 18, 2018, 12:35:39 pm »

It's possible, but first I need to fix Linux compilation with custom GCC and continue the 3D implementation 

Only with read this: "fix Linux compilation with custom GCC" you are GOD!

And is normal. 3D implementation is priority feature.

But read that is possible, for me is dream! Reverb in SS!

63

Share your code / Re: Spreadsheet data analysis SS featured games UPDATED: 17-02-2018 68% Complete

« on: February 17, 2018, 09:45:58 pm »

UPDATE 17-02-2018:
- Complete Sonic Team games analysis.
- Improve SCU data % in some games.
- Added new data of "Table fog" and "Tesselation" columns in a few games.
- Added some new game credits in Perfect/Tantalus games
- Start analysis a Altron games.
- Improve simplification data entries and English entries.
- Improve and add more color filter to know level of analysis in a lot of games.

64

Project announcement / Re: Sonic Z-Treme

« on: February 17, 2018, 07:18:20 pm »

Everything is automated, which is why the textures don't look all that great, but my pushing them further I might be able to hide it. About the open space, my engine doesn't support proper occlusion culling (pvs or portals), so I need to limit the draw distance.

When you said  "Everything is automated"... is in realtime or automated in a script when you create the scenary??
I answer myself: Script map tool make it the automated features. XD

Anyway. A great job again.

I've been reviewing forum threads. And I've seen that you were investigating the clipping user of slave engine. Finally you concluded that it helped to improve the bottleneck of the VDP1 itself. But not the DSP calculation of the SH2 or SCU.

Combine both techniques to optimize both side of the VDP1 blitter and side of the 3D transformations for loading the processors (SH2 or SCU). It would be the ideal scenario to take advantage of the SS, right?

Maybe you're doing it already.

Finally, I think I understand you right now perfectly regarding the issue of open areas. With a pvs or portals algorithm. You can discriminate rooms by size or distance and so you can stop processing or drawing before said geometries. Or even apply a more aggressive LOD more selectively, no?

On the other hand. I have been looking at the use of SCU in Burning Rangers and other games. I said that I used 3% of registers and I was wrong. Out of a maximum of 23 registers, use 3. Which are actually 13% and 70% code. Against Sonic R 53% registers and 80% code. Or cases such as Wipeout 1 (Registries 60% Code 33%) or Assault Rigs (39% Registries 60% Code) where they also use the DMA. Something must be doing this code in BR. Is it possible to help decompress the Audio ADX??

Which brings me to another question. What I see also you have been researching in the forums. Using PCM for sound takes up a lot of memory. SDK SS there are libraries to compress/decompress ADPCM, which in essence is very similar to ADX. You will save a lot of space and decompress whit the M68000 sound proccesor.

Which leads me to another question that surrounds me these days and I do not understand. The SS sound chip is a beast. That has programmable DSP. It is possible to do things with him. Because I have not seen any game with reverb. Something that is seen in many PSX games. And it is supposed SS could do. I know that for this I had to buy another sound development SDK. And that's the reason for not seeing more things like that in games.

It is also a question to Johannes. Is it possible to implement this effect in the Jo-Engine library? Reverb and / or echo effect for sound or sound channels.

Greetings!

65

Project announcement / Re: Sonic Z-Treme

« on: February 17, 2018, 12:58:11 am »

It's not perfect, but still not terrible either :
https://youtu.be/lLiq5V_nlsg
 
I might just push the LOD draw distance to hide it better or play more with the mipmapping or even hide it with gouraud shading, but it now generates textures to fit the merged quads.
One huge issue is that the quads can have different width/height, so the generated textures doesn't fit perfectly.
It's even more obvious on Quake maps, but anyway I think it can be improved.

Another awesome step! Are mip mapped manually the textures?? How many levels? Max/min resolution?

The fit problem is normal, in auto convert geometry. Whit a manually clean and adjust this problem fix it almost all cases.

A very little doubt? Is possible adjust to the max peak(1000/1200) of the showed quad geometry, to show the max or near to the max, for all most time to increase farclip??

And last. It possible adjust the levels of shades to farclip. To have more equal color fade to far. To soft the poping??

Amazing work!!

The Doom engine has large open space, which isn't too great on the Saturn.

I understand well the sentence? The open space in Doom not capable for SS?? I mean, Exhumed or Duke3D had have open space and SS had been capable to move whiteout problem. I hope in the future, We give a definitive port of Doom to SS.

66

Project announcement / Re: Sonic Z-Treme

« on: February 13, 2018, 08:49:50 pm »

About the low quality model, I just meant that by changing one line of code I could choose to display the low quality or high quality model.
About 1200 quads on screen, it's totally possible even at 30 fps. But for a FPS, that means many many pixels that keep getting rewritten over since the walls - if you don't have proper occlusion culling - can take the whole screen.
About Burning Rangers, there is a SCU DSP program in memory, so are you sure it's not used?
I don't know if Wipeout and Tomb Raider do the quad subdivision in realtime, but one way to see is to check the texture memory address : if 1 quad turns to 4, look at the top left quad. If the address is the same as the lone quad texture address, with different width/height, then you know it's probably done in realtime.
But they could also store it in memory since the low quality map should take maybe as little as 1/4 the size of the high quality map.
About SBL and SGL, both are really interconnected. For graphics, it's SGL, for almost everything else it's SBL or SBL wrappers.

Perfect thanks, now I understand.

It is true, in a FPS the probability of covering other objects and many of them is greater ... I think now about how the Slave engine does it. It's really complex, using Clipping user and system every "X" elements ... it's almost the only game I've seen that does that. Even with the sky drawn he does it in case of the Quake.

It is true that there is a program. But SSF does not detect the use of the SCU ... and seeing the registers ON, they are only 3% ... I really have doubts in this case.

I will investigate more deeply Wipeout, Tomb Raider, Nascar 98 and Courier Crisis, which are the games that until now I am sure do this.

With respect to SBL and SGL ... I understand, but they seem very separate "projects" ... And the logic should be that these should be closer still ... I do not know.

Thanks again for the quick replies!

67

Project announcement / Re: Sonic Z-Treme

« on: February 13, 2018, 07:25:24 pm »

For the depth cueing, it's what I'm using for the gouraud shading.
For the LOD/tessalation, the thing is that all the work is done offline, so there are really 2 full maps in memory, I display one or the other depending on the distance from camera for each node. It could be done manually for better results, but I don't have time for that so it's all automated during conversion.
Nothing is built in realtime.
While it would be possible to subdivide a quad in realtime, the other way around can't be done since we are dealing with distorted sprites only.
I could choose to just use the low quality model if I wanted, but the results would be bad.
About the 1200 quads on screen, SGL transforms the vertices, then does backface culling, so only half those quads reach the VDP1. Using a BSP tree could reject them earlier, but walking the tree would be slower than what could be gained (and it uses more memory).

The SCU DSP is in assembly only, and SGL doesn't have functions using it, while the SBL functions are useless : they transform vertices from a different matrix format and it is not compatible with the SGL quad processing (buffers and all).
The DSP RAM is also too limited, so I can't really use it for a depth buffer or something like that, while SGL already has a strong lightning support built in, so I wouldn't gain anything from replacing it.
It's hard to find a good use for it in these circonstances.

For Quake 2, I could just show 1 or 2 maps in my engine, but I wouldn't want to port the whole game since there are much better ways to play it already.

OK! Now I understand you. Yeah, you are implemented LOD for the scenery. But is a absolutely great achievement. And is a good opportunity to design great levels to fit in memory.

This sentence I don't understand: "I could choose to just use the low quality model if I wanted, but the results would be bad."

I think that, in Tomb Raider and Wipeout 2097 is realtime. Because both games full the memory and both are been port to PSX whit the same memory. The Wipeout 2097 is a port from PSX, that have this feature implemented in SDK.

Whit the data recompiled in Yabause. 1200 quads in screen not is are impossible situation. Sonic R(2x SH2 + SCU use) or Burning Rangers(2xSH2 no SCU use) have more than this quantity and run whiteout problem at 30FPS. 4x4 Hardcore(1500) or Tomb Raider(1300) whit slowdown to 15FPS o less in the 4x4 case. Yabause only list the elements in the screen, not the calculated or back face clipped... Maybe more really.

Is true the SBL an SGL, are a two different approach to build a "engine" or conjunction of library to arm a SDK. In some form the structure of libraries SBL is similar to PSX SDK. But SGL is more similar to PSX SDK in the name of the specify 3D functions and variables in it. I think in last version SEGA tried to fusion both... but not is perfect SDK to exploit to maximum SS.

This days I think about the fantasy, of the in time, fusion of the Tantalus/Perfect engine(Yes I think in some aspect are good features) + Argonaut Engine + Sonic Team Engine + Sonic R Engine + Slave(Lobotomy) Engine + Game Arts FMV codec... for put some of the best devs engines of the time. If SEGA would have invested in all this GREAT code, the SDK enriched greatly.

Anyway. Your work, the Johannes work and the other awesome people of the SS community homebrew, this days make true my dreams. Thank you!

GO GO GO! Greetings!

68

Project announcement / Re: Sonic Z-Treme

« on: February 12, 2018, 09:17:59 pm »

Your progress is really great! AWESOME !!

And also it's a coincidence with me! These days in my research I have added two columns to table data. Now I will update them.

1) If a game has "Tessellation" (you call it LOD) as you say Tomb Raider has it. But there are some more games. The complication is that the game has to have wireframe mode to discover it. Or that the wireframe is implemented in Yabause or other emus to see how many more games are used, surely many! Since it is a trick to gain drawing distance and perfect speed!

* The LOD is the reduction or increase of geometry in an object. Like the FIFA players or the Daytona cars. When it is in the "terrain" or "stage" in the engines they are called Tessellation.

2) I have also added a column of "Table fog" since it was an effect that in PSX also looked a lot. But also in SS!!

I was able to confirm that both effects were fully integrated into the PSX SDK. They are three functions. The "Table fog" is called "Depth Cueing" and the "Tesselation" includes two functions: 1) "Polygon Division" and 2) "Mip-map Function Library". All of them using the GTE.

On the other hand I have searched and not found in the SS SDK any function to do these things. In these cases, programmers create their own code to have these effects in their games or ports. But it is very possible, that each one did it in a different way... and none in the most optimal way. Here perhaps, SEGA failed to detect these needs and integrate them into the SDK.

I think that in this rebirth that you are building, functions like these would be very useful and necessary. I have many concrete ideas for these functions (to optimize the use of memory, VDPs and processes) and proposals in development. But specifically in these cases using the SCU-DSP, would be ideal.

EDIT: I seeing your debugging text I have remembered the polygon limit values (theoretical) in the SS SGL and in the PSX SDK.
SGl talks about:
MAX_POLYGON (number of polygons that can be used): 1800
MAX_VERTICES (number of vertices that can be used): 2500

And in the PSX SDK, in the "Performance Analyzer" manuals you can see analyzes of the game "Motortoon Grand Prix 2": 1200/1040 polygons

It is curious to see how both machines are in very similar numbers.

Greetings and thanks!!!

Hei guys, when you talk about "Tessellation" or "LOD" you mean something like this? (Go to minute 05:00).
https://www.youtube.com/watch?v=ME7Dkpgd_-k&t=175s

Different levels of detail according the distance?.

Very impresive work XL2, thank you very much for doing this.

Very similar techniques. Two link to wikipedia:
https://en.wikipedia.org/wiki/Level_of_detail
https://en.wikipedia.org/wiki/Tessellation_(computer_graphics)

In the case in this generation of 3D. The improvement of XL2 in your engine is more similar to modern Tessellation. Is a LOD automatic or calculate in real time in function of the variables. To put in screen the optimal number of data geometry.

LOD is a manual technique or static. I mean, you create the levels o detail model for a one specify model. This is used a lot in a lot of game in all catalog in SS. For example FIFA games or similar games. Is easy see the steps in the models. Or in Daytona USA in a cars.

Normally the tessellation is use in the scenario. I one part o in all. In the case of Wipeout or Nascar 98 for the road. In other case like Tomb raider or Courier Crisis for all scenario.

In this age. A lot of games use it. More in PSX for easy to implement. For me one the most aggressive Tesselation + LOD in PSX games us very last games: Terracon (2000)

Good news! I only put around 1 hour since the last video, but seems like I got the collision almost right on my first try. I still have to make some adjustments, but a level like a Quake map is fully working now (except for doors).

I still have to write the texture generation to match the merged quads and use another frustum culling technique (I have an octree, but it's not used other than collision since it can't work well with the SGL frustum culling functions).

Here is a Quake map in my engine, with fully working collision : https://youtu.be/uBa6NNM4ZU0

Still no portal or pvs or depth buffer, so it wouldn't work nice on real hardware unless I drop the texture quality (it's 64x64 right now, but with the overdraw it will be slow).

Again my congratulations!! 1200 quads in screen!! Wohooo!!!

I can wait to put in my SS and in yabause to see your work near!

In other hand. I continue my research. I can confirm, that in last SGL (3.0), SEGA implemented the Depth Cueing like PSX SDK. For the side of the use of SCU in SS SDK. Is strange. I can see a lot of function in SBL y SGL to use DMA, access to RAM storages and all stuff tu use it. But not are a code specify implemented in C to call a ASM code or similar. Only in SBL I can find some come to transform or matrix in a folder name MATHS. Is rare, because the very first game of AM2 use it SCU: VF1 and Remix

Finally! I can read in youtube a user said it, about port Quake 2 to SS in your engine. I can help you whit the graphics part. If some day this project start. One of my dreams is make a very good port of Doom. Best that PSX, I believe it possible. And because no! a Quake 2. But Sonic Z-treme is good start to put to limit a our lovely machine!! I want see more transparencies in SS, like PSX... real lens flares for example. I try to design some trick or way to do it!!

Stay in touch! See ya!!

69

Project announcement / Re: Sonic Z-Treme

« on: February 11, 2018, 01:06:04 pm »

For RLE, you mean using SBL?
I thought of something else, but everything is on the table.
It might take a few months before I truly focus on pvs.

I am using both SH2, but not the SCU DSP. The problem with the SCU DSP is that it needs to be coded in assembly (I haven't learnt it yet) and there is some overhead since you need to DMA transfer data, start the DSP program, wait for it to complete the processing, stop the program and then fetch the data. Unless you are dealing with complex maths in parallel, it's not really worth it. Most people used it for lightning calculations, but SGL for some reasons didn't use it.
It also has really limited RAM and can't do divisions, so you can't do that much with it.

For 60 fps, it would work with around 450 quads on screen, no gouraud shading, low res textures and/or no overdraw, so it's not really a solution for now with complex levels.
But it works nice with Sonic X-Treme levels.

Yes, I remember some library for that. Is a CMP LIB. In the "ST-135-R4-092295 Program Library User's Guide 3" is explain.

Are great if you are using right now the two SH2. SCU DSP is truly powerful but is high complex to use it, I know... but the memory I don't think is the problem here. First the ASM language. Second design that task make and design the process thread carefully. It is possible to tessellation(LOD) use the SH2, and for lightning or transformation use SCU to speed up. Is only intuition for my part, I not a programmer...

SGL is created mainly for AM2 team... I think in its games, the use of lighting is very limited... and not need it more speed in this area. This reason it possible to not use SCU DSP at all.

Other case is Sonic Team, whit Burning Rangers... Is are a truly lighting system... but not use SCU... is rare.

The 500 quad at 60FPS is real numbers... because in the games whit this FPS the numbers and shading used are it you said Two examples: VF2, Mass Destruction... etc...

Thanks for the feedback!!

70

Project announcement / Re: Sonic Z-Treme

« on: February 10, 2018, 08:13:26 pm »

For using untextured quads for far away objects, I won't need it to improve the framerate as I think I'm on the right track with the current technique (the per-quad collision might cause some issues thought if it's not fully optimized for early rejection).

For the clipping, my plan is still the per-octree node pvs, the main issue will be the compression used to store the data, but I'm not there yet as the new collision technique might take a while to get it right.

Sorry for not answer sooner... Absolutely you are in the right path. Your actual work and improvement in the engine talk about the quality work it for you.

My only intention is try to add something possible ideas to get new paths to improve the use of the machine.

In other hand your goals are impressive: Clipping with per-octree node pvs, the compression for store the data(in SDK SS can see the RLE compression are been implemented) and finally a new collision technique. All goal are important and impressive to build a strong engine for a "SS Sonic 3D" game

Still lot of work to be done on the per-quad collision, the view frustum culling and the LOD (or tessalation) texture generation, but here is, just to show my newer version of the engine, an old "map" revisited with the map converter (I didn't try to fix the geometry for the Saturn other than turning triangles to quads and subdividing everything once).

The intro level isn't the biggest level, but it shows that Mario 64 could have worked on the Saturn (with perhaps some compromises of course).

Video here : https://youtu.be/ScGVHMFB58M

Is a outstanding work! Really! You are use the two SH2?? Because is possible will go more far... and If you could use de DSP of SCU it be a crazy improvement!

In the video is possible see a peak of near 1000 quads in screen at 30FPS!!! is a madness!! XD Is possiible that you cap the limit FPS to 30, is possible that you rendering at more FPS, near 60FPS??

Thanks and encourage!!!! GO GO GO!

71

Project announcement / Re: Sonic Z-Treme

« on: February 08, 2018, 09:01:56 pm »

Hi XL2 !!!

I discover new game that use "tessellation" or LOD like you say in Scenary generation. Is Courier Crisis. This game has wireframe cheat to activate. This game in SS version have more draw distance and FPS that the PSX version !!! Use "Color Calc. Mode: Gouraud Shading + Half-luminance" to far "quads/distorted sprite" and also change for 3D objets and something quads to flat polygon whit base color also. Is a good example to add trick to engine.

The key I think, and the complicate is a good clipping algorithm. Is it possible to write to a simple algorithm for the engine to know when to cover the object / s behind and are to more distance to not draw them??

Greetings!!

72

Share your code / Re: Spreadsheet data analysis SS featured games UPDATED: 04-02-2018 67% Complete

« on: February 04, 2018, 10:29:43 pm »

UPDATE 04-02-2018: I add two new columns. M y N is a VDP1+VDP2 effects or use. M is new column to known the use or "table fog" effect or similar. N column is a reorder VDP1>VDP2 transparency effect data. The other new column is a AB. In this column check is a game make "tesselation" effect to render polys.

Other improvements: Simplify data to make better filters data. Add more date in a lot of games.

73

Project announcement / Re: Sonic Z-Treme

« on: February 04, 2018, 08:24:13 pm »

Your progress is really great! AWESOME !!

And also it's a coincidence with me! These days in my research I have added two columns to table data. Now I will update them.

1) If a game has "Tessellation" (you call it LOD) as you say Tomb Raider has it. But there are some more games. The complication is that the game has to have wireframe mode to discover it. Or that the wireframe is implemented in Yabause or other emus to see how many more games are used, surely many! Since it is a trick to gain drawing distance and perfect speed!

* The LOD is the reduction or increase of geometry in an object. Like the FIFA players or the Daytona cars. When it is in the "terrain" or "stage" in the engines they are called Tessellation.

2) I have also added a column of "Table fog" since it was an effect that in PSX also looked a lot. But also in SS!!

I was able to confirm that both effects were fully integrated into the PSX SDK. They are three functions. The "Table fog" is called "Depth Cueing" and the "Tesselation" includes two functions: 1) "Polygon Division" and 2) "Mip-map Function Library". All of them using the GTE.

On the other hand I have searched and not found in the SS SDK any function to do these things. In these cases, programmers create their own code to have these effects in their games or ports. But it is very possible, that each one did it in a different way... and none in the most optimal way. Here perhaps, SEGA failed to detect these needs and integrate them into the SDK.

I think that in this rebirth that you are building, functions like these would be very useful and necessary. I have many concrete ideas for these functions (to optimize the use of memory, VDPs and processes) and proposals in development. But specifically in these cases using the SCU-DSP, would be ideal.

EDIT: I seeing your debugging text I have remembered the polygon limit values (theoretical) in the SS SGL and in the PSX SDK.
SGl talks about:
MAX_POLYGON (number of polygons that can be used): 1800
MAX_VERTICES (number of vertices that can be used): 2500

And in the PSX SDK, in the "Performance Analyzer" manuals you can see analyzes of the game "Motortoon Grand Prix 2": 1200/1040 polygons

It is curious to see how both machines are in very similar numbers.

Greetings and thanks!!!

74

Share your code / Re: Spreadsheet data analysis SS featured games UPDATED: 02-12-2017 60% Complete

« on: December 14, 2017, 05:50:05 pm »

Thanks to you for your awesome work whit Sonic Z-Treme!!! GO GO GO!!

75

Share your code / Spreadsheet data analysis SS featured games UPDATED: 26-06-2018 81% Complete

« on: December 12, 2017, 12:17:24 pm »

Hi community!

I want share my little research of the technical use of the games developed in the past to our machine. To learn and discover  how deep use the machine.

The intention I have is to share and help grow the community of our great console. A small contribution and tribute to this great machine. Which he owed for a long time.

I share a spreadsheet where I am compiling technical data of a selection of games(258, aprox 20% total games)  that I think are outstanding of the system. That can help us to better understand how effects were made or how they used the machine better. Currently this half-translated into English, since my native language is Spanish.

UPDATE table and data at 26-06-2018, view better whit 65% zoom level in google docs viewer:
https://docs.google.com/spreadsheets/d/1tcTUC9oZ70ZGzVV58ca20bfwn0YEWzyJbxAMVFrAVwE/edit?usp=sharing

Some notes:
With respect to the use of memory, it has been one of the last aspects that I am compiling. They are in column AA. I have tried to break them down by the different memory wells. The data is approximate, since even the Yabause does not quantify in total values ​​the use of memory. Column AB recollect the staff credits, to find the best programmers or teams. Column AC recollect dates release to see how the delay the version or port to SS respect PSX or others...

I started by analyzing the use of transparency of VDP1, VDP2 and VDP1 + VDP2. Afterwards, with depth of color in textures and total, use of gouraud and lighting, use of layers of VDP2 and color calculation ... to gradually embrace many aspects. [EDIT 03-01-2018] Like type of FMV formats, data of the archive an quality, use of SCU for FMV or 3D, Use of VDP1 or VDP2 or both to FMV. Use of sound system of the CDXA or DSP... etc...

[EDIT 14-12-2017] It would help a lot, that Yabause not count the number of elements in the VDP1, to see how many polygons are on the scene. <- Now we can know how many elements in the scene whit the last Git version of the Yabause. I update the "Z" column whit the data of the many games. Not possible filter by types of color calculation or type or data, but is very useful new feature. Or the amount of elements that are using Gouraud, or half-transparent, filter them somehow. Since the debugger lists them all and their properties. Or ideally show the % use of the processors, this last more complicated.

I will update and translate the sheet as I can. If someone wants to contribute in the edition only that asks me, the more we are better.

Regards!