Incredibles posts these days XL2!
Well, it's a very creative solution on your part. I think that if you can facilitate the creation of content with this routine, it could be used well, in specific things. For large crystals for example.
With respect to making the transparency work in both VDP1 and VDP2. I still believe that you have to see how Burning Rangers does it. Well I think it's the best solution, using the SS pipeline.
The maximum that I have come to find out is that BR make two "spaces" of drawing are alternated in the VDP1. In first, only the opaque elements with their texture or color are drawn. And in a second the transparent elements and a part of the black opaque elements that cover the transparent elements.
The first space is drawn with a clipping system "total" and the second to half. For example for a final output resolution of 320x240 non-interlaced, with a Clipping System of VDP1 of 319x239. The second space will be 160x120 and then send it to VDP2 to NBG1 at 16bit color and using a Color calculation of this layer over the VDP1. The elements drawn by the VDP1 will look transparent over the VDP1 and VDP2 at the same time.
All for "one" frame. BR have 20FPS peaks stable.
Problems that still exist:
1) The elements of the VDP1 between them do not mix. Could we use VDP1 H-T? Solving the redrawing problems. Using only non-deformed elements in the vertical like: Scaled sprites, Normal Sprites or Distorted sprites like Billboards. Or new tricks like yours XL2.
2) Can we get to render the second space to the total resolution?
3) In Burning Renger the final layer of transparency is on top of everything, including the UI. Could we somehow avoid this problem? For example creating a mask on these parts. Or using a VDP2 layer for the UI.
Objective: Make a total Sun Lens Flare effect that works on VDP1 and 2 and that are 3D elements of VDP1.
A for all!