I know I said I wanted to move on from the shadow maps, but looks like I lied.
I refactored a lot of code so I can handle more than one and I made a system to manage custom render pass with render targets.
I still have some work left in the shader part because I still have like 1 uniform for each potential shadow_map. I'll have to use either a uniform array, or checking the vulkan bindless descriptor extension.