Geenz's office hours for graphics.
Here are the notes from 4th November 2021
These are really :(, rough notes typed by ProbablePrime. If there are errors please edit away!
Please do remember to check our Roadmaps as a lot of questions here are partially covered there. Do continue to ask them but the roadmaps are there for you to read:
We're getting there, most of the serialization changes have been completed.
- We can now convert between color and colorX(the new color changes and data types)
- You can see colorX inside Neos. Continue to not use colorX for the moment.
- New Color texture space changes - The ability to set a specific color space for a texture. Linear or SRGB. Only two that have hardware acceleration. More might be later.
- colorX Propagation - Moves colorX as a type forwards through Neos' engine including LogiX and Components.
- Generic Types are a challenge but we're working on it. Generic Types are components/nodes with <T> in them such as ValueCopy<T>.
- Getting this settled.
- we need froox's help for some of this.
On the unity side:
- After that we have to edit/get the Unity stuff all sorted so we can "flip the switch".
- Most of the work that's already happened is color space aware and done, its just waiting from a Signal from the above items.
More work will come with the new engine of course, once its ready.
- Will be deprecated eventually, we do not recommend using it at the moment but we will not remove it.
- We're going to see if we can come up with a replacement
- This replacement will be material layering and is not going to come with color management it is a separate feature.
- Layer materials complete with masks etc with one and other.
- Allows for future features too.
Material Property Questions from ZyZyl
How do material property blocks interact with the dynamic batching / GPU instancing processes? I assume that two different meshes with the same material, but texture affected by material property blocks, will result in them counting as sharing the same material for batching / instancing checks.
Are there limits on where one should use material property blocks? If I had 1000 different objects all using PBS_Metallic with the same settings, but a different main texture, would it make sense to use a MainTexturePropertyBlock on them all? Would that cause issues?
The Unity documentation for dynamic batching states (https://docs.unity3d.com/Manual/DrawCallBatching.html):
- Batching dynamic GameObjects has certain overhead per vertex, so batching is applied only to meshes containing no more than 900 vertex attributes, and no more than 300 vertices.
- If your Shader is using Vertex Position, Normal and single UV, then you can batch up to 300 verts, while if your Shader is using Vertex Position, Normal, UV0, UV1 and Tangent, then only 180 verts.
Given that, is there an easy way to determine how many vertex attributes a specific Neos material will be using so we can estimate whether a particular mesh can be batched? If not, how many would be used for the common materials (e.g. PBS_Metallic/Specular, Xiexe toon) and are there any major exceptions?
If a mesh has submeshes with different materials assigned, are those submeshes able to be independently dynamically batched with other (sub)meshes which may share a material?
Example might be 2 wine bottles each with 2 materials, one for the glass and one for the label. They have different glass materials, but the same label one. Can the label submeshes be dynamically batched together?
Is there any rendering benefit to sharing materials across meshes which cannot be dynamically batched / instanced together?
Uhm, that was a lot. Prime is trying his best with the answer here:
- The first part of the question, How do property blocks interact with the batching stuff?
- Unity is going through and saying, this is batched, this is instanced.
- With instanced, you can change properties on some materials and still have them batched.
- If you have 500 objects, with the same basic material and a texture material propert to change just the texture
- On the GPU, the mesh will be drawn in a special way where each object is drawn in their own way such as position etc. It tries to re-utilize the same draw calls etc.
- If you have 1000 objects using the same material with blocks for each objects texture, its unique from GPU to GPU. This would probably be broken down from 1000 to 63 ish draw calls.
- This is not an end all or be all.
- Dynamic Batching is different.
- Basically, your calculations are correct here to see what you can fit into a batch.
- If you went through with the wine bottle, one for glass one for label. Same Label would probably be batched together. Provided that you don't exceed vertex counts and vertex attributes..
- In cases where the meshes cant be batched, you can re-use the materials as there's a memory and processing benefit.
- 500 materials with 500 meshes. For each material render each mesh -> same thing 500 times. with significant overhead.
- The same material on 500 meshes, that are incompatible with batching and instancing. It still reduces the memory and processing by using the same material. So continue to optimize your material use.
Modern: I'm curious: how is Neos's rendering different then other games like VRchat?
- Neos uses deferred rendering.
- Tries to split up light from object rendering
- These were coupled together like a list of lights and a list of objects coupled.
- With complex scenes this is slower on GPU and CPU
- Deferred, splits this and lets us have more lights. Allows for faux ambient lighting, generally allows for more lights. (More than 4)
- Reflection Probes and upcoming light probes will benefit from this.
Modern: What is the difference between "Color" and "Colorx", specifically targeting the W value i found in the datatype.
- Color is an unmanaged color, raw color data RGBA
- ColoX has RGBA but also has a profile. Not all types in the upcoming update is colorX aware/profile aware such as Textures. For textures you have Colors in textures and then a profile that is for that color.
- Same for meshes, you dont need profiles on each color, just an overall Profile.
- The W Value. iis probably Float 4 and Color casting. W usually represents Alpha.
Turk: How often do serialization issues tend to pop up? Is it more something you encounter when doing big changes like color management or more conversion changes? It crops up when large scale changes crop up making sure data flows through the engine and systems and network. Casting between data types and major changes. Needs special precautions.
- We dont want to break content, if we do accidentally break content we can provide.
- Prime: Serialization also comes in non color spaces such as time and dates. We're working on it but I don't want to break your dates and times
GunGryphon What would be the most efficient method for evenly lighting a large indoor area after baking reflections? I worry that only using just a few point lamps will cause funky lighting artifacts.Specifically, I'd like to efficiently approximate the lighting from a row of small lights. You're doing it correct by baking reflection before lighting but if this is a dark environment you might have a dark reflection probe Your best shot is go through and bake reflections AFTER lighting. You could have 2 probes for when lights are on and off etc and swap between them.
Kulza: Material Property block for color does it exist. I think there's one, double check. If not I'll add it. Geenz checked, we don't have. Geenz will look at adding that. you can do a lot of stuff with that.
Turk: With the recent announcements by Meta(Facebook) and claims by Carmack do you think most folks will stick to unity and unreal environments or do more custom ones like we'll have someday? I ask this because with dev adoption on tools and industry standards that may be set by these type of moves. Any thoughts on this are appreciated.
- Geenz doesn't like doing opinion pieces(Prime agrees).
- This is probably a better topic for off-topic areas.
- Its gonna be a hodge podge.
- What the VR side of META should do is become a platform for VR headsets and games and stuff.
- It makes sense for a business sense
- What they should be focused on is making sure that when you have Meta VR avatar and content, they should make you be able to bring avatars and content between Meta products.
- For example, AND ONLY AN EXAMPLE WE ARE NOT WORKING WITH META/FB!!: Say there's a META avatar we could make it work inside the Quest version of Neos easily etc.
- People will continue using other engines such as unity and unreal because they may be easier or ingrained.
epicEaston197Q: when I'm not looking at a avatar what is being rendered is the ik disabled? is the Avatar completely gone if it's not within my view?
- There are some things still happening, simulations etc. Frustrum culling disables rendering if stuff isn't in view.
- If an object is not within a frustrum it will not be rendered.
- Certain CPU sides still occur
Alex rainbowdashie: Will the new colour management new engine affect the way current ambient occlusion works? On the same rendering
- Yes, anything affecting lighting will be affected with Color Management.
- Smoother Gradients.
- There are no concrete plans to change AO right now except moving it to the ambient pass of deferred rendering objects.
Modern: Will all the settings on the materials actually work? Some material's settings don't work right now.
- In time all material settings should work on materials
- If you have specific issues please file issues
- Keep in mind the PBS consolidation that will happen.
- All options on PBS condensed into one.
Modern: How material property blocks Prime went off the books here because he lost the flow of what was being said sorry.
- Given a material you feed a material property block to it and it will change the property to that value but just for that material.
- You do this on the mesh renderer.
- The GPU picks this up and can optimize and use the property blocks to render the meshes together as a part of the instancing optimizations it can do.
- Is there any plans to add some sort of LOD system for distance?
- We have an LOD system in place rigght now and its based on screen spaces etc.
- It looks at how many pixels are in place. The component is called "LOD Group"
- For distance based LODs geenz isn't really sure what Modern meant, LODS are usually done by screen size.
- Geenz isn't sure this applies here.
- Are you mixing up culling or render distance settings? Those are more affected by distance from objects or occlusion etc.
- You can use a far clip on your avatar rendering but that's not the same.
sls: vary roughly how far is neos away from a non unity based rendering engine?
- We're not talking about it too much as its still in progress and the basics are going on under the hood.
- Sometime next year is a rough estimate.
GunGryphon: I would like to have many objects (in the hundreds) use the same mesh/texture data and be able to update all of them from a single reference. What is the preferred method for this?
- We don't have a great way to suggest doing it.
- Checkout: https://github.com/Neos-Metaverse/NeosPublic/issues/2829 & https://github.com/Neos-Metaverse/NeosPublic/issues/83
- Prime: also remember the scene and object modes of the material tooltip
Sevenths Questions If color Management:
- No performance boost from color management, the same performance.
- Vertex dense meshes might hitch for a little when converting on asset load. But not continuous.
- If you're talking about frame rate, itll be the same
If reference to new engine:
- It'll be faster with stuff in .NET 6.
- .NET 6 are faster than .NET Framework and Mono
- Gives us lots of opportunity to optimize