User:Kangz/Renderer Discovery

I'm trying to learn how the renderer works, usually I write a lot of things on paper but since we have a wiki why not share my work with the other? I'll be going through all the files in src/engine/rendererGL, things in italic are things I need to understand. Please do not make changes to the page, this is a draft I'm using to learn. If you have comments feel free to express yourself on the discussion page.

Kangz

Headers

renderer/tr_public.h

2 structs refexport_t and refimport_t define the API between the client and the renderer.

refimport_t has API for:

• Printing and errors, cvars, commands
• Memory management
• File management
• Cinematics and AVI recording
• Misc (CM_debug, GL init, input, ftol)

refexport has API for:

• Shutting down the renderer (or reloading it)
• Registration of assets
  • BeginRegistration and EndRegistration respectively remove all the assets and upload all the assets
  • Registration for models, skins, shaders, fonts, fontsVM?, glyph of all sorts and the world + visibility
  • Things can be registered outside of Begin End but it is less efficient
• Drawing the scene
  • Starts with ClearScene and ends with RenderScene (takes the definition of the view, referential definition)
  • A bunch of add* functions to add elements to the scene
• Drawing 2d elements directly (it seems)
• Managing visibility queries
• Setting the color grading
• Misc (RTT, shader management, video encoding...)

renderer/tr_types.h

Companion files for tr_public.h that defines the associated structs and constants

Interesting struct are (add as I go):

• refEntity_t given for any entity that needs to be drawn
  • orientation for the different parts of the model
  • skin, texture and shader parameters
  • a lot of other crap that looks redundant
• refLight_t for any dynamic light
  • Position, orientation, projection
  • Type and shadow casting stuff
• refDef_t the view used for rendering
  • Position, orientation, FOV, screen size
  • Motion blur, color grading and fog parameters
  • Other things
• gl_config{2}_t OpenGL capabilities and settings

rendererGL/tr_local.h

All the things shared by the different rendererGL files

• More constants and enums
• Structs for everything
• A struct with the global state of the renderer
• Some global variables, including all the cvars
• An inline linked-list/queue/stack implementation
• Function definitions

Too many things to list all of it. Will detail in the different relevant files

rendererGL/tr_image.h

Includes JPEG headers

rendererGL/tr_model_skel.h

Prototype of the few functions in rendererGL/tr_model_skel.c

rendererGL/gl_sader.h

C++ prototypes for:

• GLShader and GLShaderManager
• GLUniform derived in a class per uniform type
• GLCompileMacro derived in a class per compile macro
• A class par uniform variable in a shader (derives from a uniform type class)
• A class per shader with derived from GLShader, each of the uniform class for this shader and each compile macro (no virtual method so it's ok to do this).
• A variable for each shader.

Renderer management source files

rendererGL/tr_init.cpp

Stuff that is not used every frame

Contains the definition of global variables (cvars, glconfig, glstate)

Defines some of the renderer console commands:

• modelist
• screenshot{|JPEG|PNG}
• gfxinfo
• glsl_restart

Handles screenshots and video recording stuff

Interesting OpenGL functions:

• InitOpenGL: call GLimpl_Init if not already done, set the default state, initializes the command queue, ...
• GL_CheckErrors_ used by the macro GL_CheckErrors
• RB_ReadPixels, sort of wrapper around glReadPixels
• GL_SetDefaultState

Interesting renderer functions:

• GetRefAPI: the single entry point of the renderer library
• R_Init
  • Called by R_BeginRegistration in tr_model.c
  • Calls the initialisation of everyhing: LUT, cvars, commandbuffers, OpenGL, GPUshaders, images, fbo, vbo, q3shaders, skins, models, animations, freetype, occlusion queries
  • Initialize DX10 as well, with special code for "Nvidia PerfHUD"
  • seems to start the multiprocessing
• R_Register, register cvars and console commands
• R_Shutdown
  • Removes console commands and all the OpenGL state
  • May remove the window
• R_EndRegistration, synchronizes with the render thread and the GPU

Definitions for Com_Printf, Com_DPrintf and Com_Error.

Backend source files

rendererGL/tr_backend.c

TODO

rendererGL/tr_shade.cpp

Contains the Tess_Begin and Tess_End pseudo state machine and code necessary to transform shaders in real OpenGL calls.

• Tess_Begin sets up the tess data structure with some parameters, the most important being the stage iterator: a function that will iterate over the stages of the shader and decide what to do with them (Forward render, and many iterators for deferred rendering: GBuffer, DepthFill, ...).
• Tess_End essentially calls the stage iterator on the tess data structure.
• Most stage iterators: for each stage they decide what to do with the specific surface type, calling one of the Render_ functions.
• The Render_ functions: they set up all the parameters for a specific OpenGL shader and finish by calling Tess_Draw that gives the actual GL draw calls.
• Also contains code that loads and destroys all the OpenGL shaders.

rendererGL/tr_shade_calc.c

Handles the user specified expressions and deformations in shaders:

• RB_EvalExpression returns the value of a shader expression.
• Functions that act on the tess structure to deform vertices, called if needed by Tess_DeformGeometry
• RB_CalcTexMatrix for the transformations of the texture coordinates

rendererGL/tr_surface.c

Definition of rb_surfaceTable with a "Tess" function for each type of surface. Code from the vanilla renderer just fill the arrays in the tess variable table, newer code bind VBOs and IBOs. In the whole file the modelview matrix is supposed to be valid and the calls to rb_surfaceTable are supposed to be surrounded by Tess_Begin() and Tess_End().

Q3 Shader source files

rendererGL/tr_shader.c

This file is a huge parser for shader files. At the start of the engine all .shader files are read and the different shaders are put in a hashmap. After ParseShader and potentially several ParseStage the shader is optimized (multitexturing).

Also contains:

• The shaderlist command
• The shaderexp (evals a shader expression) command
• R_InitShaders

OpenGL objects source files

rendererGL/gl_shader.cpp

C++ code dealing with OpenGL shaders.

Contains the implementation of the shader manager:

• Add defines to the shader text
• Computes shader permutation incrementally or not
• Handle binary shaders

Definition of the Compile macro and shader function (dependances, ...)

rendererGL/tr_fbo.c

Functions to create, bind or attach components to FBOs and:

• Definition of the fbolist command
• R_InitFBO, creates all the needed FBOs

rendererGL/tr_vbo.c

Functions to create, bind, ... VBOs ad IBOs:

• Definition of the vbolist command
• R_InitVBOs creates the static VBOs
• R_Create{VBO|IBO}{2}

Misc source files

rendererGL/tr_decals.c

Function to add decals on the world.

rendererGL/tr_fog.c

Seems to control the fog through the old glFog calls. Most of the code has been commented and I'm not sure if the file is ever used.

rendererGL/tr_marks.c

Functions to project decals onto the world?

rendererGL/tr_noise.c

Functions to fill noise tables.

rendererGL/tr_shadows.c

Contains a single function that change the tess data structure to make the ugly quake3 "flat" shadows.

rendererGL/tr_sky.cpp

This backend file is a big hack to make the renderer render the skybox. Contains Tess_StageIteratorSky and helper function. In Tess_Begin if we render the sky we put the real stage iterator in the second slot and StageIteratorSky will be the one that gets called. It will execute different things depending on the second stage iterator.

Model source files

rendererGL/tr_animation.c

Contains R_LoadMD5Anim and R_LoadPSA, the definition of the "animationlist" console command and R_InitAnimation.

Many functions liked to bones and skeletons (culling, interactions, adding the triangles, ...)

rendererGL/tr_model.c

Model loading and handle->model

Interesting functions:

• RE_RegisterModel: loads a model files and puts it in the store
• R_GetModelByHandle handle->model
• R_ModelInit, essentially memory alloc
• Definition for the modellist command

Also MDX loading, model tag and bone stuff

Format specific files

rendererGL/tr_animation_mdm.c

Functions related to MDM (lod, interactions, animation, ...) and comes with its own math lib.

Tess_SurfaceVBOMDMMesh and Tess_MDM_SurfaceAnim.

rendererGL/tr_model_md3.c

Definition of R_LoadMD3

rendererGL/tr_model_md5.c

Definition of R_LoadMD5

rendererGL/tr_model_mdc.c

Definition of R_LoadMDC

rendererGL/tr_model_mdm.c

Definition of R_LoadMDM

rendererGL/tr_model_psk.c

Definition of R_LoadPSK

rendererGL/tr_model_skel.c

Definition of functions for adding things to skeleton VBO

rendererGL/tr_skin.c

Seems to handle the loading of "skins" for md3 models. We don't seem to use it. Also contains the definition of skinlist

Image source files

rendererGL/tr_image.c

Image (CPU) and texture (GPU) loading, management, computations

Image functions:

• Gamma implemenation
• Image manipulation (mipmaps, blends, ...)
• R_LoadImage with different formats and operations
• Definition of imagelist command
• R_CreateBuiltinImages

Texture OpenGL functions:

• Creation of 2D, Cube and 3D textures (_and glyphs_)
• R_UploadImage with many branches
• Texture mode
• Creates the images needed by the FBOs

Other:

• R_InitImages init memory, default images, ... (charset as well)
• R_ShutdownImages
• R_SetColorMapping
• Color grading upload

Format specific files

rendererGL/tr_image_dds.c

Contains R_LoadDDSImage and compressed image uploading to the GPU

rendererGL/tr_image_exr.c

Contains LoadRGBEToHalfs and LoadRGBEToFloats used in rendererGL/tr_bsp.c (_but what for?_) (_and is it included there?_)

rendererGL/tr_image_jpg.c

Contains LoadJPG, SaveJPG and SaveJPGToBuffer

rendererGL/tr_image_png.c

Contains LoadPNG and SavePNG

rendererGL/tr_image_tga.c

Contains LoadTGA

rendererGL/tr_image_webp.c

Contains LoadWEBP