OptiX works by using user-supplied instructions (in the form of CUDA kernels) regarding what a ray should do in particular circumstances to simulate a complete tracing process.^[4]

A light ray (or perhaps another kind of ray) might have a different behavior when hitting a particular surface rather than another one, OptiX allows to customize these hit conditions with user-provided programs. These programs are written in CUDA C or directly in PTX code and are linked together when used by the OptiX engine.

In order to use OptiX a CUDA-capable GPU must be available on the system and the CUDA toolkit must be installed.

Using the OptiX engine in a ray tracing application usually involves the following steps:

• Defining programs for ray generation (e.g. rays can be shot in parallel, in a perspective fashion or like a gradient field), ray missing (when a ray doesn't intersect any object), an optional exception program (when the ray cannot be shot for some reason), a bounding box program (the program that provides a bounding box intersection test for a given object) and an intersection program.

Several examples for these programs are available with the program's SDK

// Sample code using OptiX APIs //

/* Ray generation program */
rtProgramCreateFromPTXFile( *context, path_to_ptx, "pinhole_camera", &ray_gen_program );
rtContextSetRayGenerationProgram( *context, 0, ray_gen_program );

/* Miss program */
rtProgramCreateFromPTXFile( *context, path_to_ptx, "miss", &miss_program );
rtContextSetMissProgram( *context, 0, miss_program );

/* Bounding box and intersection program */
rtProgramCreateFromPTXFile( context, path_to_ptx, "box_bounds", &box_bounding_box_program );
rtGeometrySetBoundingBoxProgram( *box, box_bounding_box_program );
rtProgramCreateFromPTXFile( context, path_to_ptx, "box_intersect", &box_intersection_program );
rtGeometrySetIntersectionProgram( *box, box_intersection_program );

Bounding box programs are used to define bounding volumes used to accelerate ray tracing process within acceleration structures as kd-trees or bounding volume hierarchies

• Create material any hit and closest hit programs: these two programs determine a ray behavior when encountering its first intersection (closest hit) or a generic intersection (any hit)

// Sample code using OptiX APIs //

rtProgramCreateFromPTXFile( context, path_to_ptx, "closest_hit_radiance", &closest_hit_program );
rtProgramCreateFromPTXFile( context, path_to_ptx, "any_hit_shadow", &any_hit_program );

/* Associate closest hit and any hit program with a material */
rtMaterialCreate( context, material );
rtMaterialSetClosestHitProgram( *material, 0, closest_hit_program );
rtMaterialSetAnyHitProgram( *material, 1, any_hit_program );

• Define buffers, variables that might be used inside the supplied programs. Buffers are memory areas that allow host code (i.e. normal CPU code) to communicate with device code (i.e. the code that gets executed on the GPU) and vice versa. Variables are OptiX's internal way of communicating and using buffers to transfer data back and forth.
• Define the OptiX hierarchy of geometry objects, groups, selectors and other nodes to generate a tree graph of the entire scene to be rendered

In order to render a complex scene or trace different paths for any ray OptiX takes advantage of GPGPU computing by exploiting Nvidia CUDA platform. Since the process of shooting rays and setting their behavior is highly customizable, OptiX may be used in a variety of other applications aside from ray tracing.

Starting from OptiX 3.5.0 a second library called OptiX Prime was added to the bundle which aims to provide a fast low-level API for ray tracing - building the acceleration structure, traversing the acceleration structure, and ray-triangle intersection. Prime also features a CPU fallback when no compatible GPU is found on the system. Unlike OptiX, Prime is not a programmable API, so lacks support for custom, non-triangle primitives and shading. Being non-programmable, OptiX Prime does not encapsulate the entire algorithm of which ray tracing is a part. Thus, Prime cannot recompile the algorithm for new GPUs, refactor the computation for performance, or use a network appliance like the Quadro VCA, etc.

• Blender has OptiX support since version 2.81 (7.1 in 2.92) ^[5]
• The Blender Add-on D-NOISE uses OptiX binaries for AI-accelerated denoising^[6]
• FurryBall - Advanced real-time GPU production quality final frame renderer using raytrace as well as rasterize - based on Nvidia OptiX
• At SIGGRAPH 2011 Adobe showcased OptiX in a technology demo of GPU ray tracing for motion graphics.^[7]
• At SIGGRAPH 2013 OptiX was featured in Pixar's realtime, GPU-based lighting preview tool.
• OptiX has been integrated into the GameWorks developers library along with PhysX and other CUDA powered graphics engines and frameworks.^[8]
• Adobe After Effects CC^[9]
• Daz Studio had OptiX Prime Acceleration since its Iray integration, however support was removed in version 4.12.1.8^[10]
• Luxrender 2.5: up to 600% acceleration ^[11]

• CUDA
• Nvidia RTX

• NVIDIA OptiX Application Acceleration Engine main page
• OptiX Programming Guide and OptiX SDK
• OptiX support Forum
• OptiX 7 Course Tutorial Code