Q: What is the difference between a texture and a window and a screen?

A: A screen corresponds to a physically connected display device. Screen 0 is the display with the Dock and menu bar. Screen 1 (on a dual-display system) is the 2nd display, and so on... The Screen('Screens') command returns a vector of all available screens – the available, connected and powered on display devices.

If you have a dual-display setup whose displays are switched into mirror-mode (both displays show same content), then this will appear to PTB as a single screen. The screen-number just defines on which physical output device you want to open/show your stimulus window.

An onscreen window is used to display your stimulus. Each screen can have exactly one full-screen onscreen window – or no PTB window at all.

All drawing commands expect a handle to an onscreen window so they know where to draw to. Onscreen windows are double-buffered: They consist of a backbuffer and a frontbuffer. The backbuffer is where all drawing operations are performed – where you build/prepare your stimulus for presentation. The frontbuffer is scanned out and displayed by your display - your beamer, flat-panel or CRT at the display refresh interval. After you've finished drawing and your stimulus image is ready in the backbuffer, you issue the Screen('Flip',...) command. This command asks PTB to switch the role of the front- and back- buffer at the next vertical retrace of your display device. This way, the former backbuffer becomes the new frontbuffer and your new stimulus gets shown to the subject. The former frontbuffer becomes the new backbuffer and is ready for drawing the next stim. As the bufferswap is synchronized with vertical retrace and always happens in less than 1 microsecond, you'll get perfect, flicker-free, tear-free stimulus presentation timing, regardless how long drawing takes – Unless obviously you try to draw something too complex in a too short period of time.

The important point is that Flip decouples drawing (with its timing variabilities) and actual stimulus presentation - something not possible in the Mac OS 9 and Win toolboxes.

Offscreen windows are the same as Textures in the Psychtoolboxes released since 2006. Internally they are the same thing, so you can always use a Psychtoolbox texture where you would use an Offscreen window and an offscreen window where you would use a texture. They only differ in creation and performance: Textures are created by Screen('MakeTexture') from a Matlab image matrix or automatically by the movie playback engine or the video capture engine. Creating them is very fast and efficient. They are also optimized for fastest possible drawing. Offscreen windows are created by the call Screen('OpenOffscreenWindow').

They are initialized with a selectable size, color depth and initial uniform color, not with image content. They are optimized for fast drawing into them – each Screen drawing command can not only draw to onscreen windows, but also into offscreen windows. After creation, they are (nearly) synonymous.

Say you have a image matrix I in Matlab, e.g., created via I = imread('myimage.png').

Then

tex = Screen('MakeTexture', win, I);

is the same as

tex = Screen('OpenOffscreenWindow', win);
Screen('PutImage', tex, I);

In just that the former is much faster than the latter. The advantage is also that the Screen('DrawTexture') command permits to scale, zoom, filter and rotate textures while drawing.

Texture drawing is extremely fast as it uses specialized hardware on your graphics card, and much more...

PTB Textures and offscreen windows are nearly the same. Both are OpenGL textures internally and treated the same by all texture related functions, that's why you can pass a texture handle into CopyWindow or an offscreen window handle into DrawTexture. The only difference is in their creation and initialization. One can create OpenGL textures in many different formats, optimized for different purposes. PTB initially creates a texture in a format that is optimized for the typical purpose of the texture, given the function you used to create it, trying to maximize speed and minimize memory consumption for a given purpose. If PTB discovers that the current format of a OpenGL texture is not suitable for an operation, it (should) convert it into a suitable format on-the-fly.

Conversion is (sometimes) an expensive operation, so its best to create a texture in a way that best matches its later usage to avoid the conversion step.

PTB assumes the texture is usually only used for drawing it quickly, not drawing into it: The texture is created as either a luminance, or RGBA texture, depending if the Matlab input matrix is a grayscale (1 layer, 2D) matrix or something bigger (> 1 layers, 3D). Image data is stored in Matlab’s column major order, reducing the execution time of MakeTexture by up to a factor of 10. Storage is 8 bits per color or luminance component. There are optional flags to MakeTexture to create textures of different formats, e.g., high dynamic range textures with 16 bit or 32bit floating point precision per color component.

• Fast creation, minimal memory usage, fast drawing and copying.
• When you try to draw into it the first time or apply an image processing operation, PTB will reformat the texture to OpenGL row-major image format and convert pure luminance textures into RGBA textures to make them suitable as rendertargets. Expensive, depending on the size and format of the texture and your gfx-hardware.

Textures from Quicktime movie playback Screen('GetMovieImage') or video capture engine Screen('GetCapturedImage'): They are created as luminace, RGB or YUV textures (the latter one is optimized for typical video data, used by some movie file formats for better compression). The images are sometimes (some movie formats) upside down. Resolution always 8 bits per component.

• Optimized for fast creation, fast drawing copying.
• When drawing into them, PTB applies conversion (flipping upside if needed, converting luminance and YUV to RGB if needed).

These "textures" are always meant for drawing into them, therefore they are always created in a suitable format with RGBA channels of either 8bpc precision or - at request - 16 bpc or 32bpc floating point precision. They are always formatted in OpenGL’s image layout (row major) and filled with their background color by a routine that is optimized for that.

• Optimized for fast filling with background color, fast drawing, fast copy, fast image processing, drawing into them. Never a need to apply any conversions.
• Filling them with Matlab image matrices via the slow 'PutImage' would be drastically slower than using 'MakeTexture'.

No conversion or optimization, PTB assumes the user of these functions to be an expert, wrong usage will lead to error or crash.   So when only drawing into a texture, start off with an offscreen window, when only drawing it, start with MakeTexture, when you need to do both, try both alternatives, depends on the gfx-hardware and cpu which method is faster for your case.

By the way, "expensive" on modern graphics hardware means something like 1 - 5 msecs, not 20-100 msecs, so expensive is a pretty relative term.