Source code can be found here: http://qt.gitorious.org/qt-labs/scenegraph
A few months ago, I wrote a series of blogs about the graphics stack in Qt and how to make the most of it. In the results I posted there, we could often see that our OpenGL graphics system did not perform as well as one would expect from a hardware accelerated API. When we look at what games can get out of the same hardware it illustrates the fact even more. It doesn’t mean that it is impossible to get really smooth apps with our current graphics stack. There are enough really nice looking QML demos to prove that, but we just believe that we should be able to do something even better.
After many attempts at improving the OpenGL performance of the imperative drawing model we have today, we’ve decided to take a step back and see what can be achieved with a different model; a scene graph. There already exist a few scene graph systems out there and while we have looked at some of them, we have not based ourselves on any,
because the problem they try to solve are different. For instance, it is rare to draw meshes larger than 50 vertices in a UI, while in games that number is so low as to be almost unheard of. Another reason is that we want to try out things and learn what works and what doesn’t from the Qt perspective.
Before going into details of our scene graph, here is an example that shows some of what we can do with it:
| Photos example, showing our scene graph running on a Nokia N900, shot with another N900. There are about 120 unique image files. It starts out in the Wii-inspired thumb-nail view, then we browse single images with a page turn effect. Finally we zoom in on an image and then return to the main view. |
To get optimal performance from hardware rendering, we must minimize the number of state changes; that is, don’t change textures, matrices, colours, render targets, shaders, vertex and index buffers etc. unless you really need to. Big chunks of data should also be uploaded to video memory once and stay there for as long as the data are needed, rather
than being sent from system memory each frame. To achieve this, we need to know as much as possible about what is going to be rendered.
Our scene graph is a tree composed of a small set of predefined node types. The most significant is the GeometryNode which is used for all visible content. It contains a mesh and a material. The mesh is rendered to screen using the shaders and states defined by the material. The GeometryNode is in theory capable of representing any graphics primitive we have in QPainter.
In addition, we have TransformNode which will transforms its sub-tree using a full 4×4 matrix, meaning it is 3D capable. Finally, we have the ClipNode, which clips its children to the clip it defines.
The tree is rendered using a Renderer subclass which will traverse the tree and render it. Since the tree is composed of predefined types, the renderer can make full scene optimizations to minimize state changes and make use of the Z buffer to avoid overdrawing, etc. The renderer will typically not render the scene in a depth-first or breadth-first manner, but rather in the order that it decides is the most optimal. As this can vary from hardware to
hardware, the renderer can be reimplemented to do things differently, as required on a per-case base.
| The filebrowser example from our source tree. In the example above, I end up scrolling through a directory with 600 files. |
In the example above, we are using the default renderer implementation which will reorder the scene based on materials. All the background gradients will be drawn together, then all the text, etc. This minimizes the state changes, resulting in better performance. The default renderer will also use the Z-buffer, and draw all opaque objects front to back followed by drawing all transparent objects back to front. This minimzes overdraw, especially when we have
opaque pixels “on top”, and again helps to improve performance.
The graph is written to target OpenGL 2.0 or OpenGL ES 2.0 exclusively. We did this because OpenGL (ES) 2.0 is available pretty much everywhere and it provides the right set of features.
In addition to reordering and Z-buffer based overdraw reduction, we also have the benefit that geometry persists from frame to frame. Text is supported as an array of textured quads that are created once, then just drawn with each call. This compares favourably with the QPainter::drawText() function where text is laid out and converted to textured quads for every call.
Since we are targeting OpenGL (ES) 2.0 we can expose shaders as a core part of the API. This opens up a number of features that are difficult to make fast in our current QPainter stack.
| The particles example, illustrating how it is possible to do a particle system that runs primarily on the GPU. |
| The metaballs example, illustrating how its possible to do custom fragment processing. |
As we’re only interested in the graphics component, our example code implements animations and input in and ad-hoc manner, as these are not within the our scope.
One might question were QPainter fits with this. The answer is that it doesn’t – not directly anyway. There is an example, paint2D, which shows one way of integrating the two. Another way would be to paint directly into the context of the scene graph before or after the scene graph renders.
So if you find this interesting, feel free to check out the code and play around with it. It’s rather sparse on documentation for now, but we’ll improve on that as time passes.
Enjoy!
Possibly related posts:
21 comments
That’s very cool. Looking forward to see that in 4.8.
Looks great! Thanks for sharing this.
dubik: This is not 4.8 material, it is ongoing research. We will see where it fits when we are a bit further down the road.
This is exactly what we need, keep on rocking!
very nice indeed
using the powervr is an exceptional way to get performance and it seems to work well for particle fields especially
the image grid, i wonder how would it cope on a real device shall look when i get a chance.
I ask because I currently have ~1500 photos.
nice to see thumbnailing code being worked on.
a cross platform qt api to access thumbs on different systems would be a positive step without each app remaking the wheel.
Finally.
Qt Graphics module is and was unfortunately not up to the job.
The scenegraph is and was the right way to go.
Yes, looks very good! Waiting to see more on this..
In the meantime, please keep on writing these graphics related blogs whenever you have time for it. Topics such as performance tuning of QGraphicsView rendering/item handling etc and an explanation of what to consider when designing and coding QGraphicsView programs would be of great value! There are very little Qt specific explanations around there of what actually affects which other things performance vice and which of the vast amount of different ways to do the same thing would result in faster rendering/less memory/smaller size etc. Reading the code has revealed some of these things, but everything is not always so obvious.
Another thing you briefly touched is the specific requirements games present. A UI on a desktop machine forgives lots of bad/slow graphics coding, but a game on a mobile need to squeeze every cycle out of the hardware. What we’d definitely need is some kind of forum subsection somewhere (coming Qt developer network forum maybe?) to discuss game specific optimizations and tricks, a place to share ideas and solutions. JavaME has many such forums and communities, but Qt does not have yet.
This is defently the right way to go. I cant await until this is in Qt.
@lcuk: That image grid video was shot on an n900, which is by all definitions and purchase-ability a real device:
http://www.amazon.com/dp/B002OB49SW?tag=tibesti-20
Pull out your wallet and live the reality!
gunnar,
IMHO, many developers (like me), would love to see Qt supporting integration with Ogre3D.
Ogre3D could be a perfect match for QT; is implemented in C++, is cross-platform and has an excellent architecture behind.
Is there any chance to see this in a near future?
Trying to build another new scene graph sub-library would be rather silly, IMO. Rather use the energy to allow better integration with existing well designed scene graphs, like OpenSceneGraph and Ogre3D. E.g., OSG+Qt works rather well already, but it could use a small integration component which made it easy to integrate out-of-the-box (with full support for multi-threaded, multi and shared-OpenGL-context views).
Thanks anyway for bringing this up.
@DrOctavius, @tylo:
While integration of Qt with OpenSceneGraph and Ogre3D is very interesting, and indeed already done by 3rd parties, I hope you’re not suggesting to use these libraries to drive the user interface of a phone like the N900. I think it makes sense that they look for a solution that makes sense in the current context of using Qt for rich, animated, but mostly 2D user interfaces.
Surely, such a domain specific scene graph designed for enriching the 2D user-interface would be useful indeed, especially if it is optimized for mobile devices. However, once available developers will immediately beg for the whole “package”.
On the other hand, I realize that this also can be done incrementally, in two steps – first establish a basic SG which is well integrated, then incorporate or interface functionality from OSG/Ogre3D type libs.
@tylo: I view this as two different tracks. The one we are on is to solve 2D and 2.5D UIs and make those fast. Integrating the existing solutions with physics engines and 3D object loaders and game scene managers, is catering to a different userbase and and best handled by groups that have an interest in that. There are already Ogre3D integration (http://www.ogre3d.org/wiki/index.php/QtOgre) and OpenSceneGraph integration (http://www.openscenegraph.org/projects/osg/browser/OpenSceneGraph/trunk/examples/osgviewerQT)
Thanks for the clarification, gunnar. Yes, you can cut and paste from the OSG example code, and add gluing code. I do acknowledge that providing full seamless integration with e.g. OSG is a matter for the OSG community itself, if by any. I’m looking forward to trying out your scene graph.
Users of Openscenegraph (OSG) are putting a lot of effort into making the Qt integration better. Also a scenegraph is going to need 3d file format import/export, manipulators, CAD/GIS routines and more to make it useful – it would make more sense to adopt an existing full funtion scenegraph rather than build your own minimal one.
The OSG community has made great strides in creating a good toolset (osgQt) for creating applications using Qt UI and OSG as 3D Scene Graph. However, some Qt/GL context/threading issues remain. E.g.: http://forum.openscenegraph.org/viewtopic.php?t=5717
Any Qt trolls interested in contributing?
Hello,
the project can not be build with Qt 4.6.2. Is this intenional? What Version of Qt do you use? I had following errors:
Reading F:/uwe/prj/qt-labs-scenegraph/benchmarks/throughput/throughput.pro
WARNING: Failure to find: coreapi/renderorderassigner.cpp
WARNING: Failure to find: coreapi/renderorderassigner.h
WARNING: Failure to find: coreapi/renderorderassigner.cpp
WARNING: Failure to find: coreapi/renderorderassigner.h
Beendet mit Rückgabewert 0.
Starte: C:/Qt/2010.02.1/mingw/bin/mingw32-make.exe -w
mingw32-make: Entering directory `F:/uwe/prj/qt-labs-scenegraph’
cd src/ && C:/Qt/2010.02.1/mingw/bin/mingw32-make -f Makefile
mingw32-make[1]: Entering directory `f:/uwe/prj/qt-labs-scenegraph/src’
c:/Qt/2010.02.1/mingw/bin/mingw32-make -f Makefile.Debug
mingw32-make[2]: Entering directory `f:/uwe/prj/qt-labs-scenegraph/src’
g++ -c -g -frtti -fexceptions -mthreads -Wall -DUNICODE -DQT_LARGEFILE_SUPPORT -DQT_NO_EGL -DSCENEGRAPH_LIBRARY -DQT_DLL -DQT_OPENGL_LIB -DQT_GUI_LIB -DQT_CORE_LIB -DQT_THREAD_SUPPORT -I’c:/Qt/2010.02.1/qt/include/QtCore’ -I’c:/Qt/2010.02.1/qt/include/QtGui’ -I’c:/Qt/2010.02.1/qt/include/QtOpenGL’ -I’c:/Qt/2010.02.1/qt/include’ -I’coreapi’ -I’convenience’ -I’renderers’ -I’c:/Qt/2010.02.1/qt/src/3rdparty/harfbuzz/src’ -I’c:/Qt/2010.02.1/qt/include/ActiveQt’ -I’debug’ -I’c:/Qt/2010.02.1/qt/mkspecs/win32-g++’ -o debug/layeredrenderer.o coreapi/layeredrenderer.cpp
In file included from coreapi/layeredrenderer.h:45,
from coreapi/layeredrenderer.cpp:42:
coreapi/renderer.h:49:24: error: qglpainter.h: No such file or directory
In file included from coreapi/node.h:46,
from coreapi/renderer.h:52,
from coreapi/layeredrenderer.h:45,
from coreapi/layeredrenderer.cpp:42:
coreapi/material.h:48:20: error: qarray.h: No such file or directory
In file included from coreapi/renderer.h:52,
from coreapi/layeredrenderer.h:45,
from coreapi/layeredrenderer.cpp:42:
coreapi/node.h:51:29: error: qglvertexbuffer.h: No such file or directory
coreapi/node.h:52:28: error: qglindexbuffer.h: No such file or directory
In file included from coreapi/renderer.h:52,
from coreapi/layeredrenderer.h:45,
from coreapi/layeredrenderer.cpp:42:
coreapi/node.h:164: error: ‘QGL::DrawingMode’ has not been declared
coreapi/node.h:164: error: ISO C++ forbids declaration of ‘QGLVertexBuffer’ with no type
coreapi/node.h:164: error: expected ‘,’ or ‘…’ before ‘&’ token
coreapi/node.h:171: error: ISO C++ forbids declaration of ‘QGLIndexBuffer’ with no type
coreapi/node.h:171: error: expected ‘,’ or ‘…’ before ‘&’ token
coreapi/node.h:172: error: ISO C++ forbids declaration of ‘QGLIndexBuffer’ with no type
coreapi/node.h:172: error: expected ‘;’ before ‘&’ token
coreapi/node.h:174: error: expected ‘;’ before ‘void’
coreapi/node.h:174: error: ISO C++ forbids declaration of ‘QGLVertexBuffer’ with no type
coreapi/node.h:174: error: expected ‘,’ or ‘…’ before ‘&’ token
coreapi/node.h:175: error: ISO C++ forbids declaration of ‘QGLVertexBuffer’ with no type
coreapi/node.h:175: error: expected ‘;’ before ‘&’ token
coreapi/node.h:177: error: expected ‘;’ before ‘void’
coreapi/node.h:177: error: ‘QGL::DrawingMode’ has not been declared
coreapi/node.h:178: error: ‘DrawingMode’ in namespace ‘QGL’ does not name a type
coreapi/node.h:185: error: ‘DrawingMode’ in namespace ‘QGL’ does not name a type
coreapi/node.h:186: error: ‘QGLVertexBuffer’ does not name a type
coreapi/node.h:187: error: ‘QGLIndexBuffer’ does not name a type
coreapi/node.h: In member function ‘void BasicGeometryNode::setGeometry(int, int)’:
coreapi/node.h:167: error: ‘vertexBuffer’ was not declared in this scope
coreapi/node.h:168: error: ‘indexBuffer’ was not declared in this scope
coreapi/node.h: In member function ‘void BasicGeometryNode::setDrawingMode(int)’:
coreapi/node.h:177: error: ‘m_mode’ was not declared in this scope
coreapi/node.h: At global scope:
coreapi/node.h:207: error: ISO C++ forbids declaration of ‘QGLVertexBuffer’ with no type
coreapi/node.h:207: error: expected ‘,’ or ‘…’ before ‘&’ token
coreapi/layeredrenderer.cpp: In function ‘bool nodeLessThan(GeometryNode*, GeometryNode*)’:
coreapi/layeredrenderer.cpp:64: warning: suggest explicit braces to avoid ambiguous ‘else’
coreapi/layeredrenderer.cpp: In member function ‘void LayeredRenderer::renderNodes(const QLinkedList&)’:
coreapi/layeredrenderer.cpp:234: error: variable ‘QGLPainter painter’ has initializer but incomplete type
coreapi/layeredrenderer.cpp:338: error: ‘class GeometryNode’ has no member named ‘vertexBuffer’
coreapi/layeredrenderer.cpp:342: error: ‘class GeometryNode’ has no member named ‘vertexBuffer’
coreapi/layeredrenderer.cpp:347: error: ‘QGLAttributeValue’ was not declared in this scope
coreapi/layeredrenderer.cpp:347: error: expected ‘;’ before ‘value’
coreapi/layeredrenderer.cpp:348: error: ‘value’ was not declared in this scope
coreapi/layeredrenderer.cpp:392: error: expected initializer before ‘&’ token
coreapi/layeredrenderer.cpp:393: error: ‘indexBuffer’ was not declared in this scope
coreapi/layeredrenderer.cpp:398: error: ‘class GeometryNode’ has no member named ‘drawingMode’
coreapi/layeredrenderer.cpp:402: error: expected initializer before ‘
uwehoeppner, try Qt 4.7 as file COMPILING.txt in folder scenegraph instructs. Hopefully this helps.
This scenegraph looks very promising!
Unfortunately I still cannot build it. (I’m entirely new to the Qt world; sorry!)
I built Qt 4.7 from git with ./configure -opengl (using gcc version 4.2.1 Apple Inc. build 5659)
Qt3D works flawlessly and yet I get the following errors when running make:
convenience/textmaskmaterial.cpp:49:45: error: private/qtextureglyphcache_gl_p.h: No such file or directory
convenience/textmaskmaterial.cpp:50:35: error: private/qfontengine_p.h: No such file or directory
convenience/textmaskmaterial.cpp:51:37: error: private/qglextensions_p.h: No such file or directory
convenience/textmaskmaterial.cpp: In member function ‘virtual void TextMaskMaterialData::activate(QGLPainter*)’:
convenience/textmaskmaterial.cpp:142: error: ‘qt_resolve_version_2_0_functions’ was not declared in this scope
convenience/textmaskmaterial.cpp:143: error: ‘qt_resolve_buffer_extensions’ was not declared in this scope
convenience/textmaskmaterial.cpp: In member function ‘void TextMaskMaterial::init()’:
convenience/textmaskmaterial.cpp:157: error: ‘QFontEngineGlyphCache’ has not been declared
convenience/textmaskmaterial.cpp:157: error: expected `;’ before ‘type’
convenience/textmaskmaterial.cpp:161: error: invalid use of incomplete type ‘struct QFontEngine’
…and so on.
Any help in the right direction is greatly appreciated.
Does not build against the 4.7 tree anymore:
The current tree does not build against Qt 4.7 because of QGLTextureGlyphCache dependency issues, unlike what COMPILING.txt says.
cl -c -nologo -Zm200 -Zc:wchar_t- /QRfpe- -DDEBUG -D_DEBUG -Zi -MDd -Gy -EHs-c- -W3 -DUNDER_CE -DWINCE -D_WINDOWS -D_UNICODE
-DUNICODE -D_WIN32 -DQT_NO_PRINTER -DQT_NO_PRINTDIALOG -DARMV4I -D_ARMV4I_ -Darmv4i -D_ARM_ -DARM -D_M_ARM -DARM -D__arm__ -DQ_OS_W
INCE_WM -DQT_NO_PRINTER -DQT_NO_PRINTDIALOG -D_WIN32_WCE=0×502 -DQT_WINCE_GESTURES -DSCENEGRAPH_LIBRARY -DQT_DLL -DQT_OPENGL_LIB -DQ
T_GUI_LIB -DQT_CORE_LIB -DQT_THREAD_SUPPORT -DQT_NO_DYNAMIC_CAST -I”..\..\4.7jtl\include\QtCore” -I”..\..\4.7jtl\include\QtGui” -I”.
.\..\4.7jtl\include\QtOpenGL” -I”..\..\4.7jtl\include” -I”coreapi” -I”convenience” -I”c:\Qt\scenegraph\src\renderers” -I”..\..\4.7jt
l\src\3rdparty\harfbuzz\src” -I”..\..\4.7jtl\include\Qt3D” -I”..\..\4.7jtl\include\ActiveQt” -I”.moc” -I”..\..\4.7jtl\mkspecs\wincew
m65professional-msvc2008″ -Fo.obj\ @C:\Users\jtlee\AppData\Local\Temp\nm907D.tmp
textmaskmaterial.cpp
convenience\textmaskmaterial.cpp(233) : error C2039: ‘context’ : is not a member of ‘QGLTextureGlyphCache’
c:\qt\4.7jtl\include\qtopengl\private\../../../src/opengl/gl2paintengineex/qtextureglyphcache_gl_p.h(65) : see declaration o
f ‘QGLTextureGlyphCache’
convenience\textmaskmaterial.cpp(233) : error C2039: ‘context’ : is not a member of ‘QGLTextureGlyphCache’
c:\qt\4.7jtl\include\qtopengl\private\../../../src/opengl/gl2paintengineex/qtextureglyphcache_gl_p.h(65) : see declaration o
f ‘QGLTextureGlyphCache’
convenience\textmaskmaterial.cpp(238) : error C2039: ‘context’ : is not a member of ‘QGLTextureGlyphCache’
c:\qt\4.7jtl\include\qtopengl\private\../../../src/opengl/gl2paintengineex/qtextureglyphcache_gl_p.h(65) : see declaration o
f ‘QGLTextureGlyphCache’
convenience\textmaskmaterial.cpp(239) : error C2039: ‘setContext’ : is not a member of ‘QGLTextureGlyphCache’
c:\qt\4.7jtl\include\qtopengl\private\../../../src/opengl/gl2paintengineex/qtextureglyphcache_gl_p.h(65) : see declaration o
f ‘QGLTextureGlyphCache’
convenience\textmaskmaterial.cpp(241) : error C2039: ‘fillInPendingGlyphs’ : is not a member of ‘QGLTextureGlyphCache’
c:\qt\4.7jtl\include\qtopengl\private\../../../src/opengl/gl2paintengineex/qtextureglyphcache_gl_p.h(65) : see declaration o
f ‘QGLTextureGlyphCache’
textnode.cpp
convenience\textnode.cpp(71) : error C2039: ‘glyphs’ : is not a member of ‘QTextLayout’
c:\qt\4.7jtl\include\qtgui\../../src/gui/text/qtextlayout.h(104) : see declaration of ‘QTextLayout’
glyphnode.cpp
convenience\glyphnode.cpp(16) : error C2039: ‘get’ : is not a member of ‘QFontPrivate’
c:\qt\4.7jtl\include\qtgui\private\../../../src/gui/text/qfont_p.h(157) : see declaration of ‘QFontPrivate’
convenience\glyphnode.cpp(16) : error C2027: use of undefined type ‘QGlyphs’
c:\qt\scenegraph\src\convenience\glyphnode.h(6) : see declaration of ‘QGlyphs’
convenience\glyphnode.cpp(16) : error C2228: left of ‘.font’ must have class/struct/union
convenience\glyphnode.cpp(16) : error C3861: ‘get’: identifier not found
convenience\glyphnode.cpp(30) : error C2027: use of undefined type ‘QGlyphs’
c:\qt\scenegraph\src\convenience\glyphnode.h(6) : see declaration of ‘QGlyphs’
convenience\glyphnode.cpp(30) : error C2228: left of ‘.glyphIndexes’ must have class/struct/union
convenience\glyphnode.cpp(30) : error C2027: use of undefined type ‘QGlyphs’
c:\qt\scenegraph\src\convenience\glyphnode.h(6) : see declaration of ‘QGlyphs’
convenience\glyphnode.cpp(30) : error C2228: left of ‘.positions’ must have class/struct/union
Generating Code…
NMAKE : fatal error U1077: ‘”C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\ce\bin\x86_arm\cl.EXE”‘ : return code ’0×2′
Stop.
NMAKE : fatal error U1077: ‘”C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\nmake.exe”‘ : return code ’0×2′
Stop.
NMAKE : fatal error U1077: ‘cd’ : return code ’0×2′
QGLTextureGlyphCache functionality does not exist in
Comments on this entry are closed.