The QtMobility family is growing once again with the addition of the Multimedia Framework. QtMobility’s media library will enable developers to easily create applications that make best advantage of the system media capabilities.
The Multimedia framework is designed to cover the items of most direct interest to developers; playback, recording, play-list and meta-data management – all the things you’d expect from a high-level media framework. All integrated nicely into Qt. We’ve taken care to strike a nice balance among simplicity, versatility and extensibility; and the result is a framework that is easy to get started with – while allowing such niceties as transparency in the locality of playback, and that is readily customizable – create your own playback or recording services.
If you are looking at using a Multimedia framework in your application and would like to investigate further, a good place to start is QMediaPlayer and QMediaPlaylist. With these two classes you’ll easily be able to create a simple media player application. You could even extend further and use QMediaMetadata to fetch cover-art and track info, a complete music player is just three classes away, well, almost 
.
This is not the finished product, there will be changes as we take it to the best place it can be, so if you have any comments don’t be shy. For testing; currently only Linux is supported as a platform, you’ll need GStreamer installed. You might also be able to have a little fun on Windows. In the future we should see implementations for all our supported Macintosh OSX and Windows platforms, Phonon on KDE, and in the future; Symbian and Maemo.
You can find the source on Gitorious. Note you will need Qt 4.6 to compile the project. Feel free to comment here or if you’d like you can send an email to qtmobility at trolltech.com.
Possibly related posts:
39 comments
So you’re creating an abstraction layer – QtMobility/Multimedia – on top of an abstraction layer – Qt Phonon – to avoid pissing off the Maemo guys by throwing away the pointless (due to Phonon) MAFW?
Could you comment on the relationship between the multimedia framework and Phonon? How are they the different/the same? How they will work together (or won’t)?
i believe this api is taken from the QtExtended media api…
So is this an API to handle multimedia? Like playlists etc.
While Phonon is just an API for playing the media?
WTF? This is bull. First you proudly announce “Hey, Phonon is sooooo great. It works on any platform. We’ll use it as our multimedia API and now this sh*t??”!??!!!”
It really does not seem to make much sense to introduce other multimedia api’s on qt, instead of working on phonon, especially because it’s not ‘phonon on kde’ but ‘phonon on qt’.
Can you please explain what these classes are supposed to do, and what are their relations with Phonon and the QtMultimedia module introduced in 4.6?
I add myself to the crowd eager to know more about the future of Phonon and its relationship with this new multimedia framework. Thank you.
hmmm… what *is* going on? Phonon development doesn’t seem to have been too fast over the last few years… is this ultimately going to replace phonon? Is there a roadmap anywhere for QtMultimedia and phonon?
Count me in. Why don’t you use and develop Phonon?
This reeks for NIH (Not-Invented-Here).
I see nothing in the QtMultimedia API that either isn’t in Phonon already (either in the stable or experimental API), or which couldn’t easily be added.
Right hand talk to left hand much?
Could it be that Phonon just doesn’t have a “qt-like” API, and they are using this to wrap it (and potentially other) media back-ends?
I very much concur with the other commenters. This multimedia layer very much smells NIH. That said it looks like quite some parts of this QtMobility effort are about NIH. It was reinventing the wheel for hardware detection and access, now doing the same for multimedia… It’s really not what people are used to from the trolls, they used to be more aware of other efforts.
I don’t understand the link between the “QtMobility’s media library” and the “QtMultimedia” of Qt 4.6.
QtMultimedia already seems to be duplicating Phonon with much less features (QAudioOutput only support Alsa on Linux, not even able to play a mp3, come on Trolls, we are used to better than that). It looks like you have no idea of where you are going on the multimedia framework.
“Could it be that Phonon just doesn’t have a “qt-like” API”
Definitely not this: it’s API is perfectly Qt-ish – in fact, I remember a period of API-review at the (then) Trolltech offices when it was still a KDE sub-project prior as a pre-requisite to inclusion in Qt itself.
There’s been a recent spate of posts on labs.trolltech.com that raise more (obvious, and sometimes worrying) questions than they answer, sending readers into a frenzy, and I have to wonder what is going on internally to have triggered this. I’m sure the gstreamer guys (who have been quite vocal in their criticisms of Phonon from the outset) are having a field day with this one
What is up with QtMobility? It is its own blogger (whereas every other blogger is and identifiable human), there is no responding to comments. The writing style of whomever writes for QtMobility is difficult hand hard to follow, the concepts are poorly explained. Then there is the mobility concept itself. Yes, the GPS data makes sense… but why a do you need yet another Multimedia framework? I’ve complained about QtMobility’s blogs before. Nothing has changed.
The Multimedia Module makes perfect sense to me, and I was expecting a separate set of classes for low level manipulation of video/audio files.
Writing an application to playback video or writing an application to edit frames from a video file has different requirements that go as deep as handling the video file. Capture single frames from a video file does not require loading frames to memory in advance, directly to graphics hardware like a video player does. Thats what phonon does.
@hugo: in the latest Phonon (from SVN), using Phonon-Xine (and almost Phonon-GStreamer), there is support for handling individual frames from video streams (it’s used by for example the video file thumbnailer in KDE).
There’s also support for exporting raw audio frames in Phonon-SVN.
Any more missing functionality should be pretty easy to add, Phonon is extremely flexible.
@Scorp1us: It has only been one day, and QtMobility is developed in Australia, so you will have to give the multimedia developers a chance to wake up, read the comments, and respond. Which I am sure they will do.
Perhaps some of you should grab the sources, read the docs.
Honestly, I have to agree with Phonon’s API being un-Qt. It was very obviously developed by a “standard C++” die-hard that had more experience working with the audio backends than developing friendly APIs. It’s a country mile better than any alternative I’ve looked at but it doesn’t “fit in” with the API of the rest of Qt.
QtMultimedia contains low-level APIs, it’s code meant for building media services on, it’s not a playback API. You could use QAudioOutput to play mp3s and given circumstances that would be sensible, but often its just easier to use something that handles everything for you QMediaPlayer style.
The Mobility media API has a lot more than plaback support; it supports playlist management, recording, radio, comprehensive metadata, and soon camera and others.
Phonon is still on our minds and it’s still supported for playback.
Phonon continues to play a role in the future QtMultiMedia framework, offering Playback functionality where you may decide it is the prefered choice and will of course, in any event, continue to be supported for the entire Qt 4 series. The new QtMultimedia APIs were designed after much consideration of developer needs for extensibility, much more functionality (not least Video/Audio Capture) and of course plaform flexibility. The decision to build something new rather than completely re-architect Phonon was something the Qt development team thought long and hard about. But a new architecture is seen as the best solution to meet developer need and we welcome your feedback on the software provided.
I am a bit confused
I thought I read that 4.6 is frozen. And this appears to be a part of 4.6. And it’s linux only? After all it seems to be providing metadata, playlist management and the like – features that ought to be cross platform. Why not wait till it’s complete?
@ Girish.
Just to be clear the New APIs introduced here are not part of Qt4.6.
The new APIs can be tested using Qt4.6, but at this stage they are quite independant from that release.
@ Gerard, oops, my bad. I was looking at http://doc.trolltech.com/4.6-snapshot/qtmultimedia.html and jumped to (random) conclusions.
This blog posting is about Mobility APIs.
The APIs you see on http://doc.trolltech.com/4.6-snapshot/qtmultimedia.html are new Multimedia APIs, to be included in Qt 4.6, that provide the lowest-level functionality in Qt to support those Qt Mobility APIs (which are themselves NOT in Qt 4.6).
Of course, most Qt Mobility APIs will eventually be integrated into Qt proper, so the things you see in http://qt.gitorious.org/qt-mobility/multimedia certainly take the form of what we expect to see in future Qt versions.
And what Phonon development, it has been stalling for months. Phonon is NOT mature as a playback API. I mean Phonon backends are not mature. Seeking is disabled on HTTP streams on Linux, On Mac the GUI freezes while a HTTP stream is buffering. The buffering progress reporting is completely broken in all backends. Etc.
I reported the first issue and have been told that they knew about it. It was not even accepted in the ticket system. Please deprecate Phonon ASAP if you’re not going to put developers behind it.
I *still* don’t get it. What’s the purpose of these classes? What’s the design of this framework (plugins, classes, etc.)? How is it related with Phonon and QtMultimedia? I see lots of duplicated functionality, so I’d like to read some clarifying words.
It’s great to see Qt getting more multimedia support, and I’m partiularly glad to see new QtMultimedia module in 4.6, but the relationship between Phonon, QtMultimedia and now QtMobility/Multimedia is very unclear. It would be great if someone from Qt could indicate the current and future (roadmap) relationship between these pieces.
Things that arn’t clear include:
1) What’s the relationship between Phonon and QtMobility/Multimedia? Is one for desktop use and the other for Mobile platforms, or will one eventually be dropped in favor of a single “code once, run anywhere” high level media API?
2) What’s the relationship between the device level QtMultimedia API and the higher level play-graph Phonon & QtMobility/Multimedia APIs? Logically one might assume Phonon to be implemented on top of QtMultimedia, but that certainly wasn’t initially the case… is it planned that eventually it will be?
3) While Phonon is a higher level interface than QtMultimedia it still overlaps in functionality given the ability to process/consume an audio/video stream via path processors (effects) or sinks. When is it appropriate to use one API vs the other? For an app interested in “consuming” an audio or video stream (e.g. voice recognition, video conferencing), is it more appropriate to use QtMultimedia or Phonon (or QtMobility/Multimedia if targetting a mobile platform)?
More generally it would also be good to understand the relationship between “Desktop” Qt and QtMobility. It would be highly desireable for Qt to maintain the “code once, run anywhere” paradigm so that something like, say, a video chat app doesn’t need to choose whether it’s targetting Qt or QtMobility in areas where there seem to be competing APIs (Phonon vs QtMobility/Multimedia).
One more question:
From looking at the source, the new Qt 4.6 QtMultimedia API is initially supported on Linux (ALSA), Windows and Mac. Will this API eventually be supported on all (incl. mobile) targets?
Some good questions and I can see clarification is indeed needed, so I hope I can answer some of your questions here.
The Qt Mobility project targets bringing many new APIs to Qt and some of those APIs are
certainly also of use outside of ‘Mobile centric’ application development.
The new Qt MultiMedia APIs introduced in this blog fall in that category, as they are of course *also* designed with the needs of desktop application developers in mind.
For convenience sake right now the development is being managed under one general programme titled (the ‘Qt Mobility Project’). But to be fair several of the APIs are equally valuable for desktop developers. (The objective being to provide developers with the ability to run their application accross multiple platforms and indeed multiple device types – ‘Qt Everywhere’ right
.
As the APIs mature they will be considered for inclusion in some future Qt version, and at
that point we may decide that some APIs appropriately reside in a new ‘mobility’ library and
others perhaps in existing more general existing libraries. But we don’t need to make that
decision today. Our current objective is share our initial work progress with you.
To try clear the confusion regarding the difference between the Qt 4.6 MultiMedia APIs and the new Qt APIs shared here:
The Qt 4.6 MultiMedia APIs are a continuance on the old familiar architecture offering media
playback via Phonon. This solution/architecture will be supported and maintained for the
entire Qt 4 series, but no substancial new features will be added.
The newly architected MultiMedia APIs, which this Labs entry is all about, target both desktop and mobile development needs, have an entirely new architecture and will ultimately offer
developers support for Video/Audio capture, Camera APIs, Playlist Management, Playback, Radio and more.
It is intended that they will become part of Qt at some future point once we have completed
development, are happy that the APIs are considered stable/mature and most important that they meet our Qt standard.
So this is your opportunity to try first hand the newly architected multimedia APIs and let us hear your feedback.
Have fun!
quoting Gerard:
“The new QtMultimedia APIs were designed after much consideration of developer needs for extensibility, much more functionality (not least Video/Audio Capture) and of course plaform flexibility. The decision to build something new rather than completely re-architect Phonon was something the Qt development team thought long and hard about.”
Perhaps it would have been worthwhile to have opened the blog with this information. I’m still not clear on what is to happen to phonon; is it to be extended to support e.g. capture (as we’ve been told it would be at the last two dev. days…)? The slow rate of development of phonon and the introduction of these new QtMultimedia classes suggest to me that it will not be developed further, but please correct me if I’m wrong.
Incidentally, it would have been great to have been given some kind of a heads up on this; my company is a QtSoftware partner specializing in multimedia…
I hope to have the opportunity to speak to some developers at the Munich dev. day about this work!
I encourage this new architected MultiMedia APIs, because Phonon has never been appealing to me…
The nice thing with Phonon is that it uses a backend system. Will we see a VLC or a Xine backend for this new framework?
Could you explain why you think Phonon is so inferior that a new architecture needs to be created? An architecture that Trolltech helped define not 2 years ago?
All the goals you give for Qt Multimedia are the goals of Phonon. I’m not saying Phonon is perfect, but its a solid base and well designed to be extended with new APIs and features.
@Ian Monroe: As far as I know, Trolltech didn’t design Phonon — Phonon was a KDE project that got integrated into Qt.
@Adam Higerd: Trolltech was involved in designing the Phonon API, through API design/review sessions held at the Trolltech offices.
Pure speculation, but I’d guess that maybe while Phonon was a good match for DirectShow, GStreamer, etc if may not have fit so well on top of whatever multimedia frameworks are available on some of these mobile platforms, so maybe the Nokia aquisition effectively changed the requirements?
I’m curious where the Qt 4.6 QtMultimedia API fits into this? I’m guessing it’s there to fill a gap until the new API replaces Phonon (assuming that’s what’s happening – only indirectly stated), but in that case I wonder what it’s future is? Does it have a direct equivalent in the new API, or will it be retained as-is?
The new audio class’s in 4.6 provide the basic building blocks for audio access at the lowest level. They have nothing to do with phonon or any other multimedia frameworks.
QAudioFormat class provides a way to describe an audio stream.
QAudioDeviceInfo and QAudioDeviceId class’s provide a way to discover audio devices and find out what they can do. QAudioOutput class provides a way to output audio, this could for example be used to create one phonon backend that would work on all Qt’s supported platforms (music playback only).
QAudioInput class provides a way to get audio input from a mic, combine this with the QAudioFormat class and you have all the information to do stream manipulation. Examples of this would be audio graphic equalizer or audio editor.
Without the overhead of a full media framework your response time is alot better so game sounds, ringtones, sound effects can be achieved with minimum delay.
In the future the audio class set will be extended to deal with mixing and volume.
With these class’s is should make it possible to create voip clients, voice recorders, games (cant have a game without sound), audio editors with Qt!
Comments on this entry are closed.