The end of sound

[build 0114] I have reached the end of sound - so the files with sound (WAV and MP3) generated on the PC open and publish correctly.

The next step is the user interface. However before that I will go back over the new code I have added and do a little cleaning and re-factoring. I am not planning anything big - just making sure that things like names are consistent, that the code is commented and documented. This sort of thing, though it takes time, pays off in the long run.

Sound - nearing the end

I think I am nearing the end of sound. I have created a new class that is heavy based on Apple's SCAudioCompress sample code and the existing MFC WAV to PCM converter class. So what I have got is more-or-less a drop in replacement for the old class.

The approach I took was to take Apple's SCAudioCompress sample code and transformed it bit-by-bit, into the form I wanted, taking fairly small steps. I took the code that I had crudely hacked into place and transformed it into the Class I wanted and properly integrated it into Cello. While transforming I continually tested the code making sure that it worked after the transformations - I did this by creating a number test files with sounds. I used some of the sounds at Wave File Sample page as samples.

I am not quite there one or two transformations left, some testing and documentation - but my guess is that it should, I hope, fall out in a couple of evenings.

Sound - Debugging

The biggest drawback for me when working with sound is that I don't know that much what I am doing. What I mean is that I have spent many years, for example, working with graphics, transformations etc. and I know the language, terminology and have encountered many problems. The upshot of this is that when working with graphics I have a context to put things into and looking through the code it "makes sense", as do the problems. With sound everything is new. In this regard OSX is basically as unfamiliar as MFC.

My one good stoke of luck is, as ever, that the code is beautfully written and well commented. From the code I know that WAV files are converted to PCM. These files are then consumed by Cello's own ADPCM converter and appear in the final output file.

Having looked through Apple's sample there is a sample SCAudioCompress command line tool that converts sounds. It also has an optional UI that means it is posible to play around with settings. Hacking this in to a class that was based on the original WAV source was not hard. My approach was to just get it in to see if I was on the right track. Once I had something working there would be an opportunity to refine.

My first test file was a short movie with a phrase from Peter Griffin (from "Family Guy").

Adding the code was not too bad. The existing tool wrote a file and loading a buffer wit the contents of a file is quite straight forward. The results were disappointing - the first sound that Cello published was short and unintelligible. Not understanding much about sound I felt a certain amount of despair - where to start?

To debug it I compared the two builds in the debugger. Running Visual Studio under VMware means that I can look at the two builds side-by-side. I also compared the two log files that are created during publishing. My discovery was that the OSX build outputted roughly a 10th of the sound than the PC build. The bug was that in calculating the duration of sound I was out by a factor of 8 - the difference between 8 bits and 1 byte. After fixing this the sound appeared. The unintelligibility of the first part of the sound was due to he strange form of Peter Griffin's voice. If I had chosen a voice like Brian - my life would have been less fraught.

QTKit and boost

I have started working on sound. Somewhere close to he start of the project I focused on getting the project to compile. The support for WAV files butted straight on to MFC so I compiled it out with a #define _NO_WAV. What it means is to work through this code I can remove the #define and work through the compile errors.

The support for WAV files in Cello is quite simple. WAV is converted to PCM using functions in MFC, and this PCM is then converted into the correct variant by some Cello code. So I need to be able to convert WAV (and other formats) to PCM. Trawling through Apple's sample code the way forward seems to be , as I suspected, to use QuickTime. I last used QuickTime, with any seriousness, about 13-14 years ago so it has changed a little.

The recent change is QTKit. QTKit is the Cocoa API for QuickTime. The new API introduces a new Cocoa object QTMovie that provides functionality like reading movies from a file. The first thing I need to do is mesh this into C++.

I could mix C++ and Objective-C without restriction but I would rather not. There are issues with two incompatible exception models - and I would like to keep the Cello engine as close as to the PC version as possible. Also I would like to write a version of the existing WAV handling class, which works, rather than to mesh in a new mishmash.

My approach, therefor, is to provide a C/C++ interface for the objects that I use, much like Apple have done with CoreFoundation. This is what I have been doing with some of the CoreFoundation objects that have richer Cocoa APIs than C APIs.

What I plan to do is to define a type for the Cocoa QTMovie as is done in CoreFoundation

typedef struct __QTMovie *QTMovieRef;

This means that I can use boost::intrusive_ptr to manage it

typedef boost::intrusive_ptr<__qtmovie> auto_QTMovie;
void intrusive_ptr_add_ref(QTMovieRef p);
void intrusive_ptr_release(QTMovieRef p);

And all I need to do is in the body of the implementation (the .mm file) to cast to a QTMovie

void intrusive_ptr_add_ref(QTMovieRef p)
{
 [(QTMovie *)p retain];
}

void intrusive_ptr_release(QTMovieRef p)
{
 [(QTMovie *)p release];
}

So an example function might look like this:

auto_QTMovie NewMovieFromUrl(CFURLRef url)
{
 NSError *theError = NULL;
 QTMovie *theMovie =
     [[QTMovie alloc] initWithURL:(NSURL *)url error:&theError];
 ThrowIfCFError((CFErrorRef)theError);
 return auto_QTMovie((QTMovieRef)theMovie, false);
}

Canned Effects - complete

[build 0096] I have completed working through all of the "Canned Effects", including the text effects, and they now work under OSX. Once I had completed the various problems with the handling of text it all fell out reasonably quickly. Getting the effects to work was little more than a handle turning exercise - get the files to compile, generate test files and test.

The next step is to support files with sound. My guess is that I will need to use QuickTime - this is more of a hunch than the result of serious analysis.

Understanding Fixes - Debugging

As I work trough the text effects I make test files on the PC version so I can test on the OSX version. The test files are reasonably straight forward - I test the range of settings, use different styling and a mixture of paragraph and character styling. In this I came across a problem. The issue was a word based effect with a change of style within the word resulted in some characters being badly positioned. The screen shot below is an example is the output from the simplest case I could create that demonstrated the problem - the "r" is kicked down below the base-line.
This bug had me floored - it was just very hard to work out why it was not working. Running the PC and OSX versions back to back can be very helpful but the in the case of text and fonts there are a lot of numbers many of which are different on the two platforms. CoreText , for example, gives different ascent and descents for fonts. Much of the math in Cello is integer and CoreText is fully floating. As an example of some or the sorts of differences you can get [even] Arial is different on the PC than then Mac. As an example below is a comparison between the same glyph in Arial on the two different platforms.

Getting nowhere for a while I wrote a list of all the possibilities that might bet the cause problem and worked through them. This got me as far as the point where the offset for positioning a glyph within an effect block was calculated. I discovered that if I abandoned the offset calculation, for the glyph that was failing, and set the offset to 0 (so it was the same offset the other glyph) everything worked!

OK- so this was a natural time to give up the chase and move on - but it really niggled me. I had no idea why forcing the offset to zero got things to work. I could kind of imagine that the offset might be zero in the PC version, or that CoreText was giving me values that were different enough for somehow this to be right or something. Checking on the PC version there was an offset that was different from mine but definitely not zero and not that far away.

Moving on and saying "well sure it works now" was tempting. I have worked for many years with a really excellent programmer who is careful, methodical, and very good. When things happen and start mysteriously to work wants to know why. He also, incidentally, writes some of the best code I have seen. I have learnt a lot from him - I carried on the chase. This morning before work I found the problem - misunderstanding some of the code I was not aggregating separate styles within a common word into the same effect block. This was an important bug to find and my setting the offset magically to zero would have been a bodge that would just have masked the problem.

Text Effects - Debugging rounding errors

The first of the text effects is an effect where glyphs are swapped our randomly. Having fixed the more obvious (large scale) problem in the positioning (as described here) there was an outstanding more subtle bug in the positioning of the effect glyphs.

One of my test files is based on some effect text that is precisely positioned on top of some normal text. When the effect ends the text come to rest so it exactly covers the normal text. In the screen shot below you can see that there is a vertical and horizontal offse that looks a bit like a drop-shadow.These offset problems were, as I suspected, due to rounding errors. The PC version of Cello deals exclusively with integer values when positioning text. CoreText, on the other hand, deals with floating point values. The output format deals with integer TWIPS. The issue with the effects is that Cello carries about bound rectangles for the effects (in this case glyph-by-glyph) that are stored as integers before being multiplied by 20 to become TWIPS.

Once I uncovered the problem (it took a while to work out where it was going wrong) I replaced the CRect (a MFC integer rectangle) with a floating point equivalent. Ultimately I need to go through all of the text calculations and make sure that floating point is used - but this is something for my to-do list rather than for now. At the moment I want things to work - but also the minimum of disruption in getting the effects to build and work.