Mounting the board - Cello's first crash.

Cello now crashes. This represents progress as it means that I have come to an end of rounding-up in the great ranch of link errors.


Cello won't do any more than allow you to choose a file - it crashes way before any part of the file reading code is called. So now to work through the crashes - I fully expect this process to pull in more and more files.

I expect that I will have to largely pause my process of Unit Testing - simply because there is nothing discrete to test. When I get to the stage of being able to open simple files I will try and add them to the Cello project as unit tests - the test being little more that the fact that Cello can open the file. The idea behind this is that if/when I break things so that a file that did open no longer opens I want to know.

CGImage & ImageIO - converting premultiplied alpha

The final part of the DIB (bitmap) equation is reading images. Cello has classes for reading and writing PNG, JPEG and a few formats including PaintShop Pro that are prevalent on windows. The change that I have made here is to use Image IO. Image IO will let you read all the most common formats - and also has support for RAW images (as spat by many digital cameras) amongst other things.

ImageIO is really easy to use you just need to give it something - a URL or a Data provider or whatever and it will turn it into a CGImage. The only issue with CGImage is that it is not possible to get to at the pixel data. In Cello there is a fair amount of code that manipulates pixels directly - so I have to facilitate this.

A solution is to draw the image onto a bitmap where you own the pixels - CGBitmapContextCreate will create a bitmap context and the following code fragment will draw it.

   CGRect drawRect = ::CGRectMake(0, 0, width, height);
   ::CGContextTranslateCTM(context.get(), 0.0, float(height));
   ::CGContextScaleCTM(context.get(), 1.0, -1.0 );
   ::CGContextDrawImage(context.get(), drawRect, theImage);

In this there are a couple of wrinkles.

• CGBitmapContextCreate won't work on an indexed bitmap. So all the images will be converted to either RGB32 or RGBA32. I don't see this as a big deal.
• CGBitmapContextCreate will only work with premultiplied alpha. Cello assumes the alpha is not premultiplied. The easiest thing to do is to wholesale convert the bitmap to nom-premultiplied alpha after the fact.

The code for this is reasonably straight-forward:

inline UInt32 Unpremultiply(UInt32 c, Float32 inverseAlpha)
{
UInt32 result = UInt32(Float32(c) * inverseAlpha);
result = std::min(result, 255);
return result;
}

void BitmapClass::UnpremultiplyRGBA()
{
UInt8* pRow = reinterpret_cast(m_pBits);
for (int row = m_height; row > 0; --row)
{
    UInt32* pPixel = reinterpret_cast(pRow);
    for (int column = m_width; column > 0; --column)
    {
        UInt32 alpha = GetAValue(*pPixel);
        if (alpha)
        {
            Float32 inverseAlpha = 255.0/alpha;

            UInt32 red = Unpremultiply(GetRValue(*pPixel), inverseAlpha);
            UInt32 green = Unpremultiply(GetGValue(*pPixel), inverseAlpha);
            UInt32 blue = Unpremultiply(GetBValue(*pPixel), inverseAlpha);

            *pPixel = RGBA(red, green, blue, alpha);
        }
        else
            *pPixel = 0;
        ++pPixel;
    }
    pRow += m_iByteWidth;
}
}

It is, of course, entirely non-optimized - but as this will happen as a single hit when an image is read I don't see it as being any kind of bottle neck. If it turns out to soak up time and that this is a problem I will attack it later.

XCode unit testing and Bundle Resources

In order to test the code that reads images I needed to set up tests that read images. In order to do this repeatably the best thing seemed to be to add them to the bundle of test application. In XCode this is quite simple - I just dragged in a folder of test images into the right place in my source files and XCode magically knows that these are to be copied - updating the copy files phase of the project.
Having done this the first part of my unit test was to get a url of a PNG in the bundle:

 CFURLRef png1 = ::CFBundleCopyResourceURL(
    CFBundleGetMainBundle(),
    CFSTR("DWCGImageSource_100x100a.png"), NULL, NULL);
 CPTAssert(png1);

This test failed immediately. It failed because URL of the main bundle when running the test is /Developer/Tools. Increasingly my calls to core foundation are lightly wrapped and CFBundleGetMainBundle() is a default parameter to my functions that get thr URLs of things relative to bundles. The solution I found was to add a #define _UNITTEST in the global defines and to write a new function to get the bundle.

 inline CFBundleRef GetMainBundle()
 {
 #if _UNITTEST
  return ::CFBundleGetBundleWithIdentifier(
    CFSTR("com.badbase.cello.unittest"));
 #else
  return CFBundleGetMainBundle();
 #endif
 }

I then use this GetMainBundle in the places where I used CFBundleGetMainBundle.

DIB to CGImage - the washing up

I changed the plans and in the end kept the class that manage the DIBs almost unchanged. The DIB class, within Cello, represents a bitmap with a physical bit-image. Give or take the fact that it can handle native compressed bit-images it handles the concept that an image is a bunch of bits and possibly a color table. A CGImage is very similar except that you can't get to the bit-image. You can make a CGImage where you can get to the bits if wrap the the bit image in a CFDataProvider. So the natural thing seemed to keep the DIB class more or less as it was but to provide a way off creating one from a CGImage.

The key change that I have made to the DIB class is to change the way that pixels are stored; I have moved from using a raw pointer for the pixels to boost::shared_array. The boost::shared_array is an array with an owner count - and it means that it is possible to deal with joint ownership. The point being that data provider needs to have ownership of the data it is providing (the pixels) as does the DIB class.

The other part of the equation is the colour palette its self. A colour palette is a list of colours - little more. In Cello there is a class that represents this list of colours as a list of RGBQUAD from which it can create the HPALETTE etc required for MFC bitmaps. I have changed the colour palette class so that it is now a thin wrapper around a CGColorSpace. You can still access a representation of it as a list of RGBQUAD - this is required for other things in Cello such as the ditherer. Changing this code and writing the attendant unit test was quite straight forward. I use boost::intrusive_ptr to manage the CFColorSpaceRef, and boost::shared_array to manage representation (cache) of RGBQUAD. In retrospect I think it would have been better not to have made this change and instead to have just provided a way of converting into a CGColorSpace. It would have been a smaller change.

Moving DIBs (device-independent bitmaps) to CGImage

The biggest remaining clot of link errors (probably about 30 or so) relate to Cello's handing of Bitmaps. There are a handful of classes, many parts of which of which are commented out or omitted from compilation, that deal with bitmaps. Hanging on to this are classes and libraries that deal with reading and writing images to and from physical files. Central to all of this is the class that manages the DIB which I have discovered is a device-independent bitmap. The wikipedia article is an excellent introduction and explained why, when I was reviewing the code, I saw code and logic that dealt with vertically flipping images (it seems that DIBs are an inverted format). There is also a fair amount of wrangling to move DIBs (which represents an actual bit image) between images.

My plan here is to replace the DIBs and indeed the image classes with a thin wrapper around CGImage. Apple's ImageIO framework provides a homogenous way to read and write these images in a variety of file-formats.

Getting the ZLib dylib to link in XCode

With the link errors dropping into the 50s the home straight of something that links, though not close, seems possible. My latest teaser was some problems linking Zlib. The Cello sources include Zlib but as it exists as standard under OSX there seemed little point in compiling it into the application - also I dimly remember that there was some issue with it compiling - not that large but using the built in zlib seemed the easiest option.

Working through the link errors I came to some errors from ZLib. Adding the library libz.dylib is just a matter of dragging it in - but the link errors were not resolved. Using Google I found someone with exactly the same problem on Apple's XCode list mailing list - with a reply from Chris Espinosa but infuriatingly my guess was that the final correspondence between them happened off-list.

I eventually figured out the solution - XCode does not set the file type correctly. Just select the dylib, choose Get Info, and set the file-type.

Implementing MFC CTime using CFDate

In the early stages of Cello I stubbed out the MFC CTime class with little more than a constructor in order to get it to compile. Working my way through the link errors this is something that I needed to flesh out.

My basic plan with implementing these MFC classes is to keep the API but to build them on top of Core Foundation objects. Core Foundation can quite ubiquitously be expressed as NS (Objective C) objects (they are toll-free bridged) which will pay dividends when I start working in cocoa for the UI. Also, and importantly, it is all done, it works and just requires hooking up.

I implemented the CTime class as a wrapper for a CFDateRef managed by a boost intrusive_pointer. Give or take a thin wipe of code and a modest set of Unit Tests that was it.

Implementing SetFileTime and GetFileTime under OSX

Working through the link errors makes for some catching up on some things that I sketched out and did not complete. I have been looking at the file handling class, file streams and writing unit tests to make sure that the they are working properly as I flesh them out.

I have backed Cello file handling class more or less straight onto STL file streams. In this class there are a couple of methods SetFileTime, and GetFileTime. In the MFC version these are more-or-less straight wraps of the API calls SetFileTime, and GetFileTime:

 BOOL WINAPI SetFileTime(
 __in      HANDLE hFile,
 __in_opt  const FILETIME *lpCreationTime,
 __in_opt  const FILETIME *lpLastAccessTime,
 __in_opt  const FILETIME *lpLastWriteTime
 );

Under OSX I have implemented them in terms of FSGetCatalogInfo and FSSetCatalogInfo. So, for example, the blood and guts of the SetFileTime method boils out like this:

 const char *path = (LPCSTR)m_dwsName;
 FSRef fsRef;
 Boolean isDirectory;
 FSCatalogInfo fsCatalogInfo;

 OSStatus err = FSPathMakeRef(reinterpret_cast(path), &fsRef, &isDirectory);
 ::ThrowIfOSErr(err);
 err = FSGetCatalogInfo(&fsRef, kFSCatInfoAllDates, &fsCatalogInfo, NULL, NULL, NULL);
 ::ThrowIfOSErr(err);

 fsCatalogInfo.createDate = *lpCreationTime;
 fsCatalogInfo.contentModDate = *lpLastWriteTime;
 fsCatalogInfo.accessDate = *lpLastWriteTime;

 err = FSSetCatalogInfo(&fsRef, kFSCatInfoAllDates, &fsCatalogInfo);
 ::ThrowIfOSErr(err); 

In my first cut I implemented FILETIME as a CFDate - however looking at FILETIME when writing the test cases I changed this to UTCDateTime which is actually practically the same.

The only wrinkle I have found is that the FSGetCatalogInfo ignore the fractional part in the UTCDateTime - which is something that tripped me in the Unit Tests

CString and Unit Tests

Right at the head of the link errors was the CString class and most of it's methods. Early on in the port I copied the class declaration from MFC, commented out almost all of the methods and then uncommented the ones that as I got compile errors. It is surprising just how few of the class methods are used. At the time I knew that I wanted to implement it as a CFString so I added constructors and accessors so that I could freely convert from CFString.

Implementing the CString methods was fairly straightforward - so much so that I paused before writing any UnitTests, not wanting to waste time.


I retire some hours later with renewed belief in writing unit tests. Many of the tests worked straight off the bat - but I found serious problems in my implementation of CString::Format that would have bitten me later and would have taken longer to fix at a later date. Also using the accesors to get the underlying CFString it has been possible to check that the owner counts of the CFString are being handled correctly - I am using boost::intrusive_pointer for the actual management - but the copy constructor and assignment operations should share the underlying pointers. Unit test identified some problems here.

There is an interesting post about unit testing by Will Shipley (Delicious Monster). As well as some good info about unit testing by Chris Hanson (where I found the link to the Will Shipely post).

Link errors - Waxing the board

If XCode is to be believed I have 460 files compiling including the key file that is responsible for opening and reading the the Cello document. The next step is to wrap the "open" call in minimal snippet of cocoa and some make-shift UI. Pictured bellow is the Nib for the full FileTest UI.

The idea from here is then to surf through the link errors and, for a while, to let them drive the development. The biggest problem I can see here is that the dead-code stripping of gcc is not so good and my previous experience of the linker is that it will fail with link errors for code that is referenced but never executed.

After this my development plan is simple.

• Once Cello links I can then work through a set of sample files testing the read.
  
• When the read appears to work I can look at the file generation code and work through that.
  
• When the file generation works I will work on the saving of the files. At this point I will pick up the pieces of the structured document and file compression that I have postponed.

The idea behind this approach is to get the core functionality demonstrably working and then to attack the UI.

QuickTime - converting Audio Formats

As I near the end of working through the output generation files I have come across the the issue of audio content. The Cello output file has sound embedded in it either as ADPCM or MP3. As Cello is a port from windows it has code that will covert from WAV (ubiquitous on Windows) to ADPCM. There is a converter class in Cello that backs onto some MFC APIs that takes care of this.

I have not done any work with sound but I figured that QuickTime might have the solution - digging around in the apple web-site in the Cocoa QuickTime sample code I found ExtractMovieAudioToAIFF that will extract a sound a movie and save it as an AFF file. QuickTime has a looser idea of what is a movie than you might think - a WAV file, for example, is considered to be a movie. The sample code as it stands will convert, for example, a WAV file to a AIFF file. It looks like I could use this code as a base or reference for converting between sound types.

I plan to postpone an work of converting sound files and will instead compile out the existing support for WAV files. I will do this as I can continue and get Cello working supporting just MP3 files. My priority is to get something working - and supporting a single audio format seems like a splendid start.

Cubics & Quadratics - Approximation results

I spent a few hours going through the Font Forge files to get a grip on how much work it would be to use it in Cello. My suspicion is that the code in Font Forge will produce better results in converting Cubics to Quadratics for fonts, simply because Fount Forge is dedicated to this kind of work and George Williams (the author) has obviously put a lot of work and consideration into the approximation - which specifically looks for various things like points of inflection when breaking the curve, and detecting Cubics that have been converted from Quadratics - something which happens with fonts.

Looking through the Font Forge code it does seem possible to map CGPaths onto the Splines and SplineSets - but it will be a bit of a fiddle to get it all working. On he other hand getting the existing Cello classes to work with CGPaths is actually quite straight forward. These classes are all written in C++ and the conversion is fairly trivial.

There is a huge amount to do so I am going with the path of least resistance. As the project progresses I can always open the Font Forge option again. It is something I would like to revisit as there is a huge amount of interesting stuff dedicated to the manipulation of splines in Font Forge.

I would personally recommend checking out the Font Forge FAQ for some great answers to questions. I particularly liked Why isn't Font Forge written in C++ - George Williams has written a great piece of software, is giving it away lock, stock and build instructions to whoever wants it for whatever purpose - it is there enjoy it.

To test the conversion of Cubics into Quadratics I took the Apple PathDemo sample code, that is in the installed with xcode in the developer folder, and made some changes. This sample code will, amongst other things, draw random paths - so the changes are quite small. I also added some code that will draw the on-curve points so that I can get an idea of the complexity of the new paths. Probably the biggest thing is Cello has it's own definitions points, lines, splines etc - and you need to satisfy the a reasonable spread of dependencies before you can use the magic ShapeVectorApproximator class.

Here are a couple of screen-shots from the test program. The grey thick line is the Cubic and the thin green one is the Quadratic.
The Quadratic on curve points are drawn in blue and the Cubic in red - so you can eaisly see that there are quite a few more cubic segments in the approximation.

If you look carefully in the image below you can see areas where the approximation breaks down and the green line strays outside of the grey - it should be in the exact centre.