Testing and POLE Dancing

Pressing on with deploying POLE I am starting to look at Unit testing. The idea is to be able to test the new iterator before deploying it. Ideas like this are nothing new in the programing world - but coming from a project that is old, a little out of control and never had any unit testing it is good to deploy a little rigor. What I am hoping to gain from this is to eliminate unknowns and upfront and remove some of the uncertainties with POLE.

XCode comes with two built in unit test frameworks CPlusTest for C & C++, and OCUnit for Objective-C. There are other unit test frameworks around, and I note a particularly good post in Dave Dribin's Blog. I have elected to go with the CPlusTest despite Dave Dribin's reservations - the integration with XCode is good and I also plan use OCUnit when I come to start work on the UI.

CPlusTest is horribly easy to use - you set up a new target in your XCode project just choose Unit test Bundle from the list of possible targets, and then choose the C++ Test Case when you add a new file. XCode drops in a stub class to fill in. You have to register our test cases in the file - it is basic boiler plate stuff and Apple's documentation is quite clear. I only mention these steps to give an idea of how simple it is.

With CPlusTest you choose when you want the test cases to be run - they will either run every time you build your main target or just as a part of the building the test target. Test failures are logged as errors in your test file as were they compiler errors. This struck me as a little odd but it is quite convenient as you are reasonably close to the problem area when you get a test failure - you can command-double-click through the identifiers to where you need to be quite quickly.

The last unit testing I did was many years ago with a particularly grim piece of code called The Test Harness back in the days where real men spat Fortran - the ease with which unit testing can be deployed deployed now really is a pleasure.

Structured Storage

I have got through the first set of utilities and have started on the nitty-gritty of the file operations. A first look at a file called DWEditFile it seems like Cello uses Structured Storage. Coming from OSX this was the first time I have come across this technology, which is proprietary to Microsoft, and not available under OSX.

Usefull Liks:

1. Mictosoft's Structured Storage Reference
2. An Introduction to Structured Storage

There are a couple of open source projects that will read and write Structured Storage documents.

1. GNOME - Structured File Library (GPL)
2. POLE - Portable C++ library to access OLE Storage (BSD)

POLE seems to very lightweight, and straight forward. Looking through the code, on the surface, it lacks:

1. Some exception handling.
2. A copy operation - for copying bits of one document into another. This is something that Cello uses.

Neither of these seem that difficult to implement. The trick to it would seem to build a test bed to create and test the document - including the new copy operation.

The final thing that is different is that the way of accessing different parts of the document (structured documents are hierarchical) using POLE is very different to the windows. MFC seems to have iterators that will iterate through a given part of the document structure. I wonder if the best way to deal with this would to build an iterator like the MFC one on top of POLE - which minimizes the changes that will need to be made to to Cello. Also I can write a separate test code for this.

After doing the easy part of deploying POLE I had some problems understanding how the existing code worked and how to port it onto POLE. Strangely the DWEditFile is not the main document file (a bad assumption on my part) - the thing now is to figure out where it is used. It could be a skeleton.

Preferences and files that will never be ported

While working through the first set of Cello utility files I came across a couple of files - the first was DWNamedTreeStorage an XML reader and writer class. I fixed the compile errors (small string-type stuff) and moved on. It occurred that I could implement whatever there is in the file with some of the Core Foundation which has good support for reading and writing XML. However that is deviating from the game-plan. The game-plan is to get a subset (just reading files and generating the output file) as quickly as possible. Everything in Cello running on Windows works and it works well - there are many other wheels looking for reinvention.

Following this was a file called DWNamedTreeStorageRegistry. There was a fair amount in this file that would not compile and much of it was very OS specific - stepping back and looking where it was used - I found that it is used to save and load preferences from the registry. I have moved this file off the hit-list and will implement it's functionality after using CFPreferences when I come to the client code that uses it. It may well be that DWNamedTreeStorage is never used.

To keep a record of files that will never be ported - but whose functionality will have to be otherwise implemented I have added a text file called 'BadList' to each library.

CVS and Cutting the first Strings

I have created a new CVS repository and checked in the original sources. CVS is not the best control control system but it is not bad and does the job. I had at the outset thought about getting the project professionally hosted - there is some good Trac SubVersion hosting out there - but for the moment I will save the money.

I have started working through some of the utility classes and getting them to build. The code is well organized, structured and generally well commented. Coming from a long legacy project which in reality is a bit out of control working with code like this is a real breath of fresh air. I was fortunate enough to have spent a day looking at the code before buying into it, during that initial review I could see that it was well written and of a high quality. The most daunting aspect of it all is the shear quantity - there are hundreds and hundreds of files.

My first step is just to get things to compile. There are a few fun and games with templates as gcc is quite strict. There are (obviously) a lot of types defined in MFC - things like BYTE DWORD etc. I am making some stub headers with the minimum of declarations in them.

I have taken the MFC CString class and turned it into a wrapper for a CFStringRef - the code has it's own string class and it occurs to me that down the road I might consolidate them. The windows version of Cello is not Unicode which is something that down the line I will want to change. Making CString a wrapper for CFStringRef is a start - and those parts of the interface to CString that are char * I am scoping in a #ifdefs so that down the line I can flip the define, have my build break and work through the the compiler errors. The other thing is that the CFString is when I come to the UI a NSString - ready for cocoa.

I have after a good days work got 8 files compiling - the first took half a day but it gets quicker!

NASM another way out?

The .intel_syntax directive is not supported under the XCode Gas assembler. This is not an option.

In searching for information for converting MASM to Gas I have found that, as I understand it, most assembly work is actually done on NASM, and indeed this assembler has been available as part of he default XCode installation for a long time. Gas, it seems, is primarily designed as an assembler for GCC rather than as a stand-alone assembler. NASM is also closer to MASM than Gas.

MASM assembler still needs to be converted to NASM. I found a Perl program Nomyso that will do a fair bit of he work. It looks far promising than intel2gas which proved to be very disappointing.

Problems using Nomyso:

1. Handling of structs seems wrong
2. Included files are copied into the resulting file rather than being converted to %include.

Some useful links:

1. A good survey of Assemblers
2. Apple NASM Man Page
3. NASM Documentation
4. Writing A Useful Program With NASM

For the moment I am going to put the business of getting jcalg to assemble under OSX/XCode to one side as I can work around it.

jcalg1 - MASM and Gas Attack

Naïvely I assumed that assembler was assembler - this assumption like the best assumptions was based on ignorance. I have done assembler back in college days college on old PDP-11, later in 6502 and a little 68000 from this it never occurred that different assemblers (with the same target language) could be so very different. As jcag1 compiles under MASM - and XCode is Gas (gcc) they are quite different. My attempts to Google-up a gas version of jcalg1 did not yield.

The options from here seem to be:

1. Convert jcag1 to Gas. I have found a tool that claims to be able to do this intel2gas - but there is a large amount that it just can't handle.
2. Use the .intel_syntax directive in Gas and fix up all the errors manually. This directive makes gas use the AT&T syntax which is closer to MASM.
3. Modify my approach - change the original windows version of Cello so that it saves and opens files that are not compressed and thereby just side-step this problem. This would allow me to still create sample files.

Of these approaches the last will obviously work - however I will have a crack at the .intel_syntax directive and see if it has millage.

Initial plan of attack

My plan of attack to get Cello out of the water is to port the file operations first. The first goal is to have something running on the Mac that can read a file (sample files generated from the Windows version) and then to be able to create the results (also a file). After that I can look at the visual representation of the file (getting it all to draw), the UI and all of the rest of it.

The program is written almost entirely in C++, so I figured that my start would be to map the file operations (etc.) to Core Foundation. Looking at the code, and from the brief tour of it from one of the Windows engineers Cello reads and writes compressed files. It uses a library called jcalg1 which is written in assembler. So the first dirty hands will be getting it to assemble in XCode.