BrightstarDB is a .NET application written in C# and I’m quite happy with the output of Sandcastle for the API documentation. However the developer and user docs were all originally created using the (commercial) Help & Manual application – which is a great documentation tool in its own right but would require contributors to the project to purchase a license in order to update or extend the docs.

Getting a local version of Sphinx running is the first step in evaluating if this approach is going to be feasible and as I’m developing on Windows 8, getting it running under Windows is my priority. So my goal here is to document how to install all you need to run Sphinx on Windows – with the qualifier that I’m doing this on Windows 8. YMMV on other Windows OSes.

1. Install Python 2.7: At the time of writing Python 2.7.5 is the latest Python 2 release. The Python project has Windows installers available at http://www.python.org/download/. As my OS is 64-bit, I used the “Python 2.7.5 Windows X86-64 Installer”.
2. Add the Python directory to your path: By default Python will install in C:\Python27 but will not add itself to your PATH environment variable. I find it useful to do that so I don’t have to keep typing the full path to the python.exe that you use to run Python scripts.
3. Install setuptools: The current version is 0.6c11 and you can get it from https://pypi.python.org/pypi/setuptools Installation instructions for Windows are on that page and differ for 32-bit vs 64-bit OS. The basic procedure is to download easy_setup.py and then run it. If you updated the PATH environment variable as suggested above, then this is as simple as opening a new command prompt, cd to the directory you downloaded easy_setup.py to and typing:

   python ez_setup.py

   You will see output from the script as it downloads and installs setuptools into your Python directory.

4. Add the Python scripts directory to your PATH: The core setuptools scripts get installed under {Python Directory}\Scripts and the Sphinx scripts will get installed in the same location so it is sensible to add this directory to your PATH environment variable. If you went for the default installation location you will add C:\Python27\Scripts to your PATH.
5. Download the Sphinx egg: Eggs are Python packages that make it easier to distribute and install Python applications. The Sphinx egg can be found at https://pypi.python.org/pypi/Sphinx. At the time of writing the version is 1.2b1.
6. Install the Sphinx egg: Open a new command prompt (to get your updated PATH) and cd to the directory you downloaded the .egg file to. You can then use setuptools to install the egg like this:

   easy_install Sphinx-1.2b1-py2.7.egg

   The package will install the Sphinx scripts and download and install a set of dependencies. The installation process produces a number of warning messages about not finding files but the installation runs through to completion OK.

7. Try a Sphinx “Hello World”: The Sphinx project have a more thorough tutorial. But in a nutshell this is what to do (assuming you are still in the same command prompt window) – the following assumes you have your Python directory and Python Scripts directory included in your PATH environment variable.
   1. Initialise your project. The following command create and initialize a directory that will contain your documentation:

      mkdir MyFirstDoc
      cd MyFirstDoc
      sphinx-quickstart

      There now follows a list of questions. Apart from the project name, author and release number, the questions have default answers which for this quick runthrough you can just accept by pressing enter.

   2. Edit index.rst. The initialization process creates a starter file (by default it will be called index.rst). You will want to keep the toctree directive, but you can add more content before or after it. The reStructuredText Primer on the Sphinx project site guides you through the basic constructs that are pretty simple to get the hang of.
   3. Make the docs. The initialization process for Sphinx has helpfully added a make.bat file to your documentation directory so the only command you need to produce your documentation in HTML is:

      make html

   4. View your work: You should now be able to view the glorious generated HTML by typing:

      _build\html\index.html

And that’s all there is to it – should take you about 10 or 15 minutes from start to end. Of course, now the hardest part, I have to start writing the docs…

c:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\ItemTemplates\CSharp\Web\MVC 4\CodeTemplates\AddView\CSHTML\List.tt(206,35): error CS0104: Compiling transformation: ‘ColumnAttribute’ is an ambiguous reference between ‘System.ComponentModel.DataAnnotations.ColumnAttribute’ and ‘System.Data.Linq.Mapping.ColumnAttribute’

The fix for this is to locate that .TT file (c:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\ItemTemplates\CSharp\Web\MVC 4\CodeTemplates\AddView\CSHTML\List.tt) and on line 206 change

var column = attribute as ColumnAttribute;

to

var column = attribute as System.Data.Linq.Mapping.ColumnAttribute;

Strangely, this error doesn’t occur in a VS2012 installation so it must have got fixed at some point but even after reinstalling MVC4 into VS2010 the error still came up.

Hopefully this post will help someone else find the fix !

Edit: This problem seems to also affect the Create.tt template (on line 122), the Delete.tt template (line 174), the Details template (line 179) and the Edit template (line 229) – the same fix works in all cases.

http://doriantaylor.com/cell-calendar

Before we go there, lets just pause a moment to think about the problem here. Take length as an example. For most folks going about their daily business, the metre is a good unit of length to use for most things – I’m 1.9m tall, my room is 3.5m by 2.5m. However, for some things we tend to prefer to use smaller length units such as the corner-to-corner size of my monitor (22″), the depth of my desk (40cm…I’m guessing that one), the diameter of the jack that goes into my iPod (3.5mm). The folks that make the chips that power my PC work in nanometres, cellular scientists probably dream in picometers. Physicists are the worst – depending on their speciality a picometer might be way too big or a kilometre way too small.  If we want a measurement system that works we need some way for all of these folks to express measurements in terms of the units that make the most sense to them but at the same time be able to freely map between them.

The HyTime answer to this is to provide a way to specify a measurement domain in terms of some basic unit of measurement and then to define as many other units of measurement for that domain in terms of their ratio to the basic measurement unit. For physical measurements of space and time, HyTime provides definitions measurement domains based on the Systeme Internationale (SI) units of SI second and SI meter. The

Its probably easier to understand this if we take one of these definitions as an example (the following are all snippets from the text of clause 9.2)

Firstly our measurement domain is defined in terms of a standard measurement unit (SMU). The standard measurement unit has an identifier, and this being HyTime and based on SGML, the identifier is a formal public identifier (FPI):

<!notation
   SImeter        -- Systeme International meter --
                  -- Reference unit of real length --

   PUBLIC "ISO/IEC 10744:1997//NOTATION
           Systeme International meter//EN"
>

If you come from the world of XML, just think of “ISO/IEC 10744:1997//NOTATION Systeme International meter//EN” as a funny-shaped URI and you’ll be doing fine. You can also think of this <!notation> tag as declaring a mapping from the short string “SImeter” to the longer identifier string. So this is essentially an SGML-flavoured namespace declaration.

Now we have a standard measurement unit defined and assigned an identifier we move on to use that identifier in a resource that describes our measurement domain and all of the different measurement units that can be used in it. In the HyTime specification each of these units is called a granule and a measurement domain can contain any number of granules. Every granule is defined as a multiple of some other granule in the same measurement domain (with at least one granule based on the SMU). The multiplier is expressed as a ratio of two numbers. These granule declarations are made in an SGML resource like this (I’ve left out many of the granules defined in the standard for brevity):

<measure smu=SImeter>
  <granule gn=um>                    1   1000 mm
  <granule gn=mm>                    1     10 cm
  <granule gn=cm>                    1     10 dm
  <granule gn=dm>                    1     10 meter
  <granule gn=meter>                 1      1 SImeter
  <granule gn=dam>                  10      1 meter
  <granule gn=hm>                   10      1 dam
  <granule gn=km>                   10      1 hm
  <granule gn=pica>                  1      6 inch
  <granule gn=barleycorn>            1      3 inch
  <granule gn=inch>                254    100 cm
</measure>

This being SGML, end-tags are optional, so if you are an XML-head you will have to imagine a </granule> inserted in the relevant places. For each granule, the granule’s magnitude is specified with two numbers and a reference to another granule. The two numbers define the ratio for one granule of the type being defined to the granule referenced by the granule definition. So, a mm is 1/10th of a cm, which is in turn 1/10 of a dm, which is 1/10 of a meter, which is exactly 1 SImeter. Note that this principle works for more complex ratio’s such as 1 inch being 254/100 cm (i.e. 1″=2.54cm). In this way, the standard measurement unit of an SI meter (with its FPI) is used as a base for defining all units of length measurement in a way which allows a conversion between any two particular units of measurement.

To my mind, this is a really cool facility because it means that all units are defined by the mathematical relationship between them making it possible for a machine to simply compare measurements given in different units (indeed, it provides a means for a machine to determine *if* a pair of measurements can be converted).

When I published the first post in this series, Bob DuCharme gave me a pointer to an ontology for Quantities, Units, Dimensions and Types. This has a similar approach to defining units. I think that I prefer the HyTime approach of defining conversions as a ratio of integers as it enables irrational numbers to be used in conversions (cf the definition of a barleycorn above as 1/3 of 1 inch) . What the QUDT ontology also does, which is extremely cool, is allow the definition of what HyTime calls “measurement domains” as a combination of other measurement domains. For example, a measurement domain for speed or acceleration could be defined in terms of the measurement domains for length and time.

Another interesting piece of work is this part of the Morfeo project. As with the QUDT ontology, it allows derivation of measurement domains. The wiki page I have linked to also discusses the problems of attaching measurement units to RDF property values, describing 4 different patterns with their pros and cons. For a topic map practitioner, these are instructive if we consider that an (unscoped) occurrence with a data resource is the same functionally equivalent to an RDF property with a literal value.

In the next post, I’ll try to start pulling together some thoughts on a Topic Maps ontology for defining measurement units and a pattern for expressing measurements.

First off, the term “Scheduling” might strike you as an odd term – it certainly fooled me. In fact all of Clause 9, with its references to “events” and “scheduling” might make you think you have accidently stumbled into a meeting of the iCal working group. However the first sentence of Clause 9.1 reassures that “scheduling” an object is no more than “giving them a position and size”. HyTime’s language is time-oriented, but the standard itself – as the rest of this clause makes clear – is neutral about how many and what types of axes we use to give an object position and size. That said, we are going to have to get used to the fact that in the rest of our exploration of this standard “scheduling” means defining position and size – i.e. measuring; and “event” means the thing being measured.

What this clause of HyTime is aiming to do is to provide us with a mechanism for interchange of the measurement of objects (real or virtual) in a way that it is possible to ground an application with the minimal amount of predefined “magic” needing to be baked into[1] an application. To do that we need some sort of model, and in Clause 9.1 the model is described in high-level terms.

The scheduling of events (positioning of objects) in HyTime is done by first defining a set of bounded axes that define the space within which those events exist – this is called a “finite coordinate space” (FCS). Each FCS is made up of a finite number of such axes, and each axis in an FCS has a finite length and is assigned to some measurement domain that defines the units of measurement along that axis. The room I am sitting in now is a FCS, with three axes whose sizes are bounded by the length, breadth and height of the room and where the measurement domain can be most conveniently (for the purposes of interchange) be expressed in metres. A piece of music can be seen as a FCS with axes denoting pitch, volume and time. Pitch could be expressed in hertz, and volume in decibels. Time in music is somewhat different however – the time in a piece of music cannot be expressed in seconds because all the timings are relative to the tempo at which the piece is performed. For this purpose HyTime allows some axes to use virtual measurement units – that is measurement units that have no absolute physical basis. The standard also allows for the fact that an axis that uses a virtual measurement unit might at some point be projected on to real, physical measurement domain (when the musician picks up their instrument and starts to play at a given tempo). A note (Note 257 if you are looking at the text of the standard) also helpfully points out that the measurement domain for an axis can even be an ordered set of things (the standard calls them topics, but all Topic Mappers know that a topic is anything…right?). So I can say that the books on my bookshelf form a measurement domain, where 1 is the first book from the left, 2 is the second one and so on.

For our purposes of expressing measurements for Linked Data, I suspect that this last interpretation for axes is probably not of much use, but the idea of allowing for axes that have a physical measurement and axes that have a relative measurement is of immense use as is the concept of mapping relative measurements on to physical axes at run-time.

So to recap what we have learnt so far:

1. To start measuring things we first need to define the space within which all of those measurements will lie.
2. The space we define can have as many dimensions to it as we want. Each dimension has its own axis.
3. Every axis must be bounded. That is it must have a start point and an end point.
4. Every axis must have an associated measurement domain that defines the units used to measure along that axis.
5. A measurement domain may be some real, physical domain such as length, frequency, volume or temperature. A measurement domain may be some virtual domain where positioning is all relative.

In the next post I’ll be looking at what the HyTime standard and other unit ontology efforts have to say about units of measurement.

[1] I suspect that the editors of the standard would use the more appropriate term “disclosed”. All interchange syntaxes that deal with semantics need to be grounded in some set of “things that need to be known by the application in advance of processing this stuff”. How a disclosure is made to an application is where standards get into the realms of hand-waving. Generally this means that a programmer (yes, you at the back) needs to hard-code stuff or write some clever (but non-standard) configuration voodoo.

In his closing keynote at this years TMRA conference (you weren’t there? you should have been!), Steve Newcomb made reference to the wonders of ISO/IEC 10744 or HyTime to its friends.

HyTime is a monster standard – it is complex and so difficult to implement in its entirety that I believe only one person has ever tried. That said, the standard contains so much that is useful and generally required for a functioning Web, that many of its pieces got cannibalized, stripped down and turned into hacker-friendly W3C “standards” – XLink. Just as XML owes its very existence to SGML, so XLink and SMIL both need to look back to HyTime as an ancestor (albeit one that never gets invited to the family parties). Anyway, Steve talked about how there is still much left in HyTime that could be useful and in particular picked out Clause 9 – Scheduling as one such piece. IMHO he is right on the money.

The Problem Statement

One of the biggest problems on the semantic web or linked data web is that we have no way to communicate measurements or positions using a grounded algorithm. There is no way that a fully conformant Topic Maps processor, RDF processor or OWL reasoner can tell that a given property is actually a point on some axis. And there is no way that these processors could convert from one axis type to another (say feet to metres). To do all this with current technology you need to bake in to your application some ontology-specific knowledge – something that tells you “Values of property X are always expressed in milimeters, and values of property Y are always expressed in seconds since midnight on 1st January 1970″.

It is staggering to realise that we can’t do this yet on the Semantic Web when you think about it. Its even more staggering to see bold statements being made for the “Web of Linked Data” without addressing the basic problem of “How do I know what units this data is expressed in”. I believe that this is where carcass of HyTime can be picked over once more

What I’m going to attempt

I have decided to go back to the HyTime standard and see what can be taken away from it for the benefit of those currently struggling with merging, comparing and meaningfully transforming Linked Data. HyTime is a massive spec, and as the Readers Guide To HyTime recommends, I’m not going to even attempt to read all 450 pages but will instead focus just on Clause 9. This part of the HyTime specification deals with the issue of describing the positioning and extent of an object in N-dimensional space and provides a mechanism for defining the units used for measuring along each axis.

My hope is that this facility can be used not only for defining the size and position of objects but also as a general purpose facility for expressing measurements of all kinds, and that is going to be my closed set of problems to address:

1. Specifying a measurement with a value and units in a way that allows an application to automatically compare and convert measurements that use different scales or units.
2. Specifying a location and/or extent of an object in N dimensions where each dimension has its own associated measurement domain.

Because this will probably take some time, and because I know you are too busy to read a three-screen-long thesis, and because I would  actually value feedback as I go along, I’m going to break this exploration up into a number of separate posts. I’ll create a Category to group all the posts together as I go. Please feel free to hit up the links above and dive in there and stick around for the next post where I’ll start to actually read this stuff.

What Do You Think ?

I would be interested in what others think about this. Do you know of some other existing ontology for measurements ? Have you tried to do something similar yourself (and if so what were your experiences) ? What chapter did you get to in the HyTime spec ? All comments, suggestions and peanuts from the gallery are welcome in the comments!

To this stage the project has been a collaboration between myself and Inigo Surguy, but making use of the tremendous work done by Jeni Tennison on the rdfquery plugin for jQuery – we are truely standing on the shoulders of giants here. We would welcome any comments/suggestions/offers of help.

Links to the project and its source code and the installer for Ubiquity can all be found on the Metatribble project homepage.

BTW We are designating this release a zeta because the concept of “beta” has been somewhat devalued by beta-ware such as gmail, we feel we need to wrap around the alphabet.

As usual though the real action was in the pub. One of the things we got to discussing was whether rdfQuery could be used to create stand-off markup on someone else’s content. Inigo Surguy pointed out that using tools such as Greasemonkey it should be pretty easy to get rdfQuery to scrape a page for the RDFa it contains and to add custom scripts to do something cool with that data. The problem comes when trying to persist any new RDF statements you might create. RDFa is a syntax for embedding RDF within HTML – so if you are in control of the page that you are adding the markup to, it is trivial to persist that markup simply by saving the modified file. If you are not in control of the page then you have some problems. The easy case is when the publisher of the page has already identified things that you might want to talk about and wrapped them in some RDFa. In this case you can simply add some more statements about those entities. What is harder is if the publisher of the page hasn’t marked up anything with RDFa. What is needed is a “bootstrap” mechanism to locate entities that you might want to talk about.

That is where OpenCalais comes in. The OpenCalais service takes content and locates entities within it, returning the content with markup added that identifies the entities within. Using some custom code interfacing to rdfQuery, it should be possible to turn the results from OpenCalais into RDFa, then you can do all the funky stuff you want with the RDF and serialize it to some persistent store (either on another web service such as the Talis platform or maybe to a local persistence mechanism such as Gears). Now, when you return to the page, your script again goes to OpenCalais to get the entities identified within and again turns this into RDF, but now you can smoosh in the RDF from your persistent store to retrieve all that cool markup you added.

What’s even better is that because OpenCalais has unique identifiers for the entities it recognizes, if you then visit another page that contains a reference to the same entity you should be able to pull in your extra markup automatically. I’m pretty sure that with this approach it should be possible to build up a personal knowledge store that can be merged into web pages as you view them, combine with some clever javascript to present that information and to allow you to extend the set of statements in the store and you have something really rather cool.

Just need to code it now

Although there is not much in the WiX documentation about this, it is possible to chain multiple MSI installers in a setup.exe wrapper so that the installers execute one after the other. The key to this is the setupbld.exe command-line tool that comes with the WiX 3.0 installation. Jon Torresdal has written a useful article on using setupbld.exe to force elevated privileges for an installer under Vista, and I recommend starting there for an explanation of how to prepare your build to generate setup.exe files. In addition to doing this, setupbld.exe can combine multiple MSI files into a single setup.exe installer. To do this use a command line like:

setupbld.exe -out $(TargetDir)setup.exe -msu FirstInstaller.msi -msu SecondInstaller.msi -setup $(ProjectDir)setup.exe