My Boost SoC Proposal

My name is Mariano Consoni. I'm an undergraduate student at
Buenos Aires University, planning to obtain a Computer Science
degree in the next few years. Here is my proposal for a plugin/extensions
library with reflection functionality.


Abstract

The project that I propose is centered in adding to Boost a library
that allows the programmer to include extensions/plugins.
Additionally this will be the start point to implement reflection
concepts.

My proposal has two parts:

• Bring Boost.Extensions [1] to completion and enhance it.
  

As I will say in the next sections, I wanted to add to Boost a library
that allows an application to have dynamic plugins/extensions
within an infrastructure.

A member of the Boost mailing list told me about a library by Jeremy
Pack called Extension. Jeremy and I agreed that the best approach
would be to complete and enhance this library. In short, the
completion consists in preparing the library to be sent for review in
Boost. The enhancements are several, and will be described later.

• Implement Reflection [2] using Extensions functionality

The Extensions functionality is the perfect platform to
implement Reflection in C++. I've analyzed several
papers about this topic and having Extensions complete
will be great to start building this new features. Also,
the Boost Community showed a great interest saying that
it would be a great addition to Boost. Discussing Reflection
with Jeremy we agreed that it could be very useful to
use the dynamic (run-time) method loading features in
Extensions to give reflection functionality to normal
C++ classes. I'll detail this point later.


Proposal history

Last year, I've been working in a GIS project. It needed
a very open architecture, and our decision was
to implement some kind of software plugin infrastructure.
This project was being written in C++ and with a
very intensive use of Boost. At that point, I checked out
if Boost had a Plugin Library.... without results.

So I designed and implemented an original infrastructure.
It was useful and I've learnt a lot during the development.
Also I've noted its pros/cons.

Since that time, I have been thinking about adding an
Extensions library to Boost, because I really like
Boost philosophy, and I use it very frequently (nowadays
I'm using it in at least three C++ projects,
and advocating its use to my colleagues).

Luckily, Summer of Code 2007 was about to start. Then I
tried to get in touch with the boost community. Since
early March I've been discussing my ideas in the lists with
positive feedback. They showed a lot of interest
and, as I said before, quickly pointed me out that there
was some work done in that area. With more luck,
Jeremy contacted me very quickly.

We started to work together that week. I started to review
his approach and he checked some of my old code. Obviously,
his approach is simpler, more concrete and really clean.
He told me that my code was very clear too, comparing it with
another libraries.

Then, we started to set the concrete goals for the project.
The result of that work is this proposal.


Detailed objectives

Objectives could be split in two parts. My first goal is to
bring to completion Boost.Extensions and then explore Reflection
functionalities.

Some of the improvements that are needed to complete
Boost.Extensions (in order to submit it for revision) were
addressed in our work with Jeremy. I think that some more will
appear when we start the work.

Then the actual tasks of this part are:

• 1.1. Preprocessor techniques to remove template redundancy. Nowadays some parts of the library are duplicated because the use of variable-arguments templates. The idea is to use the preprocessor library to avoid this. I've done it in another project, so I think that it'll be easy (but costly).

• 1.2. Provide (optionally) automatic closing of unneeded linked libraries. We think that this feature is important for many people to justify adding it in the standard library. Also, it can be added with "convenience" functions quite smoothly - just as smoothly as directly adding the functionality.
  

• 1.3. Various integrations with other Boost libraries for convenience (like boost::function).

• 1.4. Finish up documentation.

• 1.5. Write lots of more unit tests, as well as larger tests.

• 1.6. Submit to Boost for review.

Having that complete, I'll focus on the Reflection part. My main idea is
allow the C++ programmer to access a meta-level infrastructure where
he/she can load new classes/methods at run-time and get
information about them.

For that purpose I need to define an API that allows exporting methods
and then the infrastructure to load them on demand. The difference with
the Extensions approach is that in this part this methods
won't be a virtual function declared in a base class. I mean, you should
export the methods that could be used to access this meta-level and
then Extensions will have the responsibility of loading the
necessary dynamic shared objects.  Additionally, this will allow creating
objects that aren't derived of any class of the executable; instead they
are loaded on demand from a linked library.

A possible use of this API could be, for example, the development of a
C++ "interpreter".

Another interesting line of work could be having the ability to compile code
at runtime and use it ( i.e. compile it into a linked library and link to it at
runtime). If bjam was required to be in the path, and BOOST_BUILD_PATH
was all set up, this just might work well.

I know that this objectives are big but really interesting. I'll detail in the
schedule and deliverables section what I want to achieve for sure, and
what could be developed if the time allows me.


Deliverables

In the first part of the project, I mean the Extensions completion, the deliverables
are really concrete. After each step of the development (1 - 6) a new version
of the library will be released with the corresponding changes.

In the second part, the deliverables will be versions of this new Reflection
library. Mainly, the great deliverable will the library with the feature of allowing
the programmer to load classes/methods at run-time, on demand as it was
detailed in the previous section. It would be great if I can develop
some of the functionalities of on-demand compiling, but I can't guarantee
now (without having a deep knowledge in the subject).


Schedule

 Weeks in the interim period between being accepted as student and
program start

Try to get more and more feedback from the community and try to detail the
plan with Jeremy and my mentor. Maybe some prototype could be done
if I have spare time.

Week 1

Do a careful study of Boost.Extensions. I've checked it out, but I need a deep
understanding before starting. Get in touch with my mentor (as with Jeremy I'm
in touch nowadays). See carefully the requirements to review the library.

Weeks 2 - 4

First tasks, 1.1 to 1.3, and task 1.5.Initial unit tests will be written.

Weeks 5 - 6

Finish the first part (tasks 1.4 to 1.6), submitting the library to boost for revision.

Weeks 7 - 8

Explore reflection and do the first prototypes of the functionalities. Make
some units tests along the development.

Weeks 9 - 10

Go on with reflection, trying to fit it as a working and useful library.

Weeks 11 - 12

Fix bugs. Document. Make more unit tests. According to the remaining time add some
functionalities of the other optional tasks.  Plan the future after-program work.



Why is this library needed?

First, I think that this library is very useful for a very personal reason: if
it existed before I would used it in my GIS project :).

Besides that, the community showed a lot of interest in the list [3].
Moreover there is at least one individual (Vladimir Volodko) that wants
to help to develop some peripheral parts of the library.

The Reflection part also seems to be very interesting for the
community, and as far as we know there are a lot of functionalities
that could be built on top of the Reflection platform. Also, I think
that it could be the start point for Reflection functionality in Boost.


About me

My favorite language is C++. I love Boost philosophy, and since I've known
it I used it in a lot of projects including a large GIS project (we used smart
pointers, Spirit parser, StateChart, Threads, Filesystem, and many others),
a transit engineering project and some more. I've been advocating its use
in the development teams where I've participated with great results.

Besides that I like a lot functional languages (Haskell mainly) and Python.
When I knew Boost.Lambda it was a nice surprise.

I have strong experience with open source software (I installed my first Linux in
1998) and open source development. I've worked in at least 3 projects using
open source technologies, besides the work that I've done as part of university
courses. I'm familiarized with the interaction in a development community as I've been
part of LUGAr (an Argentinian Linux User Group), and of course many other
(distributed) development projects.

I've been studying Computer Science at Buenos Aires University
for some years. Here the career is long, six years at least, but when you finish you get
a degree comparable to a Msc.

In academic terms, my interests are related to static analysis. I've made some
work about Barvinok methods, Bernstein basis and Ehrhart polynomials to evaluate
some properties of source code without executing it.

I think that my strong interest in Boost mixed with my interest in static
analysis is a great mix for developing this project, and mainly the reflection
part. So, I think that maybe I am not the *best* individual to do this (that's
a very very strong assertion.. :) ), but I think that I'm very good because of
my experience with C++ (3+ years C++/Boost, and 6+ years with C) and my
past experience in different projects. Additionally, I could promise that I'll work
hard and do my best.

Apart from that, I think that building free software is really nice. I've been for
a long time a Boost User, and now I want to collaborate and have my own
library/project. I think that the GSoC idea is perfect, and it'll be great if I can
develop this library and then be part of the community as co-author of the
library and continue participating in the Boost.Reflection development.


References

[1] Boost.Extension: http://sourceforge.net/projects/boost-extension/

[2] Some reflection ideas: http://www.vollmann.com/pubs/meta/meta/meta.html

[3] Boost mailing list
. http://article.gmane.org/gmane.comp.lib.boost.devel/155445
. http://archives.free.net.ph/thread/20070314.225134.19510931.en.html


Other implementations and ideas about plugins or
reflection in C++

• http://yallara.cs.rmit.edu.au/~aholkner/dynload/index.html

 Plugin framework for C++, multiplatform.

• http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?Boost.Plugin

  Boost.Plugin idea.

• http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2074.pdf

  A paper about concepts of Plugins in C++.

• http://svn.berlios.de/viewcvs/yade/trunk/yade-libs/yade-lib-factory

/src/yade-lib-factory/?rev=HEAD

  A dynamic library loader.

• http://s11n.net/download/#class_loader

  A framework for dynamic class loading.