Beautiful Capi

Introduction

Beautiful Capi is a tool which automates the creation of compiler-independent and binary compatible C++ libraries across different C++ compilers. Libraries prepared by Beautiful Capi enable C++ libraries to be compiled once then used many times by the other C++ compilers without any recompilation. Of course, the compiled C++ libraries are compatible only on the same platforms and architectures where they were built. For instance, a shared library which was built by Visual Studio 2015 C++ compiler could be wrapped and called by Mingw Clang C++ compiler on Windows operating system (allowing both forward and backward compatibility) and vice versa.

This tool generates the required C++ and C code to wrap your C++ classes for use in a compiler-independent way.

This tool requires Python 3.6 or higher.

Another main concept is to generate a well-crafted beautiful C API, which is clear, readable and visibly suitable for human usage (not only for computers or compilers).

Beautiful Capi is written in Python 3. Note it does not parse the library source code to obtain its API description. Instead of that, as author you should provide a library API description in XML format. Usually such XML files which describe the library API are created by hand. There are no tools for creating XML API description files yet, however, such tools could be created in the future. There are plans to add support of some convenient DSL (Domain-Specific Language) in parallel to XML format.

Beautiful Capi is not intended as a tool for automating cross-language C++ library creation (for instance, the cases when a C++ library is used in Java or C# application) like SWIG. However, in the future Beautiful Capi could introduce such features and support for some other target languages. For details please see issue 7 and issue 39.

The main goal of this project is to produce highly efficient code and design elegance. This goal is informed by knowing Java programmers prefer to write wrappers by hand to avoid sub-optimal SWIG outputs.

Beautiful Capi has many examples which help to learn it step by step. You can find all examples in examples folder.

Regarding the license - the code generated by this tool can be used for any purpose, including commercial. Only the code generator tool itself is subject to GPL licensing.

C++ problems

C++ ABI

The one of the well-known C++ language problems is ABI (Application Binary Interface) incompatibility. The C++ language standard does not specify any ABI and it is implementation specific. For instance, the C++ language standard does not define size of int type. Each C++ compiler vendor can provide his own implementation of ABI. Any C++ library must be built again and again for every different C++ compiler which needs to use the library in an application.

Name mangling

The second problem in C++ is name mangling. In C++ name mangling is an encoding scheme which translates complex C++ identifiers (including overloaded functions and methods, template instantiations, namespaces, etc.) to plain C functions. The C++ language standard does not specify any name mangling scheme. Again, each C++ compiler vendor can provide his own implementation of name manging. A C++ compiler is used to build an application could use a different name mangling scheme incompatible from the C++ compiler used for building the original library. The typical result for the developer of the application is to face unresolved symbol linking errors.

C++ STL ABI

The third problem is binary incompatible C++ standard libraries. For instance, the size of std::string class is implementation specific and could vary from one C++ compiler to another, and even from one build configuration to another.

Exceptions

Different C++ compilers implement different exception throwing and catching schemas. An exception thrown from one C++ compiler runtime, in general, could not be caught and managed by another C++ compiler runtime.

Basic solutions

C++ application developers face and work-around these issues every day with some basic coding methods. A common basic solution for providing a stable ABI is to use special types which have fixed sizes. For instance, using int32_t type instead of int type, etc. The calling convention is also important, so, the developer needs to manually specify the calling convention for each method or function in the library.

The basic solution for overcoming different name mangling schemes is to manually write plain C functions. However, it is boiler-plate and error-prone, reducing development productivity.

The basic solution for the binary incompatible C++ standard libraries problem is to avoid exposing C++ standard library classes at the library public API. So, the library API should contain only primitive and fixed sized types in its API. As result, a developer should manually split some complex C++ standard library templates (such as std::vector<>, std::map<>) to primitive functions and types, which is also boiler-plate and error-prone.

Also, the developer is often forced to manually write C++ wrapper classes which will expose some higher level API rather than plain C functions and types. Such a process is laborious, boiler-plate and error-prone.

Beautiful Capi solution

Beautiful Capi is a tool which automates the creation of compiler-independent C++ libraries. It greatly helps to solve the name mangling problem, generating C++ wrapper classes, wrapping C++ STL library template classes, catching and rethrowing exceptions and much more.

Consider hello_world Beautiful Capi example. It exposes the following class:

#include <iostream>

namespace HelloWorld
{
    class PrinterImpl
    {
    public:
        void Show() const;
    };
}

void HelloWorld::PrinterImpl::Show() const
{
    std::cout << "Hello Beautiful World!" << std::endl;
}

In fact HelloWorld::PrinterImpl class is an internal class and it is not exposed directly for HelloWorld library clients. Instead of HelloWorld::PrinterImpl class an opaque void* pointer is used by the following automatic generated plain C functions:

void* hello_world_printer_default()
{
    return new HelloWorld::PrinterImpl();
}

void hello_world_printer_show_const(void* object_pointer)
{
    const HelloWorld::PrinterImpl* self = static_cast<HelloWorld::PrinterImpl*>(object_pointer);
    self->Show();
}

void* hello_world_printer_copy(void* object_pointer)
{
    return new HelloWorld::PrinterImpl(*static_cast<HelloWorld::PrinterImpl*>(object_pointer));
}

void hello_world_printer_delete(void* object_pointer)
{
    delete static_cast<HelloWorld::PrinterImpl*>(object_pointer);
}

For simplicity we show only partial details here, we do not show details such as calling conventions or C linkage options for these functions.

Note that all plain C function names have hello_world_ prefix which came from HelloWorld namespace. The second part of all plain C function names is printer_ which came from the Printer class name. The remaining parts are method names. Observe there are some simple rules for the conversion of any C++ identifier to a plain C function name.

And automatic generated C++ wrapper class:

namespace HelloWorld
{
    class Printer
    {
    public:
        Printer()
        {
            SetObject(hello_world_printer_default());
        }
        void Show() const
        {
            hello_world_printer_show_const(GetRawPointer());
        }
        Printer(const Printer& other)
        {
            if (other.GetRawPointer())
            {
                SetObject(hello_world_printer_copy(other.GetRawPointer()));
            }
            else
            {
                SetObject(0);
            }
        }
        ~Printer()
        {
            if (GetRawPointer())
            {
                hello_world_printer_delete(GetRawPointer());
                SetObject(0);
            }
        }
        void* GetRawPointer() const
        {
            return mObject;
        }
    protected:
        void SetObject(void* object_pointer)
        {
            mObject = object_pointer;
        }
        void* mObject;
    };
}

Of course, as author you need to manually create the following XML API description file to accompany the C++:

<?xml version="1.0" encoding="utf-8" ?>
<hello_world:api xmlns:hello_world="http://gkmsoft.ru/beautifulcapi" project_name="HelloWorld">
  <namespace name="HelloWorld">
    <class name="Printer" lifecycle="copy_semantic" implementation_class_name="HelloWorld::PrinterImpl" implementation_class_header="PrinterImpl.h">
      <constructor name="Default"/>
      <method name="Show" const="true"/>
    </class>
  </namespace>
</hello_world:api>

And sample usage of this class from client side:

#include <iostream>
#include <cstdlib>
#include "HelloWorld.h"

int main()
{
    HelloWorld::Printer printer;
    printer.Show();

    return EXIT_SUCCESS;
}

In this example HelloWorld::PrinterImpl is the implementation class, HelloWorld::Printer is the wrapper class. In the XML API description file HelloWorld::Printer identifier could be used for referencing this wrapped class and it is called API identifier or just identifier. In this example our wrapper class name is the same as the identifier, but in general they could be different.

Note that HelloWorld::PrinterImpl class has copy semantic. This means that the implementation class object instances are always copied when the wrapper class object instances are copied, and the implementation class object instances are deleted when the wrapper class object instances are deleted. There are other possible behaviours. In terms of this Beautiful Capi tool such behaviour is called lifecycle semantic. Beautiful Capi supports several typical lifecycle semantics.

There is hello_world example example which shows the first steps and the basic principles.

We can designate the following three code structure design concepts:

1. The implementation side. It means all code inside the C++ library, all classes, functions, methods and other types inside the C++ library. The implementation classes are used in the C++ library. Usually the C++ libraries are shared libraries which are intended to use by different C++ compilers.
2. The tiny C glue layer. The bodies of C glue functions are located inside the C++ library, in an automatically generated .cpp file. In fact these functions are written in C++ (to have access to the implementation classes) and just have C linkage option enabled. So, outside the C++ library these functions are seen as pure C functions. Beautiful Capi generates both bodies of these functions and their declarations. The declarations are visible outside of the C++ library.
3. The wrap side. It means all code generated by Beautiful Capi for clients of the C++ library, all classes, functions, methods and other types inside any client of the C++ library. Clients of the C++ library could be executable files, static libraries or shared libraries. The clients could be written in either pure C language or in C++ language. The C++ clients usually use the generated wrapper classes. The C clients use pure C functions directly. The wrapper classes are automatically generated by Beautiful Capi and visible only outside of the C++ library, inside the C++ library the wrapper classes are hidden and unavailable.

Lifecycle semantics

Beautiful Capi assumes that the implementation class object instances are always created on the heap. This fact is applied for all lifecycle semantics.

You can specify lifecycle semantic for each wrapped class in the XML API description file.

Copy semantic

Copy semantics means that the implementation class object instance is always copied when the wrapper class object instance is copied. In other words, copy semantics emulates objects by value, however, as we noted above, Beautiful Capi assumes that the implementation class object instances are always created on the heap. So, Beautiful Capi generates a special _copy C API function and the wrapper class calls the copy function.

void* namespace_prefix_class_name_copy(void* object_pointer)
{
    return new ImplementationClass(*static_cast<ImplementationClass*>(object_pointer));
}

This works only if the copy constructor for implementation class is available. If a class has a copy semantic then Beautiful Capi assumes that a copy constructor for the implementation class is available. Currently this supposition is hard-coped inside the Beautiful Capi and can not be changed.

The copy C API function is used both within the wrapper class copy constructor and the assignment operator.

Copy semantic emulates objects by value, thus, the generated wrapper classes propose to use "." (the dot sign) for accessing the wrapped class methods:

int main()
{
    // Creates the underlying implementation class on the heap of the C++ library
    HelloWorld::Printer printer;

    // Calls copy function to allocate new PrinterImpl class on the heap
    // of the C++ library by using PrinterImpl copy constructor.
    // printer and printer2 are different objects which have
    // different underlying implementation objects.
    HelloWorld::Printer printer2 = printer;

    // You have to use "." sign to access the wrapped PrinterImpl methods
    printer.Show();

    // At the end of this scope two PrinterImpl objects allocated on the heap
    // will be deallocated by using _delete function.
    // Please note that a heap manager of the C++ library will be used for that.
    return EXIT_SUCCESS;
}

The generated wrapper classes deallocate the underlying implementation class object instances by using a special generated _delete C API function. This function has _delete suffix and looks like this:

void namespace_prefix_class_name_delete(void* object_pointer)
{
    delete static_cast<ImplementationClass*>(object_pointer);
}

Delete C API function is called at the wrapper class destructor, thus memory leaks are nearly always eliminated. There is copy_semantic example which demonstrates this lifecycle semantic.

Reference counted semantic

Reference counted semantics means that the implementation class has a reference counter. The value of reference counter of the newly created objects should be equal 1. When a new reference to the object is created then the reference counter is normally increased by 1. When an existing reference to the object is destroyed then the reference counter is normally decreased by 1. The objects themselves should be deleted when the reference counter become 0.

Beautiful Capi requires the availability of the following functions for the reference counted implementation classes:

void intrusive_ptr_add_ref(ImplementationClass* object);
void intrusive_ptr_release(ImplementationClass* object);

Beautiful Capi generates _addref and _release special C API functions which use the above declarations:

void namespace_prefix_class_name_addref(void* object_pointer)
{
    intrusive_ptr_add_ref(static_cast<ImplementationClass*>(object_pointer));
}
void namespace_prefix_class_name_release(void* object_pointer)
{
    intrusive_ptr_release(static_cast<ImplementationClass*>(object_pointer));
}

The generated _addref C API function is used both in the wrapper class copy constructor and the assignment operator. The wrapper class destructor calls the generated _release C API function.

The reference counted semantic does not require any copy constructors for the implementation classes, thus allowing abstract C++ implementation classes to be used freely.

Reference counted semantic emulates object pointers (smart pointers). So, you should use "->" (the arrow) for accessing the wrapped class methods, also the generated wrapper classes have Ptr suffix by default:

int main()
{
    // Creates the underlying implementation class on the heap of the C++ library.
    // Reference counter is 1.
    HelloWorld::PrinterPtr printer;

    // Calls _addref function to create a new reference to the existing object.
    // Reference counter is 2.
    // Both printer and printer2 reference to the same underlying implementation object.
    HelloWorld::PrinterPtr printer2 = printer;

    // You should use "->" to access wrapped PrinterImpl methods
    // However, "." sign is also could be used here, i.e.: printer.Show(); instruction will be compiled fine.
    // But we recommend you to always use "->".
    printer->Show();

    // At the end of this scope the PrinterImpl underlying implementation object will be deallocated.
    // This is because printer object destructor will decrease reference counter by 1 (from 2 to 1),
    // and printer2 object destructor will decrease reference counter by 1 (from 1 to 0),
    // and the underlying implementation object will be deallocated.
    // Please note that a heap manager of the C++ library will be used for that.
    return EXIT_SUCCESS;
}

There is a reference_counted example which demonstrates this lifecycle semantic.

Raw pointer semantic

Raw pointer semantic does not have any special requirements to the implementation classes. It emulates pointers (just raw pointers, not smart pointers). The generated wrapper classes do nothing at their destructors, so you need to manually destroy the created underlying implementation objects to avoid memory leaks.

The generated wrapper classes have a special method for that, which usually has Delete() name by default. You can customize this name, see generation parameters XML schema. The special Delete() method uses a special generated _delete C API function. The generated _delete C API function is the same as the generated _delete C API function for copy semantic.

Raw pointer semantic emulates non-owning pointers. So, you should use "->" (the arrow) for accessing the wrapped class methods, also the generated wrapper classes have RawPtr suffix by default:

int main()
{
    // Creates the underlying implementation class on the heap of the C++ library.
    HelloWorld::PrinterRawPtr printer;

    // Both printer and printer2 reference to the same underlying implementation object.
    HelloWorld::PrinterRawPtr printer2 = printer;

    // You should use "->" to access wrapped PrinterImpl methods
    // However, "." sign is also could be used here, i.e.: printer.Show(); instruction will be compiled fine.
    // But we recommend you to always use "->".
    printer->Show();

    // You need to manually deallocate the previously allocated PrinterImpl object.
    // Note that you need to deallocate it once by using either printer or printer2 object.
    // Here we used printer2 for deallocation, we could use printer instead, but not both.
    // This is because a double deallocation will happen in such a case.
    // Please note that a heap manager of the C++ library will be used for deallocation.
    printer2->Delete();

    return EXIT_SUCCESS;
}

There is a raw_pointer_semantic example which demonstrates this lifecycle semantic.

Common methods of the wrapper classes

If you need to create a wrapper class object which does not reference any underlying implementation object then you can use Null() static method:

    HelloWorld::PrinterRawPtr null_pointer = HelloWorld::PrinterRawPtr::Null();

The same thing could be applied for all other semantics. This is because the underlying implementation objects are always created on the library heap, and the wrapper classes just hold pointers:

    // Copy semantic
    HelloWorld::Printer null_printer = HelloWorld::Printer::Null();

    // Reference counted semantic
    HelloWorld::PrinterPtr null_printer_ptr = HelloWorld::PrinterPtr::Null();

There is an IsNull() helper method which returns true if an internal pointer to the underlying implementation object is null. For convenience, there is an overloaded operator! in the wrapper classes, so, you can write the following code:

    HelloWorld::PrinterPtr printer_ptr = HelloWorld::PrinterPtr::Null();
    if (!printer)
    {
        std::cout << "printer_ptr is NULL" << std::endl;
    }

Command-line arguments

The main script to execute Beautiful Capi generation is source/Capi.py. If you run it with --help argument then you will have a similar output:

Beautiful Capi  Copyright (C) 2015  Petr Petrovich Petrov
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions.

usage: Beautiful Capi [-h] [-i INPUT] [-p PARAMS] [-o OUTPUT_FOLDER]
                      [-w OUTPUT_WRAP] [-s OUTPUT_SNIPPETS]
                      [-k API_KEYS_FOLDER] [-c] [-v] [-t UNIT_TESTS_FILE]

This program generates C and C++ wrappers for your C++ classes.

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
                        specifies input API description file
  -p PARAMS, --params PARAMS
                        specifies generation parameters input file
  -o OUTPUT_FOLDER, --output-folder OUTPUT_FOLDER
                        specifies output folder for generated files
  -w OUTPUT_WRAP, --output-wrap-file-name OUTPUT_WRAP
                        specifies output file name for wrapper C-functions
  -s OUTPUT_SNIPPETS, --internal-snippets-folder OUTPUT_SNIPPETS
                        specifies output folder for generated library snippets
  -k API_KEYS_FOLDER, --api-keys-folder API_KEYS_FOLDER
                        specifies output folder for generated API keys
  -c, --clean           cleans input and snippets directories
  -v, --version         shows version number
  -t UNIT_TESTS_FILE, --tests-file UNIT_TESTS_FILE
                        generates unit tests for properties into specified
                        file

The input API description file format (--input option) is described here. The generation parameters input file (--params option) has also XML format and its schema is described here. The output folder (--output-folder option) will contain the generated wrap classes and other files for using on the wrap side. The output file name for wrapper C-functions (--output-wrap-file-name option) will contain the C glue layer function bodies, this file have to be a part of the C++ library. The output folder for generated library snippets (--internal-snippets-folder option) will contain the generated snippets. For details about snippets please read snippets section. The output folder for generated API keys (--api-keys-folder option) will contain the generated keys for the C++ library secured API, more details please see in secured API section. The tests file (--tests-file option) specifies output file for the generated unit tests, please see unit tests section.

Integration with CMake

Basically integration with CMake could be done by using add_custom_command.

    add_custom_command(
        OUTPUT
            ${CMAKE_CURRENT_SOURCE_DIR}/AutoGenWrap.cpp
        COMMAND
            ${PYTHON_EXECUTABLE}
            ${beautiful_capi_SOURCE_DIR}/source/Capi.py
            -i ${CMAKE_CURRENT_SOURCE_DIR}/SampleAPI.xml
            -p ${CMAKE_CURRENT_SOURCE_DIR}/SampleAPI_params.xml
            -o ${CMAKE_CURRENT_SOURCE_DIR}/include
            -s ${CMAKE_CURRENT_SOURCE_DIR}/snippets
            -w ${generated_source}
        MAIN_DEPENDENCY
            ${CMAKE_CURRENT_SOURCE_DIR}/SampleAPI.xml
        DEPENDS
            ${CMAKE_CURRENT_SOURCE_DIR}/SampleAPI_params.xml
        WORKING_DIRECTORY
            ${CMAKE_CURRENT_SOURCE_DIR}
    )

But you should find Python3.6 interpreter before:

find_package(PythonInterp 3.6 REQUIRED)

Also you need to include AutoGenWrap.cpp to SampleAPI library:

add_library(SampleAPI SHARED
  ${CMAKE_CURRENT_SOURCE_DIR}/AutoGenWrap.cpp
  ...other files...
)

Important: You must do not insert the output folder for generated wrap classes to the library's include path. But you can do this with snippets folder.

XML API description schema

The wrapped C++ library API is described in the portable XML format for the exposed API by using this schema. This schema uses http://gkmsoft.ru/beautifulcapi XSD namespace.

The detailed description of this schema is here. The root element is api which has TBeautifulCapiRoot XSD type.

XML generation parameters schema

The Beautiful Capi generator has parameters in XML format, which has this schema. This schema uses http://gkmsoft.ru/beautifulcapi-params XSD namespace.

The detailed description of this schema is here. The root element is params which has TBeautifulCapiParams XSD type.

Mixing semantics

In real C++ programs some class could be used by different ways. These ways could be pointers to this class, references, smart pointers and values. For each class Beautiful Capi XML API description specifies a semantic. Sometimes it is not enough, because some implementation methods could accept references to this class, other implementation methods could accept pointers to this class etc.

Beautiful Capi proposes two ways to solve this problem: the first way is casting attributes, and the second way is lifecycle extensions.

Exception handling

Beautiful Capi takes care about exceptions. As described above, in C++ problems section, different C++ compilers have different exception throwing and catching schemas, also C language does not support exceptions. The exception_handling_mode attribute in XML parameters file specifies mode for exception handling. Value no_handling means that no special handling is done, so, any exception from the C++ library could crash the client application in general, unless the client application and the C++ library use the same C++ compiler.

When by_first_argument value is used then exception information is passed in a special structure. A pointer to this structure is added as the first argument to each function or method which could, potentially, throw an exception.

This was an introduction to Beautiful Capi. The full documentation is available here.