Tuesday, 7 November 2017

Abominable Types

As I recently seemingly wasn't able to finish any of the blogpostst I started*, let's have a change of mood and begin with something simple, for example with "abominable types".

What are they? And why did I never hear of them? The answer to the first question is - they are part of C++'s strange menagerie of spooky beasts, of which there are:
  1. nasal demons 
  2. SCARY iterators 
  3. abominable types 
  4. etc... ;-) ** 
Why did I never heard of them? No idea, probably because only compiler writers and template library authors needed to know about them (except from full-time language lawyers of course!). But now, as of appearance of the proposal P0172r0 this language quirk achieved a certain publicity.

1. What they are 

I said I never heard of them, but I know at least why they are called as such - the name seems to originate from Alisdair Meredith:
So what are they again? As stated in P0172r0, this are function type bearing cv or ref qualifiers, for example:
  void foo() const;
  void foo() const volatile;
  void foo() &&;
What? We all know that we cannot define a function with a const qualifier outside of a class, so are these definitions even valid? It seems they are (in a context, more about later). As it is again stated in the told proposal the reason for their mere existence is the desire to enable the following construct:
  struct host {
     int function() const;// { return 42; }

  template <typename TYPE>
  constexpr bool test(TYPE host::*) {
      return is_same_v<TYPE, int() const>;

  constexpr auto member = &host::function;

  static_assert(test(member), "This is why abominations exist");
i.e. checking if some member function is const. As we can see, the problem with abominables is that they aren't function types of member functions but are not types of free functions either! They are halflings, robbed of their context, but coming to life again when provided with such.

By the way, there is a nice, little known trick exploiting the usage shown above (code copied from P0172r0):
  class rectangle {
      using int_property = int() const;  // common signature for several

      int_property top;
      int_property left;
      int_property bottom;
      int_property right;
      int_property width;
      int_property height;
In the words of the proposal:
"... examples ... explicitly writing these types fall into the category of showing off knowledge of the corners of compilers, and winning obfuscated coding contests."
Facebook's folly::function seems to use it in that manner nonetheless:
  using ConstSignature = ReturnType(Args...) const;

  using NonConstSignature = ReturnType(Args...);

  using OtherSignature = NonConstSignature;

2. The proposal 

Because some (very smart) library writers indeed used the abominables, there was some confusion about the exact goals of the proposal:
So what is the proposal trying to achieve?

The main concern is that when writing generic code we have to consider these function signatures on the overloads, multiplying theitr number by 6. As stated in the proposal:
"In many cases, the simplest option for handling abominable function types in a trait (or other metaprogram) is simply to state that they are not supported. The main problem here is a lack of vocabulary..." 
so you have to fight with them..***. A smaller problem is that they stand out from the type system because they are crippled (remember I called them halflings?)...

OK, I digress. The proposes changes are 1. kill abominable types, 2. make them non-functions, 3. make them regular, 4. minimal library cleanup (along the lines of 2.).

As it seems, 1. was ruled out and 2. is favored. This seems logical, as the abominables are not functions, they are halflings, so they should be demoted to something less than a function - a function-like type!

PS: Maybe there is a newer version of the proposal or a some decision about its fate by now, but as I stated at the start of this post, I just wanted to finish what I started back then.

* and I just hit the round number of 100 blogpost drafts... :-/
** look up some of the menagerie here!
** like in c17-features-and-stl-fixes-in-vs-2017-15-3: "std::decay now handles abominable function types (i.e. function types that are cv-qualified and/or ref-qualified)"

Saturday, 24 June 2017

QtCreator's debugger broken on Mac???


As the product of my client is running on both Windows and Mac, we have got our fair share of Mac bugs to be fixed. Yes, we really do, despite the fact that we are using Qt as our cross-platform portability layer! Write once, test everywhere, again... ๐Ÿ˜ž

Unfortunately, we had repeatedly encountered problems with debugging. As we were using QtCreator* on Mac to do Qt-development, we were trying to use its (i.e. QtCreator's) debugger but unfortunately we ran into problems when trying to use breakpoints.

Namely, the debugger seemed to ignore the breaking points we set, as it didn't stop the execution for them! Interestingly, you could step through the code and reach every nook and cranny of it, but you cannot let it run and wait till a breakpoint will be hit. So we had to set our breakpoints directly with LLDB and debug there - no graphical GUI and IDE goodies.

It's needless to say, that this constituted a major obstacle in our quest to eradicate all bugs on Mac. After the annoyance level has hit the high-water mark, my two brave colleagues Saลกa and Bojan came up with a solution which was quite surprising. I just want to share with you, my googling fellow-programmer, and with the rest of internet for the greater good...


It turned out, that the problem was due to the format of the debug information generated by qmake - it contained relative source file paths, like ../source.cpp. When used directly in LLDB, this didn't pose any problems, everything went well.

Unfortunately, when trying to set a breakpoint out of Qt-Creator, this started to be a issue, as Qt-Creator used full file paths when trying to set a breakpoint with LLDB. You see, file paths didn't match, this just couldn't work! For the sake of completeness it has to be mentioned that QtCreator does not have its own debugger - it just integrates Clang's LLDB and provides an comfortable GUI for it (but you knew it already, I guess...).

The solution was blatantly obvious - one just have to coerce qmake to create debug information containing absolute source file names. This can be done with the
option! Done and done!

The only problem is that this isn't an official option but an undocumented one! But it saved the day for us, so we will stay with it for the moment. If you want to read it, there's an interesting article "Undocumented QMake" on the Qt-Wiki. What it says about this special option is:
it seems this represents the number of sub folder that qmake refer to with relative paths (hardcoded is depth=4 in makefile.cpp) ...
As it seems, setting it to zero will force the usage of absolute paths.


As I said earlier, this all is due to Saลกa (and Bojan) so don't give me any credit for that! I'm only the messenger.

As a last word: we are using Qt version 5.6.1 at the moment.

* with Clang as compiler and and its LLDB as debugger

Monday, 8 May 2017

Qt and "a missing vtable usually means the first non-inline virtual..."

This post is a kind of a "thank you" for a really good piece of advice from a fellow programmer. As it is published on StackOverflow, and I for my part don't use it very much and cannot even upvote it, this post should spread the word about it instead!

Or, alternatively, you can see that post as another installment of an engineering notebook, next one in the series on bug hunting or another piece of advice for the googling programmer*.

The Problem:

I was checking in my changes including promoting a class to a QtObject and everything seemed to be working at first - I tested it locally, it seemed to build in TFS (Team Foundations Server, i.e. in our Windows CI build) but in TeamCity (i.e. in our Mac build) the build was broken with an ominous error message:
NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
And it came from the linker! Initially I supposed some Clang peculiarities (or even bugs) to be held responsible for that, and as I don't have a Mac to check this, I was a little lost for ideads.

The Solution:

But then I found this piece of advice:
Another possibility is that the class in question once didn't belong to Qt meta object system (that is, it had no Q_OBJECT or maybe didn't inherit from QObject at all), so qmake needs to be run again in order to create the necessary rules for MOC. The easiest way to force qmake to be run is to make some insignificant changes to the project file to update its timestamp, like adding and then removing some white space.
As I said, I cannot upvote it (you need at least 15 reputation points for that) so I just say thank you Sergey!

Because... that's exactly it! The missing virtual methods belonged to Qt's signal infrastructure which should be generated by the MOC compiler!

The error didn't manifest itself locally because I generated a Visual Studio project from Qt's .pro files, thus the new MOC file was forced to be created. Thus the problem disappeared trivially.

The same was done in our Windows CI build, but this time only Makefiles were generated and only the "naked" Microsoft compiler was invoked in command line mode to process them. Anyway, the same applies here, MOC creation is forced each time.

On the CI build server however, the "naked" .pro definition files are used, and qmake must be nudged a little to wake up and do its job.

The Moral:

Qmake doesn't like it when we change our minds ๐Ÿ˜‰.

* You've got it? Working mathematician etc..? ๐Ÿ˜Š

Friday, 24 February 2017

Advanced Stateful Lambdas

Earlier this week I listened to Episode 51 of C++ Weekly Youtube "show" ๐Ÿ˜‰ by @lefticus:
...and I just had to write this cool stuff down before I forget it!

So treat this post as a kind of engineering notebook - not my ideas, not my code, just a remainder and a reference. So without much ado, here's the code:
int main()
  auto fib = [a = 0, b = 0] () mutable {
    struct Results {
      int& a;
      int& b;

      Results next(int num = 1) {
        while (num > 0) {
          a = std::exchange(b, b + a);
        return *this;
      operator int() {
        return a;

    return Results{a, b}.next();

  // main
  return fib().next(5);

  // or:
  // return fib().next().next().next().next().next()

  // or:
  // return fib().next().a;
I won't comment on any part of the code, as not to take credit for work of others. If you want to know what this is about, just listen to this episode of C++ Weekly. Or even better, try to find it out by yourself and listen to it after that!

By the way, I support Jason on Patreon, and I think, you should support him too because of his fantastic job with C++ Weekly!

PS: here is another interesting code, this time  from episode 50, exploring generic lambdas and the "template if":
int intTotal = 0;
int doubleTotal = 0.0;
std::common_type<decltype(intTotal), decltype(doubleTotal)> grandTotal;

auto generic_visitor[&intTotal, &doubleTotal, &grandTotal] (const auto v) {
  grandTotal += v;

  if constexpr(std::is_same<double, decltype(v)>{} {
    doubleTotal += v;
  } else {
    intTotal += v;

Tuesday, 6 December 2016

Two interesting quirks of C++ uniform initialization

We all know and love the new C++11's uniform initialization feature (aka curly braces initialization), and, to be frank, there is much to warrant this love, like:

with structs
  struct X { bool x{false}; int i; ...};
  X x = { true, 100, ... };
  X x{ true, 100, ... };
with arrays:
  struct Y { int arr[3], bool b, ...};
  Y y{ {0,1,2}, false, ... };
  int* intArray = new int[3]{1, ,2, 3};
with library classes:
  std::vector<int> a = { 1, 2, 4 }; // yesss! At last!
  QMap<QString, QVector<int>> a = { {"0x111", { 1, 1, 1} }, {"0x100", { 1, 0, 0} } };
and you can enable it for you own classes as well, writing a constructor taking std::initializer_list as argument (example missing, but you know what I mean...)!

because it's universal, you can use it for types too:
  int i{1};
  int j{};
but it's a bit redundant here, as we already could do:
  int i(1);
  int j(0); // not j()! I never used () but assumed it to be the default initialized int :(
but there are 2 additional goodies packed into that, as I learned recently, namely:

1. The first one
this is an old problem: unexpectedly, the compiler will consider this:
  TimeKeeper time_keeper();
not to be an object instantiation but a function definition! Here TimeKeeper is a class (reused) from the Wikipedia article:
  class TimeKeeper {
      int get_time();
This the compiler will balk at:
  int t = time_keeper.get_time();
But thanks to uniform initialization not at this:
  TimeKeeper time_keeper1{};
  int t = time_keeper1.get_time();
OK, you are right, that's not the most vexing parse ๐Ÿ˜‰, but simply incorrect usage of the constructor! The most vexing parse requires a parameter to the constructor! But I made this error several times myself when blindly typing ahead...  none the less, the problem is the same, only with a parameter:
  TimeKeeper time_keeper(Timer());
Here is a function with taking a function(!) like Timer mkTimer() as single, unnamed (!) parameter. Vexing? Now correct that with a single stroke (or two):
  TimeKeeper time_keeper{Timer()};
Nice to know when you need a workaround for a vexing parse!

2. The second one
Here Bjarne himself explains that:
  int x = 7.3; // Ouch!
  int x0 {7.3}; // error: narrowing
  int x1 = {7.3}; // error: narrowing
Moreover, compiler will automatically check int sizes on initializing (in Bjarne's words again):
  char c1{7}; // OK: 7 is an int, but it fits in a char  
  char c2{77777}; // error: narrowing (assuming 8-bit chars)
That's nice.

Considered I am a traditionalist and like my code to look like a old, regular C++, but these features make a nice argument in favor of using curly braces instead of the normal ones! Will for sure consider that!

Saturday, 3 December 2016

N3599 proposal, typesafe printf() and some C++ explanations

Reading the C++ Tips, 2016 Week 46 I stumbled over following syntax :
  template <typename T, T... chars> 
  constexpr CharSeq<chars...> operator""_lift() { return { }; }
Only to be instructed in the comments section that this is a non-standard extension of user defined literals, that Clang and gcc nonetheless seem to support (still learning C++14 as you can see). I wondered if my not-so-new Visual C++ 2013 compiler would support it too, so I had a closer look at it.

I soon found out*, that there's a proposal for this, namely N3599. This proposal being quite old (March 2013), I thought that my chances aren't that bad... So I took the first example usage of the feature I found in N3599 and tried to compile it in Visual Studio. Result? Not supported, of course, the "rejuvenation" of Microsoft compiler is still underway**.

At this point I got quite interested in the code itself and in the possibility to got it running even without the templated user defined literals. So here it comes, my explanation how the code is working, because it completes a non-trivial feat - to generate a function of given type from a given string (aka. textual description!).

But wait, type safe printf? Didn't Bjarne explain it somewhere already***? I think I heard something like this, but didn't check it... Stop, let us stay with the original, humble goal of understanding a piece of template code, which wasn't entirely clear at the first sight.

1. Template code analysis and explanation

The original code was:
// A tuple of types.
template<typename ...Ts> struct types {
  template<typename T> using push_front = types<T, Ts...>;
  template<template<typename...> class F> using apply = F<Ts...>;

// Select a type from a format character.
template<char K> struct format_type_impl;
template<> struct format_type_impl<'d'> { using type = int; };
template<> struct format_type_impl<'f'> { using type = double; };
template<> struct format_type_impl<'s'> { using type = const char *; };
// ...
template<char K> using format_type = typename format_type_impl<K>::type;

// Build a tuple of types from a format string.
template<char ...String>
struct format_types;
struct format_types<> : types<> {};
template<char Char, char ...String>
struct format_types<Char, String...> : format_types<String...> {};
template<char ...String>
struct format_types<'%', '%', String...> : format_types<String...> {};
template<char Fmt, char ...String>
struct format_types<'%', Fmt, String...> :
  format_types<String...>::template push_front<format_type<Fmt>> {};

// Typed printf-style formatter.
template<typename ...Args> struct formatter {
  int operator()(Args ...a) {
    return std::printf(str, a...);
  const char *str;

template<typename CharT, CharT ...String>
typename format_types<String...>::template apply<formatter>
operator""_printf() {
  static_assert(std::is_same<CharT, char>(), "can only use printf on narrow strings");
  static const CharT data[] = { String..., 0 };
  return { data };

void log_bad_guess(const char *name, int guess, int actual) {
  "Hello %s, you guessed %d which is too %s\n"_printf(
    name, guess, guess < actual ? "low" : "high");
Not quite easy to read, isn't it? But let us go through that step by step, using the old trusty top-down method.

1. log_bad_guess() function uses the custom _printf  literal operator for the format string, and then.... seems to be calling itself with more parameters??? Whassat? Are we moving towards Haskell-like unreadability?

2. Maybe not. If literal operator applied to a string can be called then it must return a callable, i.e. something with operator (). Look at _printf's definition - it is a template returning something complicated. This something must then define the call operator, so lets look for it.

3. This something is created by "calling" the apply "method" of a format_types structure with the  formatter function. Of course all of it at metaprogramming, i.e. compile-time, i.e. types-only level, so we need a short explanation here:

The apply metaprogramming "method" is one of basic type manipulation primitives which can be written quite simply with the new C++11 parameter packs. It just sets parameters for a given template expecting some parameters. In our case, it creates correctly typed version of the formatter function. Correctly typed means that the input parameter types will match the types required by the format string. And these types will be provided by  format_types type list. A type list is implemented as a template parameter pack.

The another utility is the push_front "method" - it just extends a typelist with a new type. Cool.

BTW, there is a nice, short and free ebook (by @joel_f and @edouarda14) about modern C++ metaprogramming explaining such basic operations if you want to learn more. So have a look at it, with parameter packs the typelist manipulations got so much easier in C++!

4. So how format_types is generated? This is done with a series of template specialzations which descends the format string recursively, processing 2 character at one step, and if the pair starts with %, an appropriate type is added to the type list. This is done by appropraitely specialized format_type_impl template, which ist than aliased to format_type as to directly acces it's type "member".

5. Now we have all the elements: _printf is generating a callable with function signature derived from the parsed format string, and then we are calling it with matching parameters. Was it that difficult? Not really, right?

2. Getting it runing with VS 2013 compiler

Alas, VS 2013 doesn't like the template<...> operator "" _printf construct: .... What can we do? I tried following changes:
template<typename CharT, CharT ...String>
typename format_types<String...>::template apply<formatter>
 printf() {
    static_assert(std::is_same<CharT, char>::value, "can only use printf on narrow strings");
    static const CharT data[] = { String..., 0 };
    return{ data };
You see, now we have a function printf() returning a functional object (thus it's a HOF - higher order function of kinds) which is generated based on the format string, as we have seen above.

Note that I had to change std::is_same() to  std::is_same::value as VS2013 compiler does not support constexpr, and operator() should be const here!

Then I wanted use it like that:
    void log_bad_guess(const char *name, int guess, int actual) {
    auto p = printf<char, "Hello %s, you guessed %d which is too %s\n">();
    p(name, guess, guess < actual ? "low" : "high");
but... compiler error! Seems compiler cannot match char string with CharT ...String&, hmm, lets try this:
void log_bad_guess(const char *name, int guess, int actual) {
    auto p = printf<char, '%', 's', '%', 'd', '%', 's'>();
    p(name, guess, guess < actual ? "low" : "high");
Yesss, now it's compiling! So if we change the format string to "%d" like this:
    auto p = printf<char, '%', 's', '%', 'd', '%', 'd'>();
We should get a compiler error, preferably something like: "Cannot call XXX with YYY parameters". What VS 12013 reports is however:
  error C2664: 'int formatter<format_type_impl<115>::type,format_type_impl<100>::type,format_type_impl<100>::type>::operator ()
(format_type_impl<115>::type,format_type_impl<100>::type,format_type_impl<100>::type)' : cannot convert argument 3
from 'const char *' to 'format_type_impl<100>::type'
Ok, speak about compiler template error messages... This could be probably improved with some judicious usage of static_assert, but here we are only seeking understanding, not production code quality. But we know what is going on, right?

So there is a last final touch missing: automatic conversion from char string to a char parameter pack. Well, that's the problem! The proposed (missing) user defined literal operator would do that! As one of fellow bloggers said:
" But a similar template syntax is standardized for raw numeric literals, so the lack of raw string user defined literals in C++14 is a pretty big inconsistency that I hope C++1z will resolve."
So we must implement this ourselves, using a techniques similar to one of the integer_sequence's implementations here, then just pass the char string length via the T (&) [N] syntax, an than it should be working. Or use some of modern C++ metaprogramming libraries like Brigand or Hana...

I won't implement the character sequence generation here, because this post is very long already. The second option is out of question as well, because I want only use "naked" C++ standard library... Thus our goal won't be fully achieved in this blogpost. Sorry ๐Ÿ˜’!

3. Summary

You might be asking "why are you writing that"? It's nothing earth-shattering, only some regurgitated, already known stuff... My explanation: maybe somewhere there's a (young) programmer looking at code like that and thinking "OMG, I'm never gonna to understand that, it's too advanced, it's magic". And the entire industry is asserting this view. I am against such elitism, it's just a piece of code doing some work, humans wrote it so another human might understand it. It's the same I feel about the category theory cargo-cult (read here) - don't fear, don't let discourage yourself by the general sentiment in programming industry, don't believe 10x programmer fairy tales. It's only computing, and it boils down to if, then, else and loop ๐Ÿ˜Š.

So what did we achieve? We just synthesized a function signature via a textual description, and then called a vararg function with parameters of that types. Higher order functions anyone?

* through Sumant Tambe's blogpost; "Dependently-typed Curried printf in C++"You may wonder here what that "dependently-typed" moniker might mean.... In course of demythologizing of functional programming ;) I dare say that a dependent type is a type depended on some non-type parameter, aka. tag. An I hope I'm right here, because I didn't google it, only restate what I remember about Idris...Wish me luck ๐Ÿ˜‰.

** Why there is no expression SFINAE yet, etc, quite interesting piece of MS compiler histrory: Rejuvenating the Microsoft C/C++ Compiler

*** Python Style printf for C++ with pprintpp, this goes in similar vein, but the code is somehow very complex, maybe owing to the fact that pairs of curly braces have to be found (Python syntax). Additionally it needs macros - fancy if N3599 would solve that?

Wednesday, 27 July 2016

Structure-like std::tuple usage

While reading P0095R0, WG21* I stumbled upon a cool trick. If you define following structures:
  struct x { double value; };
  struct y { double value; };
  struct z { double value; };

  using point = std::tuple<x, y, z>
you will then be able to write:
  auto x = std::get<x>(point);
  auto y = std::get<y>(point);  
  auto z = std::get<z>(point); 
Look at that! Now we can use std::get<x> to fetch the 'x' value of the tuple, std::get<y> for 'y' and so on, instead of plain, old (and ugly...):
  using point = std::tuple<double, double, double>

  auto x = std::get<0>(point);
  auto y = std::get<1>(point);  
  auto z = std::get<2>(point);
Cool, isn't it?

But the author dismisses such tricks on the spot:
"Should we use this approach everywhere and deprecate the use of struct in any context? In the author's opinion we should not. The use of wrapper types is much more complicated to both read and understand than a plain struct. For example, the wrapper types that were introduced, such as the 'x' type, make little sense outside of their corresponding tuples, yet they are peers to it in scope. Also, the heavy syntax makes it difficult to understand exactly what is intended by this code."
...on the premise that it is too clever. Come on, that little bit of extra code in one place, and then this readability all over your source code? But wait, it comes even harder:
"While the utility of type selection and SFINE for visitors is quite clear for advanced C++ developers, it presents significant hurdles for the beginning or even intermediate developer. This is especially true when it is considered that the visit function is the only way to guarantee a compilation error when all cases are not considered."
That got me thinking. First I wanted to dismiss the argument along the lines of: "what, how can a C++ developer not know what SFINAE is...", but then another thought grew stronger and stronger: "come down from the expert-only, high-church, pure-blood stance, C++ should be usable for beginners too!". Just think about yourself trying to learn an new programming language for a minute. Do you want it esoteric and complicated? Not really. So this is really a show stopper, kind of...

Because, we don't want to give up our freshly discovered syntactic candy so easily, do we?

Thus, I think we need a new syntax extension for C++17 (or C++0y?)**:
  using point = std::tuple<x : double, y : double, z : double>

  auto x = std::get<x>(point);
  auto y = std::get<y>(point);  
  auto z = std::get<z>(point);
Nowadays everybody gets his/hers pet feature added to C++17 (just look at if and switch initializers), so why not?

Come on guys! Who's with me? 

* "The Case for a Language Based Variant", C++ Standard proposal P0095R0. It discusses advantages of language based variant implementation vs. a library-only solution.

** of course we could use the brand-new destructuring like in:
  using point = std::tuple<double, double, double>

  auto [x, y, z] = point;
but that's not what we want. We need named access to single elements of the tuple. And don't say we shouldn't use tuple, because we simply want to!