Friday, 31 August 2018

Crazy UB when casting to enum values in C++


Hi folks, this time it will be a kind of photo story! Something new this time, aaaaand I must type much less text!

It started a couple of day ago, when I saw following quiz on Twitter:

My answer was: all of them of course, in C or C++98 they would be probably all fine, but who knows what crazy UB stuff did they prepare for us in C++20. So when seen logically, all of that stuff should be undefined behaviour! But wait, dind't I used extensions to enum ranges in some project in some framework by casting integers to enums? That did work ok, didn't it? Ok, it was couple of years ago, C++03 I suppose, but AFAIK nothing has changed in C++11 in that respect, so maybe we did it wrong all the time back than? So how did it work?

So I wasn't so sure anymore, and would say A+B are UB, C is something that C++11 introduced, so no idea, but probably UB too. But wait, all of them are UB again? Probably, but the trouble with that was that there wasn't such an answer in the quiz! Crazy!

When I saw the answer I was even more perplexed:
What? Why 3 but not 4? And that numeric_limits<>::max() can't be right either! In a following tween the relevant spots in the standard were shown, and I read them, but was none the wiser - standardese isn't the lightest read on earth. ☹️

But before I gave up I remembered that I've seen something similar on Twitter already, namely in (accidently also cited) tweet by @lefticus:
So had the same gut feeling as me, but he was corrected by some people well-versed in interpreting the standard (which I also happen to follow):
and:

That was than even confirmed by @CppSage himself:

This all explains it pretty well. But isn't that crazy? Binding logical range of a type to its underlying representation? I'd understand that if we were in the 80-ties, but in C++20 standard with all that lofty newfangled features? OK, I suppose it was always like that in C, but I'm too lazy to check it, sorry.... ๐Ÿ˜ฉ

BTW, now I've got an idea why this enum casting in a distant project might even have been working - we used Visual Studio compiler! And I dimly remember that there were some Microsoft extension for enums setting int as their underlying type. Was it so? For that I'd have to check my own blogpost from last year, but frankly, I cannot be bothered. ๐Ÿ˜ฉ That project has died already long ago.

TL;DR: Twitter is a good place to learn about C++, I learn almost every day!

Monday, 6 August 2018

Scripting my Vim Editor


As an inveterate vim user the first thing (or one of the first things) I'd do when starting a new project under Windows is installing vim to use it for searching, grapping and editing logfiles when I'm bugfixing. Or just a to use it as a general viewer for all sorts of files.

Typically I'd set up my toolbar buttons (for vim 7.4) like that:
"" Caution: Needs 18x18 bitmaps in "C:\Program Files (x86)\Vim\vim73\bitmaps" !!!
:amenu ToolBar.-SEP- :
:tmenu ToolBar.tabedit Open new tab (mrkkrj)
:amenu ToolBar.tabedit :tabedit<cr>

:tmenu ToolBar.darklight Switch color themes (mrkkrj)
:amenu ToolBar.darklight :colo zellner<cr>
:tmenu ToolBar.lightdark Switch color themes (mrkkrj)
:amenu ToolBar.lightdark :colo torte<cr>

:tmenu ToolBar.smallfont Smaller font (mrkkrj)
:amenu ToolBar.smallfont :set guifont=Lucida_Console:h9<cr>
:tmenu ToolBar.bigfont Bigger font (mrkkrj)
:amenu ToolBar.bigfont :set guifont=Lucida_Console:h10<c>
This gives me my standard button toolbar with buttons for switching fonts and and colors from big to small and from dark to light, plus a button for opening new tabs:


Somehow this was always enough, beacuse I just used my default dark scheme  plus one more dark and an alternative light one. As you know the colors depend on the monitir you are using and the room you are sitting in, so in my new project I wanted to toggle through several schemes and font sizes before I settle down with my favorite. Thus I introduced following functions:
:tmenu ToolBar.darklight Switch color themes (mrkkrj)
:amenu ToolBar.darklight :call ToggleLightScheme()<cr>
:tmenu ToolBar.lightdark Switch color themes (mrkkrj)
:amenu ToolBar.lightdark :call ToggleDarkScheme()<cr>

:tmenu ToolBar.smallfont Smaller font (mrkkrj)
:amenu ToolBar.smallfont :set guifont=Lucida_Console:h9<cr>
:tmenu ToolBar.bigfont Bigger font (mrkkrj)
:amenu ToolBar.bigfont :call ToggleBigFontSize()<c>
Next thing is to write the actual functions to toggle between dark and light color schemes. What I did was toggling between 2 light and 2 dark color schemes like that:
:function ToggleDarkScheme()
:  if exists('g:colors_name')
:    if g:colors_name == 'evening'
:      colo darkblue
:    else
:      colo evening
:  endif
:  else
:      colo evening
:  endif
:endfunction

:function ToggleLightScheme()
:  if exists('g:colors_name')
:    if g:colors_name == 'zellner'
:      colo delek
:    else
:      colo zellner
:  endif
:  else
:      colo delek
:  endif
:endfunction
Toggling of fonts had to be done a little different, because there's no global variable that vim would set when fonts are changing like the g:colors_name above. So I had to introduce my own: g:guifonts_name:
let g:guifonts_name = 'Lucida_Console:h10'

:function ToggleBigFontSize()
:    if g:guifonts_name == 'Lucida_Console:h10'
:      set guifont=Lucida_Console:h12
:      let g:guifonts_name = 'Lucida_Console:h12'
:    else
:      set guifont=Lucida_Console:h10
:      let g:guifonts_name = 'Lucida_Console:h10'
:  endif
:endfunction
Admittedly this is not a pretty and general solution: if I needed more colors or fonts I'd have to add more ugly if-else clauses. The ceneral soultion would be of course  to create a local array of color schemes and cycle through it like it is shown here*. But "form follows function..." and "use before reuse..." ๐Ÿ˜‰ - and frankly, I didn't have time for that, vimscript's syntax** is pretty weird.

--
* "Switch Color Schemes" in Vim Tips Wiki: http://vim.wikia.com/wiki/Switch_color_schemes

** There some good intros you can use to deepen your knowledge of vim scripting:

Saturday, 21 July 2018

More on Casablanca/cpprestdsk - Design and Comparisons


After I finished my Casablanca (aka C++ REST SDK)* project for some time then, I noticed** a new Boost library: Boost.Beast***, a library implementing: "HTTP and WebSocket built on Boost.Asio in C++11". 

That sounded interesting so I had a quick look at it, and I even found (surprise, surprise) the design rationale and comparison with other HTTP libraries. Because of my previous involvement with Casablanca* I simply had to read it. So here is my take on Beasts's critique and some closing discussion of Casablanca's* design and how I see it.

My involvement with Casablanca*/C++ REST-SDK code...
Let us start with a lengthy citation from Beast's analysis of Casablanca*, where the author lists all the missing customization points of the library:
"It is clear from this declaration that the goal of the message model in this library is driven by its use-case (interacting with REST servers) and not to model HTTP messages generally. We note problems similar to the other declarations:
  • There are no compile-time customization points at all. The only customization is in the concurrency::streams::istream and concurrency::streams::ostream reference parameters. Presumably, these are abstract interfaces which may be subclassed by users to achieve custom behaviors. 
  • The extraction of the body is conflated with the asynchronous model. 
  • No way to define an allocator for the container used when extracting the body. 
  • A body can only be extracted once, limiting the use of this container when using a functional programming style. 
  • Setting the body requires either a vector or a concurrency::streams::istream. No user defined types are possible. 
  • The HTTP request container conflates HTTP response behavior (see the set_response_stream member). Again this is likely purpose-driven but the lack of separation of concerns limits this library to only the uses explicitly envisioned by the authors."
Then the author restates it in a more concise manner:
"The general theme of the HTTP message model in cpprestsdk is "no user definable customizations". There is no allocator support, and no separation of concerns. It is designed to perform a specific set of behaviors. In other words, it does not follow the open/closed principle.
Tasks in the Concurrency Runtime operate in a fashion similar to std::future, but with some improvements such as continuations which are not yet in the C++ standard. The costs of using a task based asynchronous interface instead of completion handlers is well documented: synchronization points along the call chain of composed task operations which cannot be optimized away. See: A Universal Model for Asynchronous Operations (Kohlhoff)."
Let me state my opinion bluntly: I didn't miss these customization features at all during my almost 3 years long involvement with Casablanca*! The framework just did its work as it should, we could use it without much hassle and it performed reasonably good under load on Windows - probably through its usage of completion ports (Boost.Asio is only used on Mac Os and Linux).

OK, there was a single time where one of the above concerns did cause problems, namely:
  • A body can only be extracted once, limiting the use of this container when using a functional programming style,
but I solved it by setting the data into the message back again. Mischief managed ๐Ÿ˜ˆ, work can be continued! You could say it's not very functional programming style, but we are not in the academy but in real life.

On the other side the Boost.Beast itself library seems a little hard to use, and more like a base for another libraries than a independent library on its own, as it is stated in the project's own FAQ:
"It is not the intention of the library to provide turn-key solutions for specific HTTP or WebSocket use-cases. Instead, it is a sensible protocol layering on top of Boost.Asio which retains the Boost.Asio memory management style and asynchronous model."
So OK, we must be aware of Casablanca's* limitations, but for the real REST use case they somehow carried no weight.

TL;DR: I cannot really relate to the critique stated in the comparison, for us Casablanca worked out pretty well in general and the product we build with it is alive and kicking!

Note: Here and here I found quite interesting results of a security review of Boost.Beast by Bishop Fox. They found vulnerabilities, and I don't want to even think about their analysis of Casablanca*! Fortunately our product was set up forca more or less trusted environment. But this could be maybe a material for a separate blog post.

--
* Casablanca is the old code name for Microsoft's C++ REST SDK and I'll use it in the rest of that post, because it rolls better off the tongue than cpprestsdk! Besides during the years I grew accustomed to it and even a little nostalgic...

** to be more specific, I noticed it through Meeting C++'s library reviews initiative!
*** see "Beast: Version 100, ACCEPTED to Boost!" here.

Thursday, 5 July 2018

QLabel swallows mouse clicks in RichText mode?


As I already said in the title:
QLabel swallows mouse clicks when in rich text mode, what can I do? Help! 
I ran into that problem in my new project just yesterday* and will document the solution here, because searching the Internet didn't result in a thorough solution ๐Ÿ˜ž.
 
Actually, I ran into two problems with Qt (I'm on the 5.9 version now). It started with my desire to change line spacing in the custom icon widget I inherited with a pile of legacy code in my new project. Somehow it wasn't possible with the standard C++ API, so I resorted to the proven Qt workaround - CSS stylesheets.

Oftentimes stylesheet support implements manipulations and parametrizations not available through C++ and this case is one of them. Because setting the stylesheet directly didn't seem to work, the equivalent rich text string would do the work (note that the line-height property is stated in percent values):
  QString txt = "<p style=\"line-height:80;\">text text text</p>";
That trick seemed to have removed the problem, but in reality I just didn't notice the consecutive problem I created with this!

The text I wanted to space was a subscript under an icon, which was set as a bitmap in the class containing both the icon and the text. What I didn't notice was the fact, that a touch/click on the icon would start processing but when the user clicked/touched the re-spaced text it wouldn't! And of course falling back to a non-rich text would fix that problem!

As it turned out, in rich text mode QLabel wouldn't receive any MouseRelease events! First of all I found a slew of Qt bug-reports: 12982, 2028, 24375, all of them rejected or simply not fixed. So is it a Qt bug? Why it's not fixed? And what is a workaround for it? So I started to search the web for hints. One info I found was a short explanation by Thiago Macieira:
"Probably not a bug. Do you get the mousePressEvent? The release event is always sent to the same widget that accepted the mouse release. And QLabel with rich text might contain links, so it may need to handle mouse presses."
Needled to say I was rather devastated - this seems to be the expected behavior, and if you want to change it, you have to reimplement it!!! Needless to say, I wasn't inclined to do so. I was slowly approaching desperation, but then I found another hint"you could try ... Qt::WA_TransparentForMouseEvents". And that put every piece of the puzzle in its place for me:

1. As Qt-documentation says on Qt::WA_TransparentForMouseEvents:
"When enabled, this attribute disables the delivery of mouse events to the widget and its children. Mouse events are delivered to other widgets as if the widget and its children were not present in the widget hierarchy; mouse clicks and other events effectively "pass through" them. This attribute is disabled by default."
2. QLabel might contain links in general, but in out case we just have some plain text which has to be styled, ergo mouse events are of no use to it!

3. We cannot force QLabel to not accept() the mouse events in that case (alas), but we can cut it off from them, so that it won't steal MouseRelease and its parent widget will still receive it.

So I wrote a following method for setting the labels of icons:
  void CIconCell::SetIconLabel(const QString& label)
  {
    // correct spacing in double row texts
    // - cannot be done with std. Qt-API!
    m_Text->setTextFormat(Qt::RichText);
    m_Text->setAttribute(Qt::WA_TransparentForMouseEvents, true); // otherwise rich-text QLabel blocks mouse events!

    auto richTxt = QString("<p style=\"line-height:%1;\">").arg(m_LineSpacingRatio);
    richTxt += label;
    richTxt.replace("\n", "<br />");
    richTxt += "</p>";

    m_Text->setText(richTxt);
  }
PS: somehow the code formatter seems to interpret <br /> as line break, it should be: richTxt.replace("\n", "<br />");

TL;DR;
So essentially all the information was out there, you only had to connect the dots. But this isn't cost free, so I publish the ready solution for the good of all Qt programmers ๐Ÿ˜‰.

---
* OK, not entirely true, but at least I started to write this post on the next day...

Tuesday, 7 November 2017

Abominable Types


As I recently seemingly wasn't able to finish any of the blogpostst I started*, let's have a change of mood and begin with something simple, for example with "abominable types".

What are they? And why did I never hear of them? The answer to the first question is - they are part of C++'s strange menagerie of spooky beasts, of which there are:
  1. nasal demons 
  2. SCARY iterators 
  3. abominable types 
  4. etc... ;-) ** 
Why did I never heard of them? No idea, probably because only compiler writers and template library authors needed to know about them (except from full-time language lawyers of course!). But now, as of appearance of the proposal P0172r0 this language quirk achieved a certain publicity.

1. What they are 

I said I never heard of them, but I know at least why they are called as such - the name seems to originate from Alisdair Meredith:
So what are they again? As stated in P0172r0, this are function type bearing cv or ref qualifiers, for example:
  void foo() const;
  void foo() const volatile;
  void foo() &&;
What? We all know that we cannot define a function with a const qualifier outside of a class, so are these definitions even valid? It seems they are (in a context, more about later). As it is again stated in the told proposal the reason for their mere existence is the desire to enable the following construct:
  struct host {
     int function() const;// { return 42; }
  };

  template <typename TYPE>
  constexpr bool test(TYPE host::*) {
      return is_same_v<TYPE, int() const>;
  }

  constexpr auto member = &host::function;

  static_assert(test(member), "This is why abominations exist");
i.e. checking if some member function is const. As we can see, the problem with abominables is that they aren't function types of member functions but are not types of free functions either! They are halflings, robbed of their context, but coming to life again when provided with such.

By the way, there is a nice, little known trick exploiting the usage shown above (code copied from P0172r0):
  class rectangle {
    public:
      using int_property = int() const;  // common signature for several
methods

      int_property top;
      int_property left;
      int_property bottom;
      int_property right;
      int_property width;
      int_property height;
  };
In the words of the proposal:
"... examples ... explicitly writing these types fall into the category of showing off knowledge of the corners of compilers, and winning obfuscated coding contests."
Facebook's folly::function seems to use it in that manner nonetheless:
  using ConstSignature = ReturnType(Args...) const;

  using NonConstSignature = ReturnType(Args...);

  using OtherSignature = NonConstSignature;

2. The proposal 

Because some (very smart) library writers indeed used the abominables, there was some confusion about the exact goals of the proposal:
So what is the proposal trying to achieve?

The main concern is that when writing generic code we have to consider these function signatures on the overloads, multiplying theitr number by 6. As stated in the proposal:
"In many cases, the simplest option for handling abominable function types in a trait (or other metaprogram) is simply to state that they are not supported. The main problem here is a lack of vocabulary..." 
so you have to fight with them..***. A smaller problem is that they stand out from the type system because they are crippled (remember I called them halflings?)...

OK, I digress. The proposes changes are 1. kill abominable types, 2. make them non-functions, 3. make them regular, 4. minimal library cleanup (along the lines of 2.).

As it seems, 1. was ruled out and 2. is favored. This seems logical, as the abominables are not functions, they are halflings, so they should be demoted to something less than a function - a function-like type!

PS: Maybe there is a newer version of the proposal or a some decision about its fate by now, but as I stated at the start of this post, I just wanted to finish what I started back then.

--
* and I just hit the round number of 100 blogpost drafts... :-/
** look up some of the menagerie here!
** like in c17-features-and-stl-fixes-in-vs-2017-15-3: "std::decay now handles abominable function types (i.e. function types that are cv-qualified and/or ref-qualified)"

Saturday, 24 June 2017

QtCreator's debugger broken on Mac???


Problem

As the product of my client is running on both Windows and Mac, we have got our fair share of Mac bugs to be fixed. Yes, we really do, despite the fact that we are using Qt as our cross-platform portability layer! Write once, test everywhere, again... ๐Ÿ˜ž

Unfortunately, we had repeatedly encountered problems with debugging. As we were using QtCreator* on Mac to do Qt-development, we were trying to use its (i.e. QtCreator's) debugger but unfortunately we ran into problems when trying to use breakpoints.

Namely, the debugger seemed to ignore the breaking points we set, as it didn't stop the execution for them! Interestingly, you could step through the code and reach every nook and cranny of it, but you cannot let it run and wait till a breakpoint will be hit. So we had to set our breakpoints directly with LLDB and debug there - no graphical GUI and IDE goodies.

It's needless to say, that this constituted a major obstacle in our quest to eradicate all bugs on Mac. After the annoyance level has hit the high-water mark, my two brave colleagues Saลกa and Bojan came up with a solution which was quite surprising. I just want to share with you, my googling fellow-programmer, and with the rest of internet for the greater good...

Solution

It turned out, that the problem was due to the format of the debug information generated by qmake - it contained relative source file paths, like ../source.cpp. When used directly in LLDB, this didn't pose any problems, everything went well.

Unfortunately, when trying to set a breakpoint out of Qt-Creator, this started to be a issue, as Qt-Creator used full file paths when trying to set a breakpoint with LLDB. You see, file paths didn't match, this just couldn't work! For the sake of completeness it has to be mentioned that QtCreator does not have its own debugger - it just integrates Clang's LLDB and provides an comfortable GUI for it (but you knew it already, I guess...).

The solution was blatantly obvious - one just have to coerce qmake to create debug information containing absolute source file names. This can be done with the
QMAKE_PROJECT_DEPTH=0
option! Done and done!

The only problem is that this isn't an official option but an undocumented one! But it saved the day for us, so we will stay with it for the moment. If you want to read it, there's an interesting article "Undocumented QMake" on the Qt-Wiki. What it says about this special option is:
it seems this represents the number of sub folder that qmake refer to with relative paths (hardcoded is depth=4 in makefile.cpp) ...
As it seems, setting it to zero will force the usage of absolute paths.

Conclusion

As I said earlier, this all is due to Saลกa (and Bojan) so don't give me any credit for that! I'm only the messenger.

As a last word: we are using Qt version 5.6.1 at the moment.

--
* with Clang as compiler and and its LLDB as debugger


Monday, 8 May 2017

Qt and "a missing vtable usually means the first non-inline virtual..."


This post is a kind of a "thank you" for a really good piece of advice from a fellow programmer. As it is published on StackOverflow, and I for my part don't use it very much and cannot even upvote it, this post should spread the word about it instead!

Or, alternatively, you can see that post as another installment of an engineering notebook, next one in the series on bug hunting or another piece of advice for the googling programmer*.

The Problem:

I was checking in my changes including promoting a class to a QtObject and everything seemed to be working at first - I tested it locally, it seemed to build in TFS (Team Foundations Server, i.e. in our Windows CI build) but in TeamCity (i.e. in our Mac build) the build was broken with an ominous error message:
NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
And it came from the linker! Initially I supposed some Clang peculiarities (or even bugs) to be held responsible for that, and as I don't have a Mac to check this, I was a little lost for ideads.

The Solution:

But then I found this piece of advice:
Another possibility is that the class in question once didn't belong to Qt meta object system (that is, it had no Q_OBJECT or maybe didn't inherit from QObject at all), so qmake needs to be run again in order to create the necessary rules for MOC. The easiest way to force qmake to be run is to make some insignificant changes to the project file to update its timestamp, like adding and then removing some white space.
As I said, I cannot upvote it (you need at least 15 reputation points for that) so I just say thank you Sergey!

Because... that's exactly it! The missing virtual methods belonged to Qt's signal infrastructure which should be generated by the MOC compiler!

The error didn't manifest itself locally because I generated a Visual Studio project from Qt's .pro files, thus the new MOC file was forced to be created. Thus the problem disappeared trivially.

The same was done in our Windows CI build, but this time only Makefiles were generated and only the "naked" Microsoft compiler was invoked in command line mode to process them. Anyway, the same applies here, MOC creation is forced each time.

On the CI build server however, the "naked" .pro definition files are used, and qmake must be nudged a little to wake up and do its job.

The Moral:

Qmake doesn't like it when we change our minds ๐Ÿ˜‰.

--
* You've got it? Working mathematician etc..? ๐Ÿ˜Š