A new code example for the "Qt Performance" book

Recently I added a new example* to my "Qt5 Performance" book's resources that I didn't manage to include in the original release - the trace macros creating output in chrome:tracing format. 

I think it's a very cool technique - instrument your code and than inspect the result as a graphic! On Linux we have flame graph support in the standard tooling integrated with Qt Creator.

On Windows, however, we do not have such a thing! And because I decided to use Windows as the deveolpment platform for my book, we have a kind of problem here, so creative solutions are needed!

And I already came up with an idea for that in the book - we can use libraries (like minitrace, the library we are using in this example ) to generate profiling output in a format that the Chrome's** trace viewer will understand! You didn't know that Google's Chrome had a built-in profiler? Some people assume it’s only for profiling “web stuff” like JavaScript and DOM, but we can use it as a really nice frontend for our own profiling data.

However, due to lack of time I couldn't try it out when I was writing the book 😞, but with the new example I added  I was able to generate output you see visualizedin the figure below:

Here is the code example I used - as you can see, you have only to insert some MTR_() macros here and there:

  int main(int argc, char *argv[])
  {
    mtr_init("trace.json");

    MTR_META_PROCESS_NAME("QmlWithModel");
    MTR_META_THREAD_NAME("Main GUI tread");
    MTR_BEGIN("main", "main()");

    QCoreApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
    QGuiApplication app(argc, argv);

    QStringList countryList;
    countryList << "Thinking...";

    MTR_BEGIN("main", "load QML View");
    QQuickView view;
    QQmlContext *ctxt = view.rootContext();
    ctxt->setContextProperty("myModel", QVariant::fromValue(countryList));

    view.setSource(QUrl("qrc:/main.qml"));
    view.show();
    MTR_END("main", "load QML View");

    // get country names from network
    MTR_BEGIN("main", "init NetworkMgr");
    QNetworkAccessManager networkManager;
    MTR_END("main", "init NetworkMgr");
 
    ....

PS: heob-3 (i.e the new version of the heob memory profiler we discussed in the book) has a sampling profiler with optional flame graph output!!! I think I have to try it out in the near future!

--
* you can find the new example here, and the generated JSON file here

** Chrome browser's about:tracing tool. You can use it like this:

• go to chrome://tracing in Chrome
• click “Load” and open your JSON file (or alternatively drag the file into Chrome)
• that's all
  
  

More about basic QML optimizations

You might perhaps already know that I recently wrote a book (blogged about it here)! In the book we discuss various general program performance topics but also some more specific, Qt related ones, for example Qt's graphical or networking performance.

Recently, as I was watching some QML presentation I realized that some very basic pieces of performance advice I just glossed over in the book could (and should) be explained in a much more detailed manner to build a general understanding of this technology's caveats.

In Chapter 8.4, where the book discusses QML's performance, I simply wrote that:

"If an item shouldn't be visible, set its visible attribute to false, this will spare the GPU some work. In the same vein, use opaque primitives and images where possible to spare GPU from alpha blending work."

As I re-read this section on some occasion I somehow had to notice that what seemed pretty clear in writing, isn't that clear when read a year later! I started to think why this passage is not that lucid as it should be, and I noticed, that my intended formatting was gone! Now we have in the book:

"... set its visible attribute to false" 

instead of

"... set its visible attribute to false"

as I wrote it initially!!! The formatting went somehow missing and I didn't notice it while proofreading. For me this change rendered the sentence pretty unintelligible :-/.

But anyway, I could have explained it in a little more detailed manner* - so let's try to make a better job and explain why the visible attribute is so important!

To state it bluntly - QML won't do any optimizations to prevent drawing items the user cannot see, like items which are out of bounds or are completely obscured or clipped** by other items! It just draws every single item with visible set to true, totally unconcerned with its real visibility! So we have to optimize manually and:

"... set its visible attribute to false"

But how on earth should we know which items are invisible in which situation? Easy, just set this environment variable: QSG_VISUALIZE=overdraw and Qt will show the overdrawn items using a different color and making them visible as can be seen in this pic from Qt documentaion:

That's pretty cool!

This was my state of knowledge until recently (learned about that in this KDAB presentation) but than I read this line in Qt Quick 2D Renderer's documentation:

Qt Quick 2D Renderer will paint all items that are not hidden explicitly with either the visibility property or with an opacity of 0. Without OpenGL there is no depth buffer to check for items completely obscured by opaque items, so everything will be painted - even if it is unnecessary.

Qt Quick 2D Renderer is a raster-graphic replacement for the standard OpenGL-based QML renderer. Does that (by tertium non datur) mean that with OpenGL we do use the depth buffer after all? This could be an unfortunate formulation, so we'd like to stay on the conservative side but maybe a little more investigation would be appropriate in this case. Have you heard anything about it?

--
* This might be due to the very tight deadlines :-(

** as Qt documentation says:

"If clipping is enabled, an item will clip its own painting, as well as the painting of its children, to its bounding rectangle."

Lippincott Pattern

1. Intro

In one of my recent C++ projects I spotted some giant macro named HANDLE_ALL_EXCEPTIONS() (or so), immediately noted how ugly it was and that a better solution to that problem exists. 

My coworkers at the then-customer were nice and open minded people so they didn't bridle at that, on the contrary, they were eager to change it and asked for advice. I simply said that they should google for the "Lippincot Pattern" and thought the matter were settled.

To my surprise my buddy returned reporting that there's nothing like that on the Internets! As to remedy that sore state of affairs I decided to make a short write-up of that really cool technique.

2. On Naming

An astute reader might have noticed that the whole problem is an artificial one: there is some (admittedly still too little!) information on the "Lippincot function" out there*. And Jason Turner (aka @lefticus) made an entire episode of C++ Weekly about it. So why I'm insisting on a different name?

Well, to be honest, it's mainly because I learned it by this name! Sadly, I cannot find any article on the web, so I cannot present you with any proof, but I assure it that it is true. Has anybody besides me heard of this technique under the name Lippincott Pattern instead of Lippincott Function? Please leave a comment if you did, I'm really curious about it!

But besides of my inability to change my habits there indeed is a genuine argument for that name. I mean, it is a known technique solving a common programming problem and that's the definition of a programming pattern if I'm not mistaken! And it's of no importance if we are using a function or a set of classes to solve the problem.

And of course, it sounds much better, and, without a doubt this, counts as a third reason. 🙂

As we are in a section called "On Naming" you might ask why this technique is called Lippincott function/pattern? Elementary, Dear Watson - it's beacuse it was invented by Lisa Lippincott, a well-known programmer and (as of recent) a C++ committee member with a faible for maths:

Day 3 Keynote by Lisa Lippincott:
“The Shape of a Program”
Mathematical topology in programming. #ACCUConf pic.twitter.com/qudggtGJsj

— Victor Ciura (@ciura_victor) April 13, 2018

3. The Pattern

But enough of the idle banter, and ad rem! I maybe should have mentioned that this technique is also sometimes called "Exception Dispatcher". Now you probably can already imagine how it works.

   try
   {
      throw;  
   }
   catch (const MyNetworkException& ex)
   {
      MY_LOG_ERROR(tags::Networking, QString("Exception occurred: ") + ex.what());
   }
   catch (const std::exception & ex)
   {
      MY_LOG_ERROR(tags::Networking, QString("Unexpected exception occurred: ") + ex.what());
   }
   catch (...)
   {
      MY_LOG_ERROR(tags::Networking, "Unknown exception occurred.");
   }

As you see, we rethrow the current exception (notice that we assume we are in an exception handler!!!) and can write reusable exception handling logic without macros! We use it simply like this:

   try
   {
      a_function_that_may_throw();  
   }
   catch (...)
   {
      traceException(); // Lippincott!
   }
   

Here we just log the error and do nothing, but we could as well translate exceptions in error codes, as it is shown in the already mentioned blogpost*, and then use it like this;

   try
   {
      a_function_that_may_throw();  
   }
   catch (...)
   {
      return translateExceptionToErrno(); // Lippincott!
   }

However, that's not all - we can also here more complex logic. Let us have a look at this exception handler function from my recent HTTP project**:

  int CasablancaRestClient::handleException() const
  {
    QString errText;
    int errorCode;

    try
    {
      throw; // Lippincott pattern 
    }
    catch (const web::http::http_exception& e) 
    {
      // probably TCP conn. error!
      //  - notify conn. status change, trigger HTTP fallback if needed
      return HandleHttpError(e);
    }
    catch (const web::uri_exception& e)
    {
      QWriteLocker guard(&_serverAliveLock);

      if (!_serverAlive)
      {
        // probalbly server's URL not yet set
        errorCode = EC_NO_CONNECTION;
      }
      else
      {
        errText = tr("Internal error, bad URL: %1.").arg(e.what());
        errorCode = EC_BAD_URL;
      }
    }
    catch (const web::json::json_exception& e)
    {
      errText = tr("Internal error, bad JSON data: %1.").arg(e.what());
      errorCode = EC_JSON_FORMAT;
    }
    catch (const std::exception& e)
    {
      errText = tr("Internal error, reason: %1.").arg(e.what());
      errorCode = EC_INTERNAL;
    }
    catch (...)
    {
      errText = tr("Internal error in CCasablancaRestClientComp!!!!");
      errorCode = EC_INTERNAL;
    }

    WIN_DEBUG_CLIENT_ERR(errText.ToStdString());

    // send the error to GUI context
    emit connectionErrorMessage(errText);
  
    return errorCode;
  }

We see, we are just doing the basic translation from exception types to error codes, but also initiate a reconnection or fallback to a non-secure connection in some cases!

I was using it in following manner:

  try
  {
    sendHttpRequest(uri, data, headers);
    return EC_OK;
  }
  catch (...)
  {
    return handleException(); // Lippincott!
  }  

The beauty of this lies in its conciseness - no copy-pasted code, no macros, just a natural function call somehow obliterating the ugliness of the try-catch blocks.

4. "Modern" Lippincott variants

What we have seen above was the plain, basic, C++98-esque usage. But C++ wouldn't be itself, if we couldn't complicate things in name of progress and fashionable gimmicks 😇.

Just have a look at this code taken from the already mentioned article*:

  foo_Result lippincott()
  try
  {
     try
     {
        if (std::exception_ptr eptr = std::current_exception())
        {
           std::rethrow_exception(eptr);
        }
        else
        {
           return FOO_UNKNOWN;
        }
     }
     catch (const MyException1&)
     {
        return FOO_ERROR1;
     }
     catch (const MyException2&)
     {
        return FOO_ERROR2;
     }
     catch (...)
     {
        return FOO_UNKNOWN;
     }
  }
  catch (...)
  {
     return FOO_UNKNOWN;
  }

First irritating thing here is the function-scope try block. C++ allows you to wrap function body in a try/catch clause like this:

  ErrCode getSomeData()
  try
  {
     // do sth....
  }
  catch (...)
  {
    return PANIC_ERR;
  }

What is the purpose of this fature, you might ask? As cppreference.com states:

"The primary purpose of function-try-blocks is to respond to an exception thrown from the member initializer list in a constructor by logging and rethrowing, modifying the exception object and rethrowing, throwing a different exception instead, or terminating the program"

So there aren't any corner cases to justify its usage, and our example we already handle exceptions inside of the function  OK, now we can shed this syntax noise off. The second oddness is the usage of std::current_exception() and std::rethrow_exception() - what it is for? 

When you call std::current_exception() from within your function you can chek if there currently is an exception being handled and if it returns a nullptr then there is no exception active. Thus in the discussed code, we, in a somehow paranoid manner, are making sure that the exception dispatching function was called from the exception context - so to say implementing a "safe Lippincott" pattern. But ask yourself - would a programmer use a Lippincott function outside of a catch() block?  Well, maybe.

However, not all modern features are bad - far from that! Look at this elegant technique that uses lambdas:

  extern "C" errno_t my_amazing_c_function()
  {
    return translateException(
      [&]{
        // ... C++ code that may throw ...
      });
  }

Here we wrap code that may throw in a lambda and pass it to a  generalized exception translation function which can be implemented like that:

  template<typename Callable>
    ErrorCode translateException(Callable&& f)
  {
    try
    {
      f();
      return NO_ERR;
     }
     catch (...)
     {
      return translateExceptionToErrno(); // Lippincott!
     }
  }

Cool, innit?

P.S.: "Ceterum censeo exceptiones delendam sunt..." - M. Portius Cato

--

* for example here: http://cppsecrets.blogspot.com/2013/12/using-lippincott-function-for.html plus many mentions of this article on Stack Overflow.

** Look up the Casablanca post: http://ib-krajewski.blogspot.com/2015/09/casablanca-c-rest-framework-one-year.html

The pain(s) of C++11

Recently I wrote a small cross-platform library of widgets and didn't think much about the C++ version to be used, as my client's primary platform was Windows plus the latest Visual Studio with C++17. As the Microsoft compiler was normally the slowest to implement new standard's features I didn't worry about problems when compiling the code with GCC on Linux and just settled on the C++17 standard (or at least the subset of it supported by the used Visual Studio version).

However.... it all came out rather differently. 

 

First, I had to back-port it to Visual Studio 2015, which theoretically supported C++17, but didn't have std::optional in its STL 😨, but this is material for another article...

If that weren't enough, it turned out that the GCC version used in our build server doesn't support C++17, so I had backport it again to C++11. OK, I thought, C++11 was the giant step for C++ so it shouldn't be that difficult to go back, all the most important language features should already be present there. 

You'll imagine my surprise, when this innocent code snippet failed to compile on the build server!

   // impl. helpers:
   template<typename Enum>
      constexpr auto enumToInt(Enum e) noexcept  // C++20: requires(std::is_enum_v<Enum>) 
   {
      return static_cast<std::underlying_type_t<Enum>>(e);
   }

You see, optimistic as I am, I was already thinking about what would be possible in C++20!

I won't show you the exact error message because it is incomprehensible for most people*. Enough to say it wouldn't compile.

So this is what I had to use instead:

   template<typename Enum>
      constexpr auto enumToInt(Enum e) noexcept        // C++20: requires(std::is_enum_v<Enum>) 
        -> typename std::underlying_type<Enum>::type   // C++11 backward compatibility :-(
   {
      return static_cast<typename std::underlying_type<Enum>::type>(e);
   }           

As we can see, I needed first to sptinkle some typename keyword's instances all over the place to help and guide the compiler in parsing of the template. Secondly, in C++11 the XXX_type_t shortcuts for the standard library types aren't available. I had also to explicitely specify the return type of the lambda.

You see - much more mindless typing in the C++11 version! In some respect it's almast as verbose as the older Java releases. And not to mention lots of time used for deciphering template error messages to find out those problems!

Conclusion:

In conclusion I must say, we should appreciate C++14/17 much more than we do, because it really saves us the time to write and later to read the code!!! Somehow I wasn't fully aware of that.

P.S.:
Imagine, at first I was even tempted to use this monster:

   template<typename Enum>
      constexpr auto enumToInt(Enum e) noexcept 
        -> decltype(static_cast<typename std::underlying_type<Enum>::type>(e)) // C++11 backward compatibility :-O     
   {
      return static_cast<typename std::underlying_type<Enum>::type>(e);
   }   

--

* Personally. I always found the complaints to be somehow exagerated - with a little practice you can find the problem in not too long a time.  But it is a quest and not an automated procedure. On the other hand, isn't it always like that with compilers? Good error messages are rather a recent invention and still more like an exception than a rule.

A study in bad naming

Recently, while working for one of my clients on someone other's code I spotted the following definitions of members in some class:

What's that???!!!  Can you see it? Are you as much shocked as I was on its first sight? You are not? OK, I will explain my abomination - every declaration had to be commented! And that's not enough - the comments are essential in understanding the purpose of each member!

Why I do not like it, you might ask? Well, I'm lazy and commenting each declaration seems like too much work... You might ask 'why?' again - well, if I would like to add some definition to this code, I'd have to write a comment as well in order to maintain the (arguably flawed) coding style! I'm only a service provider for my client and wouldn't like to break this code's look and feel.

As you might know I'm not a friend of the newfangled "no comments" politics, but this is an example where we could definitevely use some of it. But why's that so bad again? It's because you could fix the example code with careful usage of naming* and it's even not that difficult! 

Let's have a go at it:

  /* State m_state => */ State m_dateValidity;

here we don't have to include "state" in the name, as it is already stated as the domain of the variable.

  /* QAction* m_action => */ QAction* m_datePickerPopupAction;

Here, however, we somehow inconsistently included "Action" in the member's name mainly because of my vague gut feeling. In this instance we could indeed try to improve the naming by considering shortening it, maybe even to m_datePickerAction;

  /* QDate m_date => */ QDate m_parsedDate;

A no-brainer.

  /* QStringList m_parseFormats => */ QStringList m_dateParsingFormats;

Another one!

  /* bool m_twoYearIsPast => */ bool m_treat2DigitYearsAsPast;

Here we could try to make the name a little better - maybe m_twoDigitYearsInThePast? 

  /* QDate m_minDate => */ QDate m_earliestDate;

As minimum and maximum are't commonly used with dates.

  /* QDate m_maxDate => */ QDate m_latestDate;

Idem.

  /* bool m_justFocused => */ bool m_focusReceived; // used for workaround ABC... 

Here I see the only place where a comment is needed.

So, what do you say? It wasn't even that difficult. Arguably the result isn't perfect but it's a lot better than the original. Maybe it's even good enough? 

So watch your names, they are important - because code readibility is important!

--

* As they say -"there are only 2 hard problem in comp. sci. - cache invalidation and naming things"...

A technique for connection checking in Qt

Recently, I stumbled upon a technique for checking if the connect() call for Qt signals/slots was successfull. You may ask what the problem is - just check the result of the call, how difficult can that be?

  bool ok = connect(...);
  Q_ASSERT(ok);

This doesn't look as it's much! However, in a standard Qt project you won't see that because it doesn't scale really well. What you'll normally see is this:

  connect(a, SIGNAL(...), b, SLOT(...));
  connect(c, SIGNAL(...), d, SLOT(...));
  connect(e, SIGNAL(...), f, SLOT(...));
  // etc..

Repeated insertions of Q_ASSERT(ok) would kill readability in such case! In a couple of projects I've seen that people were trying somehow to have the cake and eat it - do not sacrifice readibility but nonetheless report a failed connect. This was done by means of custom Qt builds, where the connect() function was changed to include an assert or to throw an exception in case of failure.

Digression: I'm normally not a fan of exceptions but here they seem to be perfect a perfect match, don't they? Just signal some error condition without polluting the code with error handling! What's here not to love? But on a closer inspection it poses problem in Release builds, as in production environment we do not want to crash a program when some connection are not right. Ok, with enough testing and exception handling code it's not a problem, but asserts are a much simpler method to achieve the same goal.

Technique:

So let us proceed to the advertised technique:

    v << connect(...);

How is the shift omperator implemented? Let's have a look:

    ConnectionVerifier& ConnectionVerifier::operator<<(const QMetaObject::Connection& connection)
    {
        verify(connection);
        return *this;
    }

    void ConnectionVerifier::verify(const QMetaObject::Connection& connection)
    {
        const bool connected = (connection != nullptr);
        //Q_ASSERT(connected);
        if (!connected)
        {
            throw Exception("Could not establish signal-slot connection.");
        }
    }

Now we can easily scale:

  v << connect(a, SIGNAL(...), b, SLOT(...));
  v << connect(c, SIGNAL(...), d, SLOT(...));
  v << connect(e, SIGNAL(...), f, SLOT(...));
  // etc..

Well, this is the code as I found in to be used in the project - somehow the dychotomy of exceptions and asserts we discussed before wasn't resolved by the authors. Admittedly, both alternatives have their merits.

Summing up:

So will I use this technique? Well, it's most useful for the old connect() syntax where the signals and slots are passed as (Q)strings using the SLOT() and SIGNAL() macros - here it is very easy to accidentally misstype the name or parameters of the corresponding slot/signal, believe me,

But with advent of Qt 5 we have also the modern, type-safe syntax alternative:

  connect(a, &A::singnalA, b, &B::slotB);

which I'm using quite exlusively right now. Theoretically there are some corrner cases where it also could fail, but the standard problem we had in the old days, i.e. the typos, is solved now.

However, if you inherited some giant legacy Qt application wehich uses the old connect() syntax, you might be thankfult to have this trick up your sleeve!

Example Programs form the Qt 5 Performace Book

Hi all! You will meanwhile probably have learned that I wrote a book about C++ program performance in the context of the Qt framework (you can have a look at the TOC here).

But along the book I also created quite a few example programs, which, as a rule, aren't discussed in the book. Some of them are pretty nice, so I decided to write up what they are and what they do.

There are two kinds of example programs in that collection: ones that illustrate some performance optimization idea, but also ones which just show how to use some of discussed Qt features. So if you want just learn some Qt usage, you might find it interesting. Also if you are interested in details on the techniques mentioned in the book, you'd better look into the examples, because the book didn't have place for step-by-step exlanations.

Chapter 1: Intro

At the moment no code and no resources here. Sorry. Some examples for usage of basic performance techniques would fit here in nicely, so maybe in the future...

Chapter 2: Profiling

QmlWithModel

This is the program we will be using to show how to take advantage of profiling tools. It has a QML GUI showing all the countries where Qt is used. This list is fetched with an HTTP request from the internet:

As you can check in its code, we can enable:

•  a memory leak and 
•  excessive burning of CPU cycles 

in order to show how to find these problems using profilers.

Additionally, there's a recorded ETW trace file of the above application to be browsed with the UI for ETW tool mentioned in the book:

Here you can take your time and look into different metrics which can be collected with ETL!

Addendum: I always wanted to include an example usage of  some flame graph generators in your code, but never got round to it. Maybe I'll find some spare time for it.

Chapter 3: C++ and Compilers

ClassicTemplateTricks

Here we show some of the template techniques mentioned in this chapter, e.g, expression templates for optimized string concatenation, CRTP for avoiding run-time polymorphism and compile-time calculations using recursive templates.

Addendum: Because Qt 5.12 required C++11, the techniques shown in book are the ones supported by that standard. But for example the expression templates could be replaced with fold expressions in C++17!*

Compiler Explorer Tests

Some code you can copy and paste into Matt Godboldt's Compiler Explorer and look at the assembly it produces. Here we can observe such optimization as:

- compiler optimization examples: the basic examples discussed in the book - avoidance of direct summation, elision of allocations, etc.

- constexpr factorial: here you can see the proof that a factorial can be calcualted at comple time! This time without templates, but using constexpr instead!

- constexpr hash: and here is the equivalent proof for a compile-time hash function!

- restrict and pure: here we can see how the restriced and pure attributes can guide the compiler for better optimization and how it's absence worsens the generated code. Unfortunately these attributes are only avaliable as compiler extensions.

- UB examples: here we have a few UB cases mentioned in the book

So just grab the snippets from example files, paste them into Compile Explorer an see live how different compilers optimize your code behind the curtains!

MemoryMgmExamples  - these are examples of a custom memory managers and allocators. They will trace to cout to show when they are used:

MoveTest - in a similar manner, an example usage of the move semantics, also tracing to show that it's working:

Chapter 4: Data Structs and Algorithms

Interning of strings - an example implementation of string interning. As this would be to detailed to explain in the book, I just provided some code for the curious to read through:

Addendum: with Visual Studio compiler we can activate string pooling option to automatically remove duplicate strings!

Schwartzian transform - here we are using the ominous Schwarzian transforrm to paint triangles and christmas trees 🎄!

StdSearch - here we show the usage of the new C++17 Boyer-Moore searcher

Chapter 5: Multithreading

ActiveObject - an implementation of the "Active Object" pattern. You surely remember that we can use this pattern to implement a (nearly) share-nothing multithreading scheme, do you?

ConcurrentAndFutures -  here we load images asynchronously using futures and can compare this to a simple synchronous loading strategy by pressing the "Load All at Once" button:

QFutureInterfaceExample - this is an example explaining usage of the undocumented QFutureInterface class which is widely used in Qt Creator

ThreadsAndTimers - basic example for starting timers in threads (because it can be somehow confusing)

Chapter 6: Case Studies

In this chapter we could do with some more examples, don't you think? Because we have only this single one:

SvgCache - here I show an implementation for the SVG cache we discussed in 6.4.

Chapter 7: I/O and Parsing

JsonExample - example of basic JSON parsing, nothing special, I know

MemMappedFile -  a hands-on demonstration of how to map a file onto memory:

SqlLiteExample - shows example usage of the SQLite database.

First we have to create the DB offline:

then we wil perform some simple SQL manipulations on it:

StreamReaderExample - again a basic example: parsing XML data this time

Chapter 8: Graphics

These examples mainly illustrate the discussion of various Qt graphic interfaces, but the minimal FPS counter can be used as a performance tool as well.

CustomQmlElementsTest - here we paint a simple triangle in QML using several ways to define our own, custom QML element. The FrameBufferTriangle uses custom OpenGL painting on current OpenGL context.

FpsCounterExample - want to know how to implement a live FPS counter for your application? Here we use a particle system to stress the QML runtime measure (and display it in an extra widget!) the FPS rate that can we are able to achieve:

OpenGLWidgetExample - we use Qt's OpenGL support for widgets to draw this classic triangle picture:

It used the most basic (and old...) OpenGL 1.0 API, but hey, this book isn't a graphic tutorial - I just needed this picture for the explanations of how the graphics pipeline is working. Please bear with me.

Qt3DSphereExample - look what is possible with Qt 3D! We animate a rotating sphere using the Phong lighting model:

It is also a very basic program, but the result is rather cool (at least for me).

Chapter 9: Networking

Here I wanted to write some basic demonstrators running client and server on the same machine to show basic usage of TCP sockets and HTTP transport mechanism.

EchoServerTcpExample - an echo server: the client will connect on a given port and the server will echo every data it receives from client!

HttpDownloadExample - here we will connect to a given web address, then fetch and save some resources from there. Additionally, the progress of file download is visualised with a progress bar.

If you have Python installed on your machine, you could start a local web server on some directory and then run this testapp locally. As I mentioned in the book, Qt doesn't have a HTTP server component and doesn't allow us to easily write one.**

Chapter 10: Mobile and Embedded

Here we do not have much resources (sorry, publisher's deadline didn't allow that 😒) but at least one example pertinent to embedded data presentation could be included.

OpenGLAcceleratedChart - an example of using OpenGL acceleration with Qt Charts. You can parametrize the number of points if you want to stress your hardware!

There is another example implementation I should have provided is the polyline simplification, but due to time shortages I missed out on this one. But at least I provided you with this link: https://www.kdab.com/a-speed-up-for-charting-on-embedded/, which describes the implementation of a polyline simplification technique that uses a flattened min-max tree.

Addendum: If I'll find some time, I could add some example project for my Raspberry-Pi, as I was initially planning. You know, the deadlines etc... :-\.

Chapter 11: Testing and Deploying

ExampleSubdirProjectWithTests - this is an example of how to add tests to an existing project using the Qt Creator:

The tests are wiritten using QTest library, and integrate nicely with Qt Creator's UI, as you can see on the above figure. Also a minimal Qt application is created to be tested here and in the Squish tests below.

SimplestQMLProject - this one is a very simple QML project we will need in the example Squish tests below.

Squish - here we have a definition of Squish tests for both Widgets and QML applications we've introduced above:

Summing Up

So, that's all. I hope you've found something interesting there!

---
* expression templates can be replaced with fold expressions (C++17) for string concatenation, as shown in this post: https://www.qt.io/blog/efficient-qstring-concatenation-with-c17-fold-expressions

** e.g. I couldn't port my old Qt4-based embeddable web server project to Qt5 because of removal of HTTP parsing classes.

My answer to a young programmer

You maybe know that I offer C++ mentoring via Twitter (DMs are open!) and, as it happened, someone approached me and wanted to be mentored. So I asked him for his questions and then wrote some responses, which ended up to to be qute long... Then the idea to share what I wrote in this blog - maybe someonw will find it interesing.

Without much ado, to the questions.

1. Questions

I first asked about the goals. Thes answers are probably in line with many young programmers want:

1. I just need to be able to solve programming problems and strong myself in
this part, to be qualified for big tech companies interviews like Google,
Microsoft and Amazon. 

2. No the programming language doesn't matter for me, I know the basics of C, C++ and Python but I just want to be a problem solver and think like a programmer 

3.  I just want to understand the theory behind Algorithms and Data Structure

Then also came a list of resources that one finds on the internet using uncle Google. I won't show them here, as thea are quoted in the response.

2 The Response

Here's my response, in full, without any later editing:

Hi Xyz!

I had a look at your references:

>Courses:

>Algorithms and Data structures by Revi
>https://youtube.com/playlist?list=PLEbnTDJUr_IeHYw_sfBOJ6gk5pie0yP-0

Here we have much complexity theory but little of algos - only basic sorting methods.

>Algorithms by Abdul Bari
>https://youtube.com/playlist?list=PLDN4rrl48XKpZkf03iYFl-O29szjTrs_O

Don't know what to think about it's TOC - maybe too academic (too much about recurrences) but also covers a broad range of themes you'll need. So-so.

>MIT 6.006 Introduction to Algorithms
>https://youtube.com/playlist?list=PLUl4u3cNGP61Oq3tWYp6V_F-5jb5L2iHb

Had a look the TOC and liked it! If you can grasp these themes, you'll have good working knowledge of Comp.Sci.

>Books:

>Data Structures and Program Design in C by Robert L. Kruse
>https://goodreads.com/book/show/27846.Data_Structures_and_Program_Design_in_C

Looks like it's pretty old. Probably outdated, looks like waste of time.

>Introduction to Algorithms By Thomas H. Cormen
>https://mitpress.mit.edu/books/introduction-algorithms-third-edition

This is a classic, maybe a little heavy on the academic side, but a good reference to check what the options are. For an algorithms book the "The Algorithm Design Manual" by Steven Skiena is a much lighter read I'd recommed!

>Think Like a Programmer by V. Anton Spraul
>https://goodreads.com/book/show/13590009-think-like-a-programmer

I don't know this one. Had a look at the TOC and it can be OK or even good. It has a couple of programming problems but the title seems to be kind of a clickbait. Notheless. maybe it's good.

>Practice:

>https://hackerrank.com

Don't know this one (sorry) and couldn't check it out, as it requires a login. Back in the day Google recommended topcoder.com for testing your problem-solving skills in competitions, but I don't know the current status of it.

>https://leetcode.com

Don't know this one (sorry), but it looks somehow inetersting at first glance. You have specific questions for Google, Facebook, etc. Could be worth it.

And responses:

>I just need to be able to solve programming problems and strong myself in this part, to be qualified for big tech companies interviews like Google, Microsoft and Amazon.

A classic book for that is "Cracking the Coding Interview" by Gyle Laakmann McDowell - but it's a real tome! "Programming Interviews Exposed" by J. Morgan et. all was a lighte read. "Algorithms for Intreviews" by A. Aziz & A. Prakesh has many tough problems, but the solutions aren't that well explained - a challenge!

>Q1: No the programming language doesn't matter for me, I know the basics of C, C++ and Python but I just want to be a problem solver and think like a programmer

To be a (real) problem solver you need some deep knowledge in at leas one OO-langauge! Programmers think in patterns, you have to learn them by solving programming problems! On a little higher level, you'll need at least cursory knowledge in design patterns - look at the 4 Amigos book!

>Q2: I just want to understand the theory behind Algorithms and Data Structure

You'll have to undertstand the O-notation at least and to be able to estimate the O-complexity of your algoriths. The Master theorem is not really needed though. You have at least to understand NP-complete complexity class (NP=P?). You'll have to be able to estimate run time of an algorithm - which algos would need thousands of years to complete?

Regards,
Marek

3. Summing Up

Hopefully some young programmer(s) will find it helpful!

Trip Report Qt Word Summit, Berlin, 5-6 November 2019

You've probably noticed that I'm not a hell of a conference goer, didn't you? The last conference I attended was the first "Meeting C++" conference in 2014, at that time still taking place in Neuss, Rhineland* instead of the worldly and hipster Berlin.

I tried to go to the two consecutive Meeting C++ conferences in Berlin, but I was both times unable to do it, so eventually I gave up and called it a day on this idea.

But because recently I got somehow more seriously involved with the Qt framework** I thought, hey, why not just go to this one Qt conference*** in Berlin and visit some of my people there as well? No sooner thought than done. And now, three months something later, another idea struck me - why not write up some impressions? So let us go over things I came to remember most vividly.

Of course you could start moaning that it is much toooo late for a trip report, but better late than sorry, so let us begin!

1. Lars Knoll's talk

Lars Knoll, the CTO of the Qt Company was giving a talk on general directions for the future Qt 6 development and it was rather interesting. You can check it out here (PDF), but basically QML will be reworked to be a typed programming language, Javascript will be only optional and graphics will support not only OpenGL but also newer APIs like Vulcan, Metal or Direct3D 12. All that's very interesting, but what struck me the most was the following slide:

As it seems Qt is predominantly used on Windows! That's kind of surprising, taking into account that all the books and articles you read always assume Linux as a platform! I personally did a lot of Qt work on Windows and couldn't ever understand that.

This slide gave me a nice, fuzzy feeling, because my decision to write my Qt book from a point of view of a Windows developer was a right one!

Other books always describe the tooling you can use on Linux, but this doesn't help me on Windows, because most of that tooling (open source) doesn't exist there. So writing of the Chapter 2 (Profiling and Profilers) was a kind of a discovery journey - how can I make open source tooling run and correctly work on Windows?

2. Keynotes

As much as I don't like keynotes - normally they are rather full of business-y bla bla - but one slide of the Natalie Nahai's presentation really impressed me, I mean this one:

Un ejemplo de @NathalieNahai de mejora de #UX: Más donaciones solo por cambiar la página: los botones, Obama mirando los botones.. En #QtWs19 pic.twitter.com/th2eC527q8

— fgutierrez (@fgutierrez_) November 5, 2019

You can see the change of the vibes of a webpage from the left (boring, lots of distractions, unusable!) to right (cool, clear, using 1-click donation buttons). Nice! And it really happened during Obama's campaign.

3. Caliatys hydrogen mobility talk

This one was also a keynote (or I think so...) but the CTO of the company has shown us the slide containing a decision table they used for choosing the Qt framework as a platform for mobile development. The application in question should run on both iPhone and Android device. After the recent switch of Dropbox away from a commonn C++ codebase for both platforms you would maybe expect that C++ isn't good for mobile.

See their reasons for choosing Qt in the pic below:

The cloud connector had to be written separately for each platform, apart from this Qt provided for platform-independence!

4. QML on Microcontrollers

I already said, that the new QML will be a compiled, statically typed language in Qt 6. Already today, we can have a foretaste of the future with Qt Quick Ultralite (Qt QUL) that can be run on MCUs (microcontrollers) - a feat in itself, as up to this day Qt and QML needed a much more beefy processors to be run onto! There were quite a few talks centering around that theme, because it seems to be the future of Qt.

Qt Quick Ultralite is thus basically a subset of QML which can be compiled to C++ and run on bare metal (or an RTOS). This is also a direction in which QML will be going in Qt 6. I was pleasently surprised to learn, that Qt QUL was developed by the Qt Company in cooperation with a polish enterprise that ported it to many MCUs (see the picture below 👇).

As it's my native country, I couldn't help but to be very pleased. Good job Poland! 😍

5. People and stories

As it goes, I stumbled upon 2 of my old clients at the conference! I tool the opportunity and asked them about how my old projects fared after I left. We had quite good chats and I wanted to talk to them on the next day also, but - you won't believe it - I couldn't spot them anymore! The conference is a pretty big event with masses of people and finding someone is rather matter of luck.

I also could (and was honored to) meet some, i.e. two 🙂, Qt-involved people, I only knew from Twitter, in the real world. As I expected. they turned out to be very nice indeed! Plus I also wound up talking to people in the queue to the canteen and at the exhibitor stands - I even could ask Qt's CTO a question face-to-face after his talk.

Now let us proceed to the "stories" part of this section:

One of my customers I met at the venue told me a nice story right form the threnches, They had a beast of a performance problem with the embedded device in my old project (accidentally same project as in the perf. story 6.4 in the book**), namely, after a day being powered on, the device started to slow down up to a total grind - the UI became totally unresponsive and you had to reboot it. What is your guess for the reason of that?

Yes, you are right, a memory leak.

But you'll never guess what couse it! When displaying the time of the day, on every update (i.e. every second) the displayed string was changed and then restyled using CSS styles. As it turned out, there was a bug in the Qt version we used and restyling leaked memory at that place! How could you possibly find that?!?!, I asked, quite shocked. Well, it was accomplished by the old and proven method of pinching off parts of the programm until the problem disappears. A kind of "divide and conquer" of debugging. The solution? Quite a pragmatic one - just don't use styling in the updates. Unfortunaltely, I forgot to ask how's the look and feel of the widget now.

Want another story? Here we go - I met 2 guys in the queue in the canteen and chattet with them for quite a long time. At the evening, I went out with my Berlin people to their favourite craft beer bar in the neighbourhood. After a couple of pints I catched a glimple of some guy at the counter and had to ask myself - do I know that guy? No, impossible, only locals frequent this bar... But after a couple of further glimpses I just had to settle that and chatted up the guy and his friend. Guess what - they were the exact people I met at the conference. Then we had one more chat and I even could pitch my book** to them! Moral - everybody can google, there isn't such a thing like a hidden gem or an insider-only venue. 😔

6. Merchandise

Of course I also got some merchandise:

• a Qt sticker set 📗,
• a T-shirt and a lanyard keychain from Qt 👉,
• sleeping mask (Batman style!) 🦇, also from Qt,
• some small stickers from exhibitor companies 

But not that much I'd expect. There was a raffle where you could win a cool drone (sponsored by one of the exhibitors), but this time I was kinda outta luck.

7. Summing up

You tell me - do you think it was a good conference? The main lecture hall was very impressive in any case. An you could see the Berlin TV-tower right out of the window.

---
* early supporter! 🙋

** see this book: https://www.packtpub.com/application-development/hands-high-performance-programming-qt-5

*** i.e. to the Qt World Summit conference. BTW, here are all QtWS-2019 videos to watch: https://resources.qt.io/qt-world-summit-2019