Qt Event Loop, Networking and I/O API

Qt event loop, networking and I/O API

Thiago Macieira, Qt Core Maintainer San Francisco, November 2013 Who am I?

• Open Source developer for 15 years • C++ developer for 13 years • Software Architect at Intel’s Open Source Technology Center (OTC) • Maintainer of two modules in the Qt Project ‒ QtCore and QtDBus • MBA and double degree in Engineering • Previously, led the “Qt Open Governance” project

• The event loop

• Event loops and threads

• Networking and I/O

• QAbstractEventDispatcher • QEventLoop • QCoreApplication • QTimer & QBasicTimer • QSocketNotifier & QWinEventNotifer • QThread

• While !interrupted: • If there are new events, dispatch them • Wait for more events

• Event loops are the inner core of modern applications ‒ Headless servers and daemons, GUI applications, etc. ‒ Everything but shell tools and file-processing tools

• Receives and translates GUI • Integrates with: events ‒ Generic Unix select(2) ‒ NSEvents (on Mac) ‒ Glib ‒ MSG (on Windows) ‒ Mac’s CFRunLoop ‒ BPS events (on Blackberry) ‒ Windows’s WaitForMultipleObjects ‒ X11 events (using XCB) • Services timers ‒ Wayland events ‒ With coarse timer support ‒ Many others • Polls sockets • Manages the event queue (priority queue) • Polls Windows events • Allows for event filtering • Thread-safe waking up

• Each thread has one instance • Only of interest if you’re porting to a new OS...

virtual bool processEvents(QEventLoop::ProcessEventsFlags flags) = 0; virtual bool hasPendingEvents() = 0;

virtual void registerSocketNotifier(QSocketNotifier *notifier) = 0; virtual void unregisterSocketNotifier(QSocketNotifier *notifier) = 0;

virtual void registerTimer(int timerId, int interval, Qt::TimerType timerType, QObject *object) = 0; virtual bool unregisterTimer(int timerId) = 0;

#ifdef Q_OS_WIN virtual bool registerEventNotifier(QWinEventNotifier *notifier) = 0; virtual void unregisterEventNotifier(QWinEventNotifier *notifier) = 0; #endif

virtual void wakeUp() = 0; virtual void interrupt() = 0;

• Millisecond-based timing • One signal: timeout() • Convenience function for single-shot timers • Three levels of coarseness: ‒ Precise: accurate to the millisecond ‒ Coarse (default): up to 5% error introduced so timers can be coalesced ‒ VeryCoarse: rounded to the nearest second

QTimer timer; timer.setInterval(2000); QObject::connect(&timer, &QTimer::timeout, []() { qDebug() << "Timed out"; qApp->exit(1); });

• Polling for read, write or exceptional activity on sockets • On Unix, can poll anything that has a file descriptor • One signal: activated(int)

PipeReader::PipeReader(int fd, QObject *parent) : QObject(parent), notifier(nullptr), m_fd(fd) { notifier = new QSocketNotifier(m_fd, QSocketNotifier::Read, this); connect(notifier, &QSocketNotifier::activated, this, &PipeReader::readFromFd); }

Atomic variable

while (!interrupted.load()) { struct timeval tm = maximumWaitTime(); fd_set readFds = enabledReadFds; fd_set writeFds = enabledWriteFds; fd_set exceptFds = enabledExceptFds;

emit aboutToBlock(); ::select(maxFd + 1, &readFds, &writeFds, &exceptFds, &tm); emit awake();

sendPostedEvents(); dispatchFds(&readFds, &writeFds, &exceptFds); dispatchExpiredTimers(); }

• Windows is different... • Similar to QSocketNotifier • One signal: activated(HANDLE)

• Objects receive signals and event • Starting with Qt 5, signals can be connected to lambdas, non- member functions, functors and some types of member functions

public slots: void PipeReader::closeChannel() void start(); { ::close(m_fd); private slots: emit channelClosed(); void closeChannel(); } void readFromFd();

• One instance of QCoreApplication per application ‒ QGuiApplication if you want to use QWindow ‒ QApplication if you want to use QWidget • Defines the “GUI thread” and connects to the UI server • Starts the main event loop

int main(int argc, char **argv) { QGuiApplication app(argc, argv);

AnalogClockWindow clock; clock.show();

app.exec(); }

class PipeReader : public QObject { Q_OBJECT public: PipeReader(int fd, QObject *parent = 0);

signals: void dataReceived(const QByteArray &data); void channelClosed(); void timeout();

private slots: void closeChannel(); void readFromFd(); };

int main(int argc, char *argv[]) { QCoreApplication a(argc, argv); PipeReader reader(fileno(stdin)); QObject::connect(&reader, &PipeReader::channelClosed, &QCoreApplication::quit); QObject::connect(&reader, &PipeReader::dataReceived, [](const QByteArray &data) { qDebug() << "Received" << data.length() << "bytes"; });

QTimer timer; timer.setInterval(2000); QObject::connect(&timer, &QTimer::timeout, []() { qDebug() << "Timed out"; qApp->exit(1); });

timer.start(); return a.exec(); }

• Make non-blocking operations into “blocking” ones • Without freezing the UI

• Avoid if you can!

QByteArray nestedLoop() { QEventLoop loop; QByteArray data; PipeReader reader(fileno(stdin)); QObject::connect(&reader, &PipeReader::dataReceived, [&](const QByteArray &newData) { data += newData; }); QObject::connect(&reader, &PipeReader::channelClosed, &loop, &QEventLoop::quit);

loop.exec(); return data; }

• It appears in modal GUI classes ‒ QDialog ‒ QProgressDialog ‒ QFileDialog

‒ QMenu • Like QEventLoop’s exec(), avoid if you can! ‒ Use show() and return to the main loop

int progressDialog() { QProgressDialog dialog("Bytes received", "Stop reading", 0, 100); // expect 100 bytes PipeReader reader(fileno(stdin)); QObject::connect(&reader, &PipeReader::channelClosed, &dialog, &QProgressDialog::accept); QObject::connect(&reader, &PipeReader::dataReceived, [&](const QByteArray &data) { qDebug() << "Received" << data.length() << "bytes"; dialog.setValue(qMin(dialog.maximum(), dialog.value() + data.length())); });

dialog.exec(); return dialog.result() == QDialog::Accepted ? 0 : 1; }

• Manages a new thread in the application ‒ Starting, waiting, requesting or forcing exit, notifying of exit, etc. • Also provides some methods for the current thread ‒ Sleeping, yielding

Good reasons Bad reasons • Calling blocking functions • To use sleep() • CPU-intensive work • Networking and I/O • Real-time work ‒ Except for scalability • Scalability

class MyThread : public QThread { public: void run() { // code that takes a long time to run goes here } };

void run() { forever { QByteArray buffer(4096, Qt::Uninitialized); ssize_t r = ::read(m_fd, buffer.data(), buffer.size()); if (r <= 0) return; buffer.resize(r); emit dataReceived(buffer); } }

• Each QObject is associated with one thread • Where its event handlers and slots will be called • Where the object must be deleted

ThreadedPipeReader2::ThreadedPipeReader2(int fd, QObject *parent) : QThread(parent), reader(new PipeReader(fd)) { reader->moveToThread(this); connect(reader, &PipeReader::dataReceived, this, &ThreadedPipeReader2::dataReceived, Qt::DirectConnection); connect(reader, &PipeReader::channelClosed, [=]() { delete reader; quit(); }); }

Automatically moves children

• DirectConnection: ‒ Slot is called immediately, by simple function call ‒ Synchronous, same thread • QueuedConnection: ‒ An event is posted so the slot is called later ‒ Asynchronous, potentially run in another thread • BlockingQueuedConnection: ‒ Same as QueuedConnection, but includes a semaphore to wait ‒ Synchronous, always run in another thread • AutoConnection: choose at emission time

void ThreadedPipeReader2::run() { // preparation goes here exec(); // clean-up goes here }

Random access Sequential access • QFile • QNetworkReply ‒ QTemporaryFile • QProcess ‒ QSaveFile • QLocalSocket • QBuffer • QTcpSocket • QUdpSocket • QWebSocket (coming)

• Base class of all Qt I/O classes • Provides: ‒ CRLF translation ‒ read(), readAll(), readLine(), write(), peek() ‒ Signals: readyRead(), bytesWritten() ‒ Buffering support, bytesAvailable(), bytesToWrite() ‒ atEnd(), size(), pos(), seek()

• Defining feature: seek() and size() • Device can be put in unbuffered mode • All I/O is synchronous ‒ I/O latency is not considered as blocking • No notification support (signals not emitted)

Can open a file name, a FILE* or a file descriptor

QFile f; f.open(stdin, QIODevice::ReadOnly | QIODevice::Text); while (!f.atEnd()) { QByteArray line = f.readLine().trimmed(); qDebug() << "Line is" << line.length() << "bytes long"; }

• Defining feature: you must read sequentially • I/O functions are non-blocking (asynchronous) • All operations are buffered ‒ Unlimited size by default ‒ Actual I/O happens in the event loop • Signals notify of incoming or outgoing activity

• Exception to the non-blocking rule • Provided for synchronous I/O • Operate on the buffers • Matched 1:1 with an I/O signal ‒ readyRead → waitForReadyRead ‒ bytesWritten → waitForBytesWritten ‒ connected / started → waitForConnected / waitForStarted ‒ disconnected / finished → waitForDisconnected / waitForFinished

• Remember: read() and write() operate on buffers • Must call waitForReadyRead() and/or waitForBytesWritten() • Both functions execute both input and output ‒ No buffer deadlock

• read() when readyRead() is emitted • write() when necessary • Event loop takes care of the rest! • To limit buffer size: ‒ Input buffer: setReadBufferSize() ‒ Output buffer: manual control using bytesToWrite()

• Networking in the GUI thread? Always asynchronous • Multiple I/O in the same thread? Asynchronous • Short child process? Synchronous • Writing a blocking function? Synchronous

AsyncProcess() { auto proc = new QProcess; connect(proc, (void (QProcess::*)(int))&QProcess::finished, [=]() { proc->readLine(); // skip first line proc->deleteLater(); emit dataReceived(proc->readLine().trimmed()); }); proc->start("qmake", QStringList() << "-v"); }

QByteArray syncProcess() { QProcess proc; proc.start("qmake", QStringList() << "-v"); proc.waitForFinished();

proc.readLine(); // skip first line return proc.readLine().trimmed(); }

QByteArray http09Downloader() { QTcpSocket socket; socket.connectToHost("qt-project.org", 80); socket.write("GET /\r\n"); socket.waitForDisconnected(); return socket.readAll(); }

QByteArray gopher() { QTcpSocket socket; socket.connectToHost("gopher.floodgap.com", 70); socket.write("/feeds/latest\r\n"); socket.waitForDisconnected(); return socket.readAll(); }

Thiago Macieira [email protected] google.com/+ThiagoMacieira

Links: Website: http://qt-project.org Mailing lists: http://lists.qt-project.org IRC: #qt and #qt-labs on Freenode