Discussion:
Another integer typedef OR how to prepare for 64-bit in Qt 5
(too old to reply)
Thiago Macieira
2018-11-02 03:42:52 UTC
Permalink
We have a lot of API that, for Qt 6, we've already decided to extend to 64-bit
on 64-bit platforms, but keep as decently-sized 32-bit on 32-bit ones. We've
already created the type we want for Qt 6 and that's "qsizetype": a signed
integer the same size as size_t and ptrdiff_t. And a lot of new API already
uses it, for example:
https://doc.qt.io/qt-5/qstringview.html#at
http://doc-snapshots.qt.io/qt5-dev/qcborarray.html#at

And internally, we can use decltype(), auto or one of the typedefs to make
optimal use. Example:
struct ByteData
{
QByteArray::size_type len;

But how do we begin porting our API? For example:
https://doc.qt.io/qt-5/qstring.html#at
https://doc.qt.io/qt-5/qbytearray.html#at

These two functions are declared as taking int parameters. Similarly,
https://doc.qt.io/qt-5/qstring.html#indexOf
returns an int index.

This porting will take a lot of time, since the use of "int" is so pervasive
in our code. And we can't change them to qsizetype right now, it would be a BC
break.

What do we do?

Option 1: do nothing, wait for Qt 6 and do the change then
This is the simplest solution for now, but hardest in the future. It would
mean finding all the issues in a very short amount of time and it would make
merges across 5.15 and 6.0 very difficult.

Option 2: insert #if in our API, starting now
This works and is the same amount of work as almost all of the rest. But it's
VERY ugly.

Option 3: use #if per class, starting now
For example, change QString to have:
#if QT_VERSION >= QT_VERSION_CHECK(6,0,0)
typedef qsizetype size_type;
#else
typedef int size_type;
#endif
And then change all uses in QString from "int" to size_type in the API.

This is the cleanest and follows the precedence of the Standard Library. But
as a side-effect, we may have very long names in our code and they'll show up
in the documentation too. This would also require adding such typedefs to
classes that currently don't have them, like QPixmap and QImage.

Option 4: create a central #if and use this new type, starting now
Same as #3, but instead of per class, it would be central. Moreover, with one
preprocessor trick (-Dnewinttype=int), we can fool qdoc into generating
documentation as it is today. In Qt 6, once we decide we don't need Qt 5
merging anymore, we can also do a global search-and-replace to qsizetype
(optional).

Other options?

I think Option 4 is the best solution, among those that I listed.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Thiago Macieira
2018-11-02 04:44:42 UTC
Permalink
On Thursday, 1 November 2018 20:42:52 PDT Thiago Macieira wrote:
> What do we do?

Example: https://codereview.qt-project.org/244445

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Иван Комиссаров
2018-11-02 07:06:12 UTC
Permalink
I vote for 3. The (small) argument is: with qint one cannot write an algorithm taking std:: class (std::vector<uchar>) and a Qt class but with size_type it can.

Something like
template<typename T> T::size_type * getSize(const T &t) { return t.size(); }
auto d1 = getSize(std::vector<uchar>()); // ok
auto d2 = getSize(QString()); // ok

Seeing size_type in doc is OK - the users will be aware that it may vary depending on Qt ver.

> 2 нояб. 2018 г., в 5:44, Thiago Macieira <***@intel.com> написал(а):
>
> On Thursday, 1 November 2018 20:42:52 PDT Thiago Macieira wrote:
>> What do we do?
>
> Example: https://codereview.qt-project.org/244445
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
>
>
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
Thiago Macieira
2018-11-02 14:56:26 UTC
Permalink
On Friday, 2 November 2018 00:06:12 PDT Иван Комиссаров wrote:
> I vote for 3. The (small) argument is: with qint one cannot write an
> algorithm taking std:: class (std::vector<uchar>) and a Qt class but with
> size_type it can.
>
> Something like
> template<typename T> T::size_type * getSize(const T &t) { return t.size(); }
> auto d1 = getSize(std::vector<uchar>()); // ok
> auto d2 = getSize(QString()); // ok
>
> Seeing size_type in doc is OK - the users will be aware that it may vary
> depending on Qt ver.

size_type would be defined for the Qt types anyway, so your argument does not
exclude #4.

What do you suggest we do for QImage?

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
André Pönitz
2018-11-02 08:02:37 UTC
Permalink
On Thu, Nov 01, 2018 at 08:42:52PM -0700, Thiago Macieira wrote:
> What do we do?
>
> Option 1: do nothing, wait for Qt 6 and do the change then
> Option 2: insert #if in our API, starting now
> Option 3: use #if per class, starting now
> Option 4: create a central #if and use this new type, starting now

Option 5: Leave as-is, also in Qt 6.

Known and bearable effort to handle the recurring 'why do you use
int?' threads.

No effort for existing users, which otherwise will face lots
of size mismatch warnings in their code.

Andre'
Lars Knoll
2018-11-02 08:52:07 UTC
Permalink
> On 2 Nov 2018, at 09:02, André Pönitz <***@t-online.de> wrote:
>
> On Thu, Nov 01, 2018 at 08:42:52PM -0700, Thiago Macieira wrote:
>> What do we do?
>>
>> Option 1: do nothing, wait for Qt 6 and do the change then
>> Option 2: insert #if in our API, starting now
>> Option 3: use #if per class, starting now
>> Option 4: create a central #if and use this new type, starting now
>
> Option 5: Leave as-is, also in Qt 6.
>
> Known and bearable effort to handle the recurring 'why do you use
> int?' threads.
>
> No effort for existing users, which otherwise will face lots
> of size mismatch warnings in their code.

And the drawback, that people can’t use large data sets with our containers.

The size mismatch warnings users will get in Qt 6 are IMO not a huge issue. The narrowing to a smaller size does not break compatibility, as long as no container overflows in size (which it couldn’t with code based on Qt 5).


So my vote actually goes for option 4. We introduce a central type that’s int in Qt 5, qsizetype in Qt 6, and start using it. That allows us to introduce it then class by class and fix any issues we find around it.

Cheers,
Lars
André Pönitz
2018-11-02 09:26:07 UTC
Permalink
On Fri, Nov 02, 2018 at 08:52:07AM +0000, Lars Knoll wrote:
> > On 2 Nov 2018, at 09:02, André Pönitz <***@t-online.de> wrote:
> >
> > On Thu, Nov 01, 2018 at 08:42:52PM -0700, Thiago Macieira wrote:
> >> What do we do?
> >>
> >> Option 1: do nothing, wait for Qt 6 and do the change then
> >> Option 2: insert #if in our API, starting now
> >> Option 3: use #if per class, starting now
> >> Option 4: create a central #if and use this new type, starting now
> >
> > Option 5: Leave as-is, also in Qt 6.
> >
> > Known and bearable effort to handle the recurring 'why do you use
> > int?' threads.
> >
> > No effort for existing users, which otherwise will face lots
> > of size mismatch warnings in their code.
>
> And the drawback, that people can’t use large data sets with our containers.

For large data sets the Qt containers usually aren't a good choice
anyway, as the trade-off of the implicit sharing is often not worth
it in that case.

> The size mismatch warnings users will get in Qt 6 are IMO not a huge
> issue. The narrowing to a smaller size does not break compatibility,
> as long as no container overflows in size (which it couldn’t with
> code based on Qt 5).
>
>
> So my vote actually goes for option 4. We introduce a central type
> that’s int in Qt 5, qsizetype in Qt 6, and start using it. That allows
> us to introduce it then class by class and fix any issues we find
> around it.

*shrug* I won't fight such a solution, but reserve the right to say
"told you so" every now and then. 1/2 ;-)

Andre'
Edward Welbourne
2018-11-02 14:36:26 UTC
Permalink
On Thu, Nov 01, 2018 at 08:42:52PM -0700, Thiago Macieira wrote:
>>>> What do we do?
>>>>
>>>> Option 1: do nothing, wait for Qt 6 and do the change then
>>>> Option 2: insert #if in our API, starting now
>>>> Option 3: use #if per class, starting now
>>>> Option 4: create a central #if and use this new type, starting now

On 2 Nov 2018, at 09:02, André Pönitz <***@t-online.de> wrote:
>>> Option 5: Leave as-is, also in Qt 6.
>>>
>>> Known and bearable effort to handle the recurring 'why do you use
>>> int?' threads.
>>>
>>> No effort for existing users, which otherwise will face lots
>>> of size mismatch warnings in their code.

On Fri, Nov 02, 2018 at 08:52:07AM +0000, Lars Knoll wrote:
>> And the drawback, that people can’t use large data sets with our containers.

André Pönitz (2 November 2018 10:26)
> For large data sets the Qt containers usually aren't a good choice
> anyway, as the trade-off of the implicit sharing is often not worth
> it in that case.

On the other hand, not all users shall care: time to develop and ship
the product may be more important to them than optimality of the code.
In any case, datasets get bigger over time, and what was once though of
as prohibitively large eventually becomes normal-sized.

Twenty years ago, I was implementing a virtual file-system in a GIS to
let us deal with files > 2GB on systems where fseek() and ftell()
couldn't handle that (because long was 32-bit) with real files. Those
files held datasets that represented a mapping agency's knowledge of the
world, so were considered huge at the time. Nowadays, similarly huge
files are increasingly commonplace; and lots of code glibly assumes it
can just open a file, grab all its contents into a buffer, work on it
and emit the results to wherever it's going, without worrying about the
file being "too big". If the system's got the memory to allocate a 5 GB
QByteArray without problems, there's going to be folks who do just that
and who aren't going to be amused if our APIs (like fseek and ftell back
in the '80s, before fpos_t was invented; or well into the '90s when O/S
vendors were still slow to recognise that fpos_t needed to be 64-bit so
that f[gs]etpos can take over) break that easy-to-code (though
resource-squandering) pattern. Which, at present, they do.

So I'm definitely in favour of ending the use of int in Qt's APIs, at
least for offsets into (potentially huge) datasets; and Thiago's option
4 looks pretty reasonable to me.

Eddy.
André Pönitz
2018-11-02 17:28:00 UTC
Permalink
On Fri, Nov 02, 2018 at 02:36:26PM +0000, Edward Welbourne wrote:
> On Thu, Nov 01, 2018 at 08:42:52PM -0700, Thiago Macieira wrote:
> >>>> What do we do?
> >>>>
> >>>> Option 1: do nothing, wait for Qt 6 and do the change then
> >>>> Option 2: insert #if in our API, starting now
> >>>> Option 3: use #if per class, starting now
> >>>> Option 4: create a central #if and use this new type, starting now
>
> On 2 Nov 2018, at 09:02, André Pönitz <***@t-online.de> wrote:
> >>> Option 5: Leave as-is, also in Qt 6.
> >>>
> >>> Known and bearable effort to handle the recurring 'why do you use
> >>> int?' threads.
> >>>
> >>> No effort for existing users, which otherwise will face lots
> >>> of size mismatch warnings in their code.
>
> On Fri, Nov 02, 2018 at 08:52:07AM +0000, Lars Knoll wrote:
> >> And the drawback, that people can’t use large data sets with our containers.
>
> André Pönitz (2 November 2018 10:26)
> > For large data sets the Qt containers usually aren't a good choice
> > anyway, as the trade-off of the implicit sharing is often not worth
> > it in that case.
>
> On the other hand, not all users shall care: time to develop and ship
> the product may be more important to them than optimality of the code.

Correct. But in this context most of the QList-should-be-replaced-by
-something-else arguments are void anyway.

Also, the "don't touch" option is exactly that: People should not have to
worry about their existing code or potential regressions.

> In any case, datasets get bigger over time, and what was once though of
> as prohibitively large eventually becomes normal-sized.

Also correct. And sometimes this needs a different container or a new
algorithm. But then this happens in response to an actual need and
the resulting change can be expected to be tested reasonably. That's
different from an s///g shotgun that can hit "accidentally working" areas.

> Twenty years ago, I was implementing a virtual file-system in a GIS to
> let us deal with files > 2GB on systems where fseek() and ftell()
> couldn't handle that (because long was 32-bit) with real files. Those
> files held datasets that represented a mapping agency's knowledge of the
> world, so were considered huge at the time. Nowadays, similarly huge
> files are increasingly commonplace; and lots of code glibly assumes it
> can just open a file, grab all its contents into a buffer, work on it
> and emit the results to wherever it's going, without worrying about the
> file being "too big". If the system's got the memory to allocate a 5 GB
> QByteArray without problems, there's going to be folks who do just that
> and who aren't going to be amused if our APIs (like fseek and ftell back
> in the '80s, before fpos_t was invented; or well into the '90s when O/S
> vendors were still slow to recognise that fpos_t needed to be 64-bit so
> that f[gs]etpos can take over) break that easy-to-code (though
> resource-squandering) pattern. Which, at present, they do.

People will even expect to put a QByteArray into a QDataStream on one
machine and get it back on a second. Even if the first was some 32bit
embedded device and the second their 64bit desktop. Or the other way
round. People's expectations are sometimes strange and cannot always
be met.

> So I'm definitely in favour of ending the use of int in Qt's APIs, at
> least for offsets into (potentially huge) datasets; and Thiago's option
> 4 looks pretty reasonable to me.

I am aware of that.

Andre'
Thiago Macieira
2018-12-02 23:01:54 UTC
Permalink
On Sunday, 2 December 2018 13:29:35 PST Giuseppe D'Angelo via Development
wrote:
> Has anyone actually tried changing size_type to a 64bit datatype, and
> checking how many warnings start appearing in user code? (E.g. just try
> with QString, QList, QVector; and build Creator, or some KDE libs). I've
> got the gut feeling that it will be an absolutely gigantic number,
> especially on Windows...

Not yet. Yes, it will be a huge number.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Philippe
2018-12-03 10:37:31 UTC
Permalink
>> I've got the gut feeling that it will be an absolutely gigantic number,

Same feeling here ++

>> 1) is it worth it?

No, IMHO. For those rare cases when huge indexes / sizes are required,
the std containers can be used.

and BTW, I don't use QDataStream with Qt containers, must maybe there
would be an issue between Qt 5 and 6 data binaries (?).

Philippe

On Mon, 3 Dec 2018 11:02:31 +0100
Giuseppe D'Angelo via Development <***@lists.qt-project.org> wrote:

> Il 03/12/18 00:01, Thiago Macieira ha scritto:
> >> Has anyone actually tried changing size_type to a 64bit datatype, and
> >> checking how many warnings start appearing in user code? (E.g. just try
> >> with QString, QList, QVector; and build Creator, or some KDE libs). I've
> >> got the gut feeling that it will be an absolutely gigantic number,
> >> especially on Windows...
> > Not yet. Yes, it will be a huge number.
>
> Which compels the questions:
>
> 1) is it worth it?
>
> 2) should we offer users some way (configure time switch, template parameter, ...) to keep int-based indexes and avoid all these warnings while they fix their code to use a ssize_t-like index?
>
> Cheers,
> -- Giuseppe D'Angelo | ***@kdab.com | Senior Software Engineer
> KDAB (France) S.A.S., a KDAB Group company
> Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
> KDAB - The Qt, C++ and OpenGL Experts
>
Thiago Macieira
2018-12-03 16:27:46 UTC
Permalink
On Monday, 3 December 2018 02:02:31 PST Giuseppe D'Angelo via Development
wrote:
> Il 03/12/18 00:01, Thiago Macieira ha scritto:
> >> Has anyone actually tried changing size_type to a 64bit datatype, and
> >> checking how many warnings start appearing in user code? (E.g. just try
> >> with QString, QList, QVector; and build Creator, or some KDE libs). I've
> >> got the gut feeling that it will be an absolutely gigantic number,
> >> especially on Windows...
> >
> > Not yet. Yes, it will be a huge number.
>
> Which compels the questions:
>
> 1) is it worth it?

I think so, but we can document that going above 2 GB is experimental and not
well tested, since it will be difficult for us to test. But getting the
containers and QString and QByteArray done should be doable by the 6.0
release.

> 2) should we offer users some way (configure time switch, template
> parameter, ...) to keep int-based indexes and avoid all these warnings
> while they fix their code to use a ssize_t-like index?

No. There will be no warnings. We'll fix the all before the release.

The question is only if we've identified all places that need to be fixed,
those that don't produce a warning.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Edward Welbourne
2018-12-03 17:08:27 UTC
Permalink
On Monday, 3 December 2018 02:02:31 PST Giuseppe D'Angelo via Development wrote:
>> 2) should we offer users some way (configure time switch, template
>> parameter, ...) to keep int-based indexes and avoid all these
>> warnings while they fix their code to use a ssize_t-like index?

Thiago Macieira (3 December 2018 17:27)
> No. There will be no warnings. We'll fix the all before the release.
>
> The question is only if we've identified all places that need to be
> fixed, those that don't produce a warning.

That covers all the warnings in the Qt code-base.
I think Giuseppe had client code in mind; we can't fix that.
Shall your changes leave client code getting type conflicts ?
I guess it shall, in so far as they're using int by default.
That has, after all, been The Qt Way for decades ...

Eddy.
Thiago Macieira
2018-12-04 02:38:24 UTC
Permalink
On Monday, 3 December 2018 09:43:00 PST Giuseppe D'Angelo via Development
wrote:
> Yes, indeed. The number of warnings raised in user projects makes me a
> bit uncomfortable with the plan, that's why I was asking if someone had
> already done a test.

If you really want to do this, you can always patch qglobal.h where qsizetype
is defined. Since this is a binary-incompatible build anyway, you'll have to
always rebuild from source, so patching is not a huge step to require.

Whether we support that or not is a different story. A lot of code will assume
that sizeof(qsizetype) == sizeof(quintptr) and that might break if you patch
Qt.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
André Pönitz
2018-12-03 19:43:17 UTC
Permalink
On Mon, Dec 03, 2018 at 11:02:31AM +0100, Giuseppe D'Angelo via Development wrote:
> Il 03/12/18 00:01, Thiago Macieira ha scritto:
> > > Has anyone actually tried changing size_type to a 64bit datatype, and
> > > checking how many warnings start appearing in user code? (E.g. just try
> > > with QString, QList, QVector; and build Creator, or some KDE libs). I've
> > > got the gut feeling that it will be an absolutely gigantic number,
> > > especially on Windows...
> > Not yet. Yes, it will be a huge number.
>
> Which compels the questions:
>
> 1) is it worth it?

No.

It's really rare to need a container of that size in a Qt context,
and if one does, there's always std:: containers - which are likely
to be used at sizes much smaller than INT_MAX anyways.

> 2) should we offer users some way (configure time switch, template
> parameter, ...) to keep int-based indexes and avoid all these warnings while
> they fix their code to use a ssize_t-like index?

Not needed when 1) is resolved as 'No.'

Andre'
Bernhard Lindner
2018-12-03 23:33:06 UTC
Permalink
Hi!

> > > Not yet. Yes, it will be a huge number.
> >
> > Which compels the questions:
> >
> > 1) is it worth it?
>
> No.

What about those people who want (or must) implement clean interfaces in their code using
standard size_t?

Even if the number of objects/sizes/counts is usually below 2^31, it still is a pain to
either suffer from numerous static analysis warnings (when implicitly converting size_t
and int) or from implementing numerous casts.

In my book it would be a BIG improvement if the Qt interfaces would cleanly integrate into
compliant environments without nasty workarounds.

Qt's int-loaded API is probably not the most burning pain when using Qt but definitely the
most persisting one (for me).

--
Best Regards,
Bernhard
Thiago Macieira
2018-12-04 06:34:17 UTC
Permalink
On Monday, 3 December 2018 15:33:06 PST Bernhard Lindner wrote:
> What about those people who want (or must) implement clean interfaces in
> their code using standard size_t?

You're inverting the argument we've had for using int. It's not that it's
always sufficient, but it's sufficient for a sufficient number of use-cases,
thus avoiding unnecessary casts *and* avoiding mistakes in conversion.

The literal "1" is int, not size_t or long, so it's far easier to write int
code than size_t.

In any case, Qt will not use an unsigned type.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Bernhard Lindner
2018-12-04 16:35:58 UTC
Permalink
Hi!

> You're inverting the argument we've had for using int.

Yes, because I have other priorities.

> It's not that it's
> always sufficient, but it's sufficient for a sufficient number of use-cases,
> thus avoiding unnecessary casts *and* avoiding mistakes in conversion.

Avoiding cast? Creating casts! I rarly need to cast unsigned sizes to signed in a
compliant environment. But I need a LOT of casts whenever integrating Qt with compliant
code.

Whoever ignores standards is the one who creates interfacing effort. And Qt is
(unnecessarily) ignoring standards. If the world would consit of Qt only, then maybe you
would be right. But most of the world is not Qt.

Using int instead of size_t (or a compatible type) for me means: More unit tests, more
error checks and asserts, more documentation, more document reading, lots of casts when interfacing standard compliant code. All due to the negative value range.

> The literal "1" is int, not size_t or long, so i,'s far easier to write int
> code than size_t.

You think this is a strong argument?

> In any case, Qt will not use an unsigned type.

So qsizetype will be signed in Qt6?

What a pitty. I assumed the non-compliant int usage would be some legacy burden and hoped
it would be fixed at the earliest opportunity.

Anyway the usage of qsizetype would be an improvment.

--
Best Regards,
Bernhard
Christian Kandeler
2018-12-04 16:59:51 UTC
Permalink
On Tue, 4 Dec 2018 17:35:58 +0100
Bernhard Lindner <***@bernhard-lindner.de> wrote:

> > It's not that it's
> > always sufficient, but it's sufficient for a sufficient number of use-cases,
> > thus avoiding unnecessary casts *and* avoiding mistakes in conversion.
>
> Avoiding cast? Creating casts! I rarly need to cast unsigned sizes to signed in a
> compliant environment. But I need a LOT of casts whenever integrating Qt with compliant
> code.

What is this "compliance" you speak about?


Christian
Thiago Macieira
2018-12-04 17:02:52 UTC
Permalink
On Tuesday, 4 December 2018 08:35:58 PST Bernhard Lindner wrote:
> > The literal "1" is int, not size_t or long, so i,'s far easier to write
> > int
> > code than size_t.
>
> You think this is a strong argument?

No, it's a weak argument, but an argument nonetheless. If you wrote
n + 1
and n is size_t, then you have a type mismatch. Whether a compiler warns about
that or not is unpredictable, so you will need to cast the 1, or at least make
it unsigned:
n + 1U

> > In any case, Qt will not use an unsigned type.
>
> So qsizetype will be signed in Qt6?

Yes, as it is in Qt 5. And as is the current recommendation of the C++
standard committee: use unsigned *only* where you need modulo-2 overflow
arithmetic, otherwise signed.

Also note how the following expression:
if (i - 1 < size())

Can have surprising results if i and size() are unsigned.

> What a pitty. I assumed the non-compliant int usage would be some legacy
> burden and hoped it would be fixed at the earliest opportunity.

Compliant to what? Nothing in the standard requires use of a given type or
another. If anything, we're going to be more compliant to the committee wishes
than the standard containers themselves.

> Anyway the usage of qsizetype would be an improvment.

Indeed, that's why I am pushing for it.

Note also how the "n + 1" code with n being a qsizetype would not warn.
--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Marco Bubke
2018-12-04 17:37:23 UTC
Permalink
To quote the standard committee meeting:

"There was an evening discussion on the signedness of std::span’s size method return type. The result: we’ll change std::span’s size return type from signed to unsigned<https://wg21.link/P1089>, and add a std::ssize free function that can be used to get the size of a container as a signed value<https://wg21.link/P1227>."


________________________________
From: Development <development-***@lists.qt-project.org> on behalf of Thiago Macieira <***@intel.com>
Sent: Tuesday, December 4, 2018 6:02:52 PM
To: ***@qt-project.org
Subject: Re: [Development] Another integer typedef OR how to prepare for 64-bit in Qt 5

On Tuesday, 4 December 2018 08:35:58 PST Bernhard Lindner wrote:
> > The literal "1" is int, not size_t or long, so i,'s far easier to write
> > int
> > code than size_t.
>
> You think this is a strong argument?

No, it's a weak argument, but an argument nonetheless. If you wrote
n + 1
and n is size_t, then you have a type mismatch. Whether a compiler warns about
that or not is unpredictable, so you will need to cast the 1, or at least make
it unsigned:
n + 1U

> > In any case, Qt will not use an unsigned type.
>
> So qsizetype will be signed in Qt6?

Yes, as it is in Qt 5. And as is the current recommendation of the C++
standard committee: use unsigned *only* where you need modulo-2 overflow
arithmetic, otherwise signed.

Also note how the following expression:
if (i - 1 < size())

Can have surprising results if i and size() are unsigned.

> What a pitty. I assumed the non-compliant int usage would be some legacy
> burden and hoped it would be fixed at the earliest opportunity.

Compliant to what? Nothing in the standard requires use of a given type or
another. If anything, we're going to be more compliant to the committee wishes
than the standard containers themselves.

> Anyway the usage of qsizetype would be an improvment.

Indeed, that's why I am pushing for it.

Note also how the "n + 1" code with n being a qsizetype would not warn.
--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Ville Voutilainen
2018-12-04 18:17:12 UTC
Permalink
On Tue, 4 Dec 2018 at 20:11, Marco Bubke <***@qt.io> wrote:
>
> To quote the standard committee meeting:
>
> "There was an evening discussion on the signedness of std::span’s size method return type. The result: we’ll change std::span’s size return type from signed to unsigned, and add a std::ssize free function that can be used to get the size of a container as a signed value."

Minor correction: that's not from an evening session, the committee is
moving to a direction where decisions are not made in evening
sessions.
Such a decision was, however, made in a daytime session with a large
group consisting of much more than just LEWG.

The relevant part is this:

Option 4: add std::ssize() alongside std::size(), change span::size()
to unsigned, no additional members. (P1227, reduced)
7/18/12/5/6

I did clarify whether this option had consensus, because if no option
would've, we would've stayed with the status quo:

VV: Does option 4 have consensus?
TW: 25 to 11--yes.
Bernhard Lindner
2018-12-04 18:43:31 UTC
Permalink
Well, those are interesting information. Thank you!
Frankly spoken I do not follow the committee's work.

Then let me rephrase my opionen: What every you plan to change regarding size types in
Qt6... do it the standard (C++20?) way.

--
Best Regards,
Bernhard
Thiago Macieira
2018-12-04 19:42:08 UTC
Permalink
On Tuesday, 4 December 2018 10:43:31 PST Bernhard Lindner wrote:
> Then let me rephrase my opionen: What every you plan to change regarding
> size types in Qt6... do it the standard (C++20?) way.

We're doing it the way the committee says it should have done in the first
place, not the way it has done: signed.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Ville Voutilainen
2018-12-04 20:02:05 UTC
Permalink
On Tue, 4 Dec 2018 at 21:55, Thiago Macieira <***@intel.com> wrote:
>
> On Tuesday, 4 December 2018 10:43:31 PST Bernhard Lindner wrote:
> > Then let me rephrase my opionen: What every you plan to change regarding
> > size types in Qt6... do it the standard (C++20?) way.
>
> We're doing it the way the committee says it should have done in the first
> place, not the way it has done: signed.

A *very* big part of why span's size is being changed to unsigned is
consistency, rather than the committee thinking
that unsigned is necessarily the right kind of type. There's some
ongoing work to alleviate the compatibility issues
between signed and unsigned types, but that work will most likely not
bear fruit in time for C++20.
Thiago Macieira
2018-12-04 22:06:41 UTC
Permalink
On Tuesday, 4 December 2018 12:02:05 PST Ville Voutilainen wrote:
> A *very* big part of why span's size is being changed to unsigned is
> consistency, rather than the committee thinking
> that unsigned is necessarily the right kind of type. There's some
> ongoing work to alleviate the compatibility issues
> between signed and unsigned types, but that work will most likely not
> bear fruit in time for C++20.

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0907r4.html

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Ville Voutilainen
2018-12-04 22:30:30 UTC
Permalink
On Wed, 5 Dec 2018 at 00:20, Thiago Macieira <***@intel.com> wrote:
>
> On Tuesday, 4 December 2018 12:02:05 PST Ville Voutilainen wrote:
> > A *very* big part of why span's size is being changed to unsigned is
> > consistency, rather than the committee thinking
> > that unsigned is necessarily the right kind of type. There's some
> > ongoing work to alleviate the compatibility issues
> > between signed and unsigned types, but that work will most likely not
> > bear fruit in time for C++20.
>
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0907r4.html

Not even close, I'm talking about the future follow-up to
http://open-std.org/JTC1/SC22/WG21/docs/papers/2018/p0946r0.html
Bernhard Lindner
2018-12-04 20:29:32 UTC
Permalink
> > Then let me rephrase my opionen: What every you plan to change regarding
> > size types in Qt6... do it the standard (C++20?) way.
>
> We're doing it the way the committee says it should have done in the first
> place, not the way it has done: signed.

If this plays well with the language and standard library features I am ok with it. It
doesn't remove all the additional effort I mentioned but it will prevent the unnecessary
interfacing/casting. As soon as user code is be rebased to ssize/C++20 and qsizetype
everything is consistent which is the most important thing. And I suppose other libraries
will follow the coming standard soon.

BTW Are there plans/roadmaps regarding C++20 support?

--
Best Regards,
Bernhard
Thiago Macieira
2018-12-04 22:52:50 UTC
Permalink
On Tuesday, 4 December 2018 12:29:32 PST Bernhard Lindner wrote:
> > > Then let me rephrase my opionen: What every you plan to change regarding
> > > size types in Qt6... do it the standard (C++20?) way.
> >
> > We're doing it the way the committee says it should have done in the first
> > place, not the way it has done: signed.
>
> If this plays well with the language and standard library features I am ok
> with it. It doesn't remove all the additional effort I mentioned but it
> will prevent the unnecessary interfacing/casting. As soon as user code is
> be rebased to ssize/C++20 and qsizetype everything is consistent which is
> the most important thing. And I suppose other libraries will follow the
> coming standard soon.

The point is that you don't need size_t. And with the Qt API, you do need
negative indices, since we have things like
indexOf(T t, qsizetype from = 0)
where a negative index means to search from the end.

Restricting the discussion to modern, Von Neumann / modified Harvard systems
where sizeof(ptrdiff_t) == sizeof(void*), the only way you can have an in-
memory container with more than PTRDIFF_MAX items is if the size of those
items is 1. If it were 2, then PTRDIFF_MAX items would mean SIZE_MAX-1 memory
used, which is impossible. It also rules out any container with an overhead of
1 byte or more per element, such as linked lists.

You can't have a memory block that is larger than PTRDIFF_MAX bytes either,
otherwise end - begin < 0 and that's nonsense. That rules out
std::vector<char> too.

So you need a container of char/byte that has an overhead of less than one
byte per element, but does use more than one memory block. That's
std::deque<char>. How often do you use that?

So my conclusion is that if you need to operate on something that even comes
close to PTRDIFF_MAX elements, you need to look into an out-of-memory
representation (think database).

> BTW Are there plans/roadmaps regarding C++20 support?

No different than C++17. I already have some experimental code using <compare>
and the spaceship operator, which showed that the Clang/libc++ implementation
is incomplete. See qcborvalue.h.

Someone should add the "c++2a" config to qmake too.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Konstantin Tokarev
2018-12-05 13:36:36 UTC
Permalink
05.12.2018, 02:03, "Thiago Macieira" <***@intel.com>:
> On Tuesday, 4 December 2018 12:29:32 PST Bernhard Lindner wrote:
>>  > > Then let me rephrase my opionen: What every you plan to change regarding
>>  > > size types in Qt6... do it the standard (C++20?) way.
>>  >
>>  > We're doing it the way the committee says it should have done in the first
>>  > place, not the way it has done: signed.
>>
>>  If this plays well with the language and standard library features I am ok
>>  with it. It doesn't remove all the additional effort I mentioned but it
>>  will prevent the unnecessary interfacing/casting. As soon as user code is
>>  be rebased to ssize/C++20 and qsizetype everything is consistent which is
>>  the most important thing. And I suppose other libraries will follow the
>>  coming standard soon.
>
> The point is that you don't need size_t.

Unless you do need it, when interfacing with code which uses size_t. For example,
WebKit code base uses size_t as size type across the board, which causes lots of
conversions to signed in code which glues platform-independent code with Qt APIs.

> And with the Qt API, you do need
> negative indices, since we have things like
>         indexOf(T t, qsizetype from = 0)
> where a negative index means to search from the end.
>
> Restricting the discussion to modern, Von Neumann / modified Harvard systems
> where sizeof(ptrdiff_t) == sizeof(void*), the only way you can have an in-
> memory container with more than PTRDIFF_MAX items is if the size of those
> items is 1. If it were 2, then PTRDIFF_MAX items would mean SIZE_MAX-1 memory
> used, which is impossible. It also rules out any container with an overhead of
> 1 byte or more per element, such as linked lists.
>
> You can't have a memory block that is larger than PTRDIFF_MAX bytes either,
> otherwise end - begin < 0 and that's nonsense. That rules out
> std::vector<char> too.
>
> So you need a container of char/byte that has an overhead of less than one
> byte per element, but does use more than one memory block. That's
> std::deque<char>. How often do you use that?
>
> So my conclusion is that if you need to operate on something that even comes
> close to PTRDIFF_MAX elements, you need to look into an out-of-memory
> representation (think database).
>
>>  BTW Are there plans/roadmaps regarding C++20 support?
>
> No different than C++17. I already have some experimental code using <compare>
> and the spaceship operator, which showed that the Clang/libc++ implementation
> is incomplete. See qcborvalue.h.
>
> Someone should add the "c++2a" config to qmake too.
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
>   Software Architect - Intel Open Source Technology Center
>
> _______________________________________________
> Development mailing list
> ***@lists.qt-project.org
> https://lists.qt-project.org/listinfo/development

--
Regards,
Konstantin
Thiago Macieira
2018-12-05 16:20:07 UTC
Permalink
On Wednesday, 5 December 2018 05:36:36 PST Konstantin Tokarev wrote:
> > The point is that you don't need size_t.
>
> Unless you do need it, when interfacing with code which uses size_t. For
> example, WebKit code base uses size_t as size type across the board, which
> causes lots of conversions to signed in code which glues
> platform-independent code with Qt APIs.

Yes, you're right: size_t is needed when dealing with code that already used
size_t.

But for new API, we shouldn't use it. See Ville's email on another section of
this thread.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Marco Bubke
2018-12-06 10:57:12 UTC
Permalink
We could let return .size() size_t and .ssize() ssize_t. So you can choose. std::span is going in that direction and I find it very pragmatical.

________________________________
From: Development <development-***@lists.qt-project.org> on behalf of Thiago Macieira <***@intel.com>
Sent: Wednesday, December 5, 2018 5:20:07 PM
To: ***@qt-project.org
Subject: Re: [Development] Another integer typedef OR how to prepare for 64-bit in Qt 5

On Wednesday, 5 December 2018 05:36:36 PST Konstantin Tokarev wrote:
> > The point is that you don't need size_t.
>
> Unless you do need it, when interfacing with code which uses size_t. For
> example, WebKit code base uses size_t as size type across the board, which
> causes lots of conversions to signed in code which glues
> platform-independent code with Qt APIs.

Yes, you're right: size_t is needed when dealing with code that already used
size_t.

But for new API, we shouldn't use it. See Ville's email on another section of
this thread.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Thiago Macieira
2018-12-06 17:03:39 UTC
Permalink
On Thursday, 6 December 2018 02:57:12 PST Marco Bubke wrote:
> We could let return .size() size_t and .ssize() ssize_t. So you can choose.
> std::span is going in that direction and I find it very pragmatical.

Generic code should deal with it being signed or unsigned.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Jean-Michaël Celerier
2018-12-05 18:57:15 UTC
Permalink
On Tue, Dec 4, 2018 at 8:45 PM Thiago Macieira <***@intel.com>
wrote:

> On Tuesday, 4 December 2018 10:43:31 PST Bernhard Lindner wrote:
> > Then let me rephrase my opionen: What every you plan to change regarding
> > size types in Qt6... do it the standard (C++20?) way.
>
> We're doing it the way the committee says it should have done in the first
> place, not the way it has done: signed.
>
>
Thanks, underflowing operations on std::containers's size_t have been an
unending source of bugs in the projects I work on.
(thankfully nowadays clang can check for it with -fsanitize=undefined).



> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
>
>
>
> _______________________________________________
> Development mailing list
> ***@lists.qt-project.org
> https://lists.qt-project.org/listinfo/development
>
Ville Voutilainen
2018-12-05 19:24:25 UTC
Permalink
On Wed, 5 Dec 2018 at 21:11, Jean-Michaël Celerier
<***@gmail.com> wrote:
>
> On Tue, Dec 4, 2018 at 8:45 PM Thiago Macieira <***@intel.com> wrote:
>>
>> On Tuesday, 4 December 2018 10:43:31 PST Bernhard Lindner wrote:
>> > Then let me rephrase my opionen: What every you plan to change regarding
>> > size types in Qt6... do it the standard (C++20?) way.
>>
>> We're doing it the way the committee says it should have done in the first
>> place, not the way it has done: signed.
>>
>
> Thanks, underflowing operations on std::containers's size_t have been an unending source of bugs in the projects I work on.
> (thankfully nowadays clang can check for it with -fsanitize=undefined).

That is indeed one argument for a signed type, as opposed to an
unsigned type. For a signed {over,under}flow, a sanitizer can detect
your mistakes. For an unsigned {over,under}flow, it can't, since
there's nothing to sanitize, all behavior is well-defined, sane or
not.

For historical perspective, John Lakos makes this argument (and mostly
every other argument speaking in favor of a signed
type and against an unsigned type) in his very old book, "Large-Scale
C++ Software Design". To repeat, the reason
why the committee keeps insisting span::size() returning a size_type
(which is practically a size_t) is not because we think
it's great, we (well, in the most recent discussion, a consensus
majority of us) just think that span should not be given a
magical exception when everything else returns an unsigned type.

A significant part of the overall concern, at least as far as stdlib
vendors goes, is ABI. We do have the opportunity at Qt 5->6 to break
ABI.
We *are* going to break ABI just by the nature of changing what the
library is to link against, so we have the opportunity to do useful
ABI breaks.
In that sense, we have far less stringent compatibility
requirements/expectations/wishes/aspirations than a stdlib
implementation like
libstdc++ (or to historically less extent, but changing over time,
libc++, and same for msvclib) has.
Jean-Michaël Celerier
2018-12-05 19:37:17 UTC
Permalink
> For an unsigned {over,under}flow, it can't, since
there's nothing to sanitize, all behavior is well-defined, sane or
not.

-fsanitize=undefined also detects unsigned overflow even though the
behaviour is technically defined because it is such a large source of bugs
(I mean, they write the compiler; they can make it pop out butterflies in
A4 paper upon integer increment if they want).

See https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html



On Wed, Dec 5, 2018 at 8:24 PM Ville Voutilainen <
***@gmail.com> wrote:

> On Wed, 5 Dec 2018 at 21:11, Jean-Michaël Celerier
> <***@gmail.com> wrote:
> >
> > On Tue, Dec 4, 2018 at 8:45 PM Thiago Macieira <
> ***@intel.com> wrote:
> >>
> >> On Tuesday, 4 December 2018 10:43:31 PST Bernhard Lindner wrote:
> >> > Then let me rephrase my opionen: What every you plan to change
> regarding
> >> > size types in Qt6... do it the standard (C++20?) way.
> >>
> >> We're doing it the way the committee says it should have done in the
> first
> >> place, not the way it has done: signed.
> >>
> >
> > Thanks, underflowing operations on std::containers's size_t have been an
> unending source of bugs in the projects I work on.
> > (thankfully nowadays clang can check for it with -fsanitize=undefined).
>
> That is indeed one argument for a signed type, as opposed to an
> unsigned type. For a signed {over,under}flow, a sanitizer can detect
> your mistakes. For an unsigned {over,under}flow, it can't, since
> there's nothing to sanitize, all behavior is well-defined, sane or
> not.
>
> For historical perspective, John Lakos makes this argument (and mostly
> every other argument speaking in favor of a signed
> type and against an unsigned type) in his very old book, "Large-Scale
> C++ Software Design". To repeat, the reason
> why the committee keeps insisting span::size() returning a size_type
> (which is practically a size_t) is not because we think
> it's great, we (well, in the most recent discussion, a consensus
> majority of us) just think that span should not be given a
> magical exception when everything else returns an unsigned type.
>
> A significant part of the overall concern, at least as far as stdlib
> vendors goes, is ABI. We do have the opportunity at Qt 5->6 to break
> ABI.
> We *are* going to break ABI just by the nature of changing what the
> library is to link against, so we have the opportunity to do useful
> ABI breaks.
> In that sense, we have far less stringent compatibility
> requirements/expectations/wishes/aspirations than a stdlib
> implementation like
> libstdc++ (or to historically less extent, but changing over time,
> libc++, and same for msvclib) has.
>
Marco Bubke
2018-12-04 17:32:04 UTC
Permalink
One solution would be std::ssize(vectorWithSignedSize) == std::ssize(vectorWithUnsignedSize) where you always get ssize_t back. This is proposed for C++20. You can write your own version if you want.

________________________________
From: Development <development-***@lists.qt-project.org> on behalf of Bernhard Lindner <***@bernhard-lindner.de>
Sent: Tuesday, December 4, 2018 5:35:58 PM
To: ***@lists.qt-project.org
Subject: Re: [Development] Another integer typedef OR how to prepare for 64-bit in Qt 5

Hi!

> You're inverting the argument we've had for using int.

Yes, because I have other priorities.

> It's not that it's
> always sufficient, but it's sufficient for a sufficient number of use-cases,
> thus avoiding unnecessary casts *and* avoiding mistakes in conversion.

Avoiding cast? Creating casts! I rarly need to cast unsigned sizes to signed in a
compliant environment. But I need a LOT of casts whenever integrating Qt with compliant
code.

Whoever ignores standards is the one who creates interfacing effort. And Qt is
(unnecessarily) ignoring standards. If the world would consit of Qt only, then maybe you
would be right. But most of the world is not Qt.

Using int instead of size_t (or a compatible type) for me means: More unit tests, more
error checks and asserts, more documentation, more document reading, lots of casts when interfacing standard compliant code. All due to the negative value range.

> The literal "1" is int, not size_t or long, so i,'s far easier to write int
> code than size_t.

You think this is a strong argument?

> In any case, Qt will not use an unsigned type.

So qsizetype will be signed in Qt6?

What a pitty. I assumed the non-compliant int usage would be some legacy burden and hoped
it would be fixed at the earliest opportunity.

Anyway the usage of qsizetype would be an improvment.

--
Best Regards,
Bernhard
André Pönitz
2018-12-05 04:48:05 UTC
Permalink
On Tue, Dec 04, 2018 at 12:33:06AM +0100, Bernhard Lindner wrote:
> Hi!
>
> > > > Not yet. Yes, it will be a huge number.
> > >
> > > Which compels the questions:
> > >
> > > 1) is it worth it?
> >
> > No.
>
> What about those people who want (or must) implement clean interfaces in their code using
> standard size_t?
>
> Even if the number of objects/sizes/counts is usually below 2^31, it still is a pain to
> either suffer from numerous static analysis warnings (when implicitly converting size_t
> and int) or from implementing numerous casts.

Assuming that this "clean interface" means std:: snake_style and lower case
types "those people" will end up with shallow wrappers already for aestetical
reasons, anyway. "Translating" between int and what_ever_thing::size_type is
done there, too, as are conversion between std::string and QByteArray etc.

Andre'
Thiago Macieira
2018-12-04 06:31:55 UTC
Permalink
On Monday, 3 December 2018 11:43:17 PST André Pönitz wrote:
> > 1) is it worth it?
>
> No.
>
> It's really rare to need a container of that size in a Qt context,
> and if one does, there's always std:: containers - which are likely
> to be used at sizes much smaller than INT_MAX anyways.

It may have been in the past, but I don't think we can honestly say it will
continue so in the future. As a simple example, someone may want to load a
file larger than 2 GB via QIODevice::readAll(). It's probably a stupid idea to
do that, but with a fast SSD that operation could complete in 10 seconds or
less.

Asking users to use std:: containers is not sufficient if we don't use them
ourselves. How would the case above return a QByteArray with more than 2 GB?

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Adam Light
2018-12-04 16:28:13 UTC
Permalink
On Mon, Dec 3, 2018 at 11:02 PM Thiago Macieira <***@intel.com>
wrote:

> On Monday, 3 December 2018 11:43:17 PST André Pönitz wrote:
> > > 1) is it worth it?
> >
> > No.
> >
> > It's really rare to need a container of that size in a Qt context,
> > and if one does, there's always std:: containers - which are likely
> > to be used at sizes much smaller than INT_MAX anyways.
>
> It may have been in the past, but I don't think we can honestly say it
> will
> continue so in the future. As a simple example, someone may want to load a
> file larger than 2 GB via QIODevice::readAll(). It's probably a stupid
> idea to
> do that, but with a fast SSD that operation could complete in 10 seconds
> or
> less.
>
> Asking users to use std:: containers is not sufficient if we don't use
> them
> ourselves. How would the case above return a QByteArray with more than 2
> GB?
>
>
Another place where this can be a problem is QMimeData, which uses a
QByteArray. We'd like for our application to be able to read/write >2GB
from/to the clipboard, but that's not currently possible. Reimplementing
all clipboard handling ourselves on two platforms would be a lot of work
and using std:: containers with Qt's clipboard API isn't an option.

Adam
Thiago Macieira
2018-12-04 17:05:30 UTC
Permalink
On Tuesday, 4 December 2018 08:28:13 PST Adam Light wrote:
> Another place where this can be a problem is QMimeData, which uses a
> QByteArray. We'd like for our application to be able to read/write >2GB
> from/to the clipboard, but that's not currently possible. Reimplementing
> all clipboard handling ourselves on two platforms would be a lot of work
> and using std:: containers with Qt's clipboard API isn't an option.

Right.

My point is that even if we didn't want to make the effort to apply it for
most containers, we'd need to do it in QString and QByteArray. Those two share
most of their internals with QVector, so doing it for QVector (especially
after my changes from Qt 5.0 still unpushed) should be acceptable. Then QList
will become QVector in Qt 6, so that gets done too.

QHash and QMap will probably be rewritten, so they're backed by
std::unordered_map, so it's a good time to do it too.

That just leaves QLinkedList.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Jedrzej Nowacki
2018-11-02 13:50:50 UTC
Permalink
On Friday, November 2, 2018 4:42:52 AM CET Thiago Macieira wrote:
> We have a lot of API that, for Qt 6, we've already decided to extend to
> 64-bit on 64-bit platforms, but keep as decently-sized 32-bit on 32-bit
> ones.

Smells like qreal, with all problems that it causes... We could reconsider
costs of using 64bit everywhere, it would streamline debugging of edge cases.

> (...)
> Option 4: create a central #if and use this new type, starting now
> Same as #3, but instead of per class, it would be central. Moreover, with
> one preprocessor trick (-Dnewinttype=int), we can fool qdoc into generating
> documentation as it is today. In Qt 6, once we decide we don't need Qt 5
> merging anymore, we can also do a global search-and-replace to qsizetype
> (optional).

If I have to I would also pick option 4, but:
1. qsizetype can _not_ be a built in metatype in Qt5 (at best an alias).
2. qsizetype needs to have QDataStream operators so it doesn't fallback to a
platform dependent size
3. QDataStream stream autotests of class that uses the type would need to be
extended.
4. I bet there is more, issues to solve, caused by pointer arithmetic ...

I would not try to trick qdoc to not see it. The consequences of size changing
"randomly" are far too important, to be hidden.

Cheers,
Jędrek
Thiago Macieira
2018-11-02 15:02:55 UTC
Permalink
On Friday, 2 November 2018 06:50:50 PDT Jedrzej Nowacki wrote:
> On Friday, November 2, 2018 4:42:52 AM CET Thiago Macieira wrote:
>
> > We have a lot of API that, for Qt 6, we've already decided to extend to
> > 64-bit on 64-bit platforms, but keep as decently-sized 32-bit on 32-bit
> > ones.
>
> Smells like qreal, with all problems that it causes... We could reconsider
> costs of using 64bit everywhere, it would streamline debugging of edge
> cases.

[Intel hat on]
I don't mind this. All Intel processors are 64-bit, all Linux workloads are
64-bit.

[Qt maintainer hat on]
Sorry, that would mean 32-bit builds use a larger-than-a-register type which
means extra carry operations and slower multiplications. That's probably not a
good idea.

> If I have to I would also pick option 4, but:
> 1. qsizetype can _not_ be a built in metatype in Qt5 (at best an alias).

It's not going to be. It's a typedef to either int or long long, depending on
your pointer's size.

> 2. qsizetype needs to have QDataStream operators so it doesn't fallback to
> a platform dependent size

Since it's a typedef, that can't happen. But you've got a point: we need to
serialise in explicit 64-bit widths.

> 3. QDataStream stream autotests of class that uses the type would need to be
> extended.
> 4. I bet there is more, issues to solve, caused by pointer arithmetic ...

No doubt. That's why I wanted to start now.

> I would not try to trick qdoc to not see it. The consequences of size
> changing "randomly" are far too important, to be hidden.

So you want users to know now, in Qt 5, that this particular type in the API
will become qsizetype in Qt 6?

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Иван Комиссаров
2018-11-02 15:39:52 UTC
Permalink
Yeah, forget my argument. Go for 4.

Иван Комиссаров

> 2 нояб. 2018 г., в 16:02, Thiago Macieira <***@intel.com> написал(а):
>
>> On Friday, 2 November 2018 06:50:50 PDT Jedrzej Nowacki wrote:
>>> On Friday, November 2, 2018 4:42:52 AM CET Thiago Macieira wrote:
>>>
>>> We have a lot of API that, for Qt 6, we've already decided to extend to
>>> 64-bit on 64-bit platforms, but keep as decently-sized 32-bit on 32-bit
>>> ones.
>>
>> Smells like qreal, with all problems that it causes... We could reconsider
>> costs of using 64bit everywhere, it would streamline debugging of edge
>> cases.
>
> [Intel hat on]
> I don't mind this. All Intel processors are 64-bit, all Linux workloads are
> 64-bit.
>
> [Qt maintainer hat on]
> Sorry, that would mean 32-bit builds use a larger-than-a-register type which
> means extra carry operations and slower multiplications. That's probably not a
> good idea.
>
>> If I have to I would also pick option 4, but:
>> 1. qsizetype can _not_ be a built in metatype in Qt5 (at best an alias).
>
> It's not going to be. It's a typedef to either int or long long, depending on
> your pointer's size.
>
>> 2. qsizetype needs to have QDataStream operators so it doesn't fallback to
>> a platform dependent size
>
> Since it's a typedef, that can't happen. But you've got a point: we need to
> serialise in explicit 64-bit widths.
>
>> 3. QDataStream stream autotests of class that uses the type would need to be
>> extended.
>> 4. I bet there is more, issues to solve, caused by pointer arithmetic ...
>
> No doubt. That's why I wanted to start now.
>
>> I would not try to trick qdoc to not see it. The consequences of size
>> changing "randomly" are far too important, to be hidden.
>
> So you want users to know now, in Qt 5, that this particular type in the API
> will become qsizetype in Qt 6?
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
>
>
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
André Somers
2018-11-02 19:18:56 UTC
Permalink
Hi,


> On 2 Nov 2018, at 16:02, Thiago Macieira <***@intel.com> wrote:
>
>> On Friday, 2 November 2018 06:50:50 PDT Jedrzej Nowacki wrote:
>>> On Friday, November 2, 2018 4:42:52 AM CET Thiago Macieira wrote:
>>>
>>> We have a lot of API that, for Qt 6, we've already decided to extend to
>>> 64-bit on 64-bit platforms, but keep as decently-sized 32-bit on 32-bit
>>> ones.
>>
>> Smells like qreal, with all problems that it causes... We could reconsider
>> costs of using 64bit everywhere, it would streamline debugging of edge
>> cases.
>
> [Intel hat on]
> I don't mind this. All Intel processors are 64-bit, all Linux workloads are
> 64-bit.
>
> [Qt maintainer hat on]
> Sorry, that would mean 32-bit builds use a larger-than-a-register type which
> means extra carry operations and slower multiplications. That's probably not a
> good idea.
>
>> If I have to I would also pick option 4, but:
>> 1. qsizetype can _not_ be a built in metatype in Qt5 (at best an alias).
>
> It's not going to be. It's a typedef to either int or long long, depending on
> your pointer's size.
>
>> 2. qsizetype needs to have QDataStream operators so it doesn't fallback to
>> a platform dependent size
>
> Since it's a typedef, that can't happen. But you've got a point: we need to
> serialise in explicit 64-bit widths.
>
>> 3. QDataStream stream autotests of class that uses the type would need to be
>> extended.
>> 4. I bet there is more, issues to solve, caused by pointer arithmetic ...
>
> No doubt. That's why I wanted to start now.
>
>> I would not try to trick qdoc to not see it. The consequences of size
>> changing "randomly" are far too important, to be hidden.
>
> So you want users to know now, in Qt 5, that this particular type in the API
> will become qsizetype in Qt 6?

Yes. Why not take it one step further and just call it qsizetype too? The docs can say that this will be int in Qt5, but will change in Qt 6 to what you started with?

André

>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
>
>
>
> _______________________________________________
> Development mailing list
> ***@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
Thiago Macieira
2018-11-02 19:36:15 UTC
Permalink
On Friday, 2 November 2018 12:18:56 PDT André Somers wrote:
> > So you want users to know now, in Qt 5, that this particular type in the
> > API will become qsizetype in Qt 6?
>
> Yes. Why not take it one step further and just call it qsizetype too? The
> docs can say that this will be int in Qt5, but will change in Qt 6 to what
> you started with?

Because qsizetype already exists in Qt 5 and it will remain as-is in Qt 6.

We need a type that is int in Qt 5 and qsizetype in Qt 6.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Jedrzej Nowacki
2018-11-08 07:40:09 UTC
Permalink
Dnia piątek, 2 listopada 2018 16:02:55 CET Thiago Macieira pisze:
> On Friday, 2 November 2018 06:50:50 PDT Jedrzej Nowacki wrote:
> > On Friday, November 2, 2018 4:42:52 AM CET Thiago Macieira wrote:
> > > We have a lot of API that, for Qt 6, we've already decided to extend to
> > > 64-bit on 64-bit platforms, but keep as decently-sized 32-bit on 32-bit
> > > ones.
> >
> > Smells like qreal, with all problems that it causes... We could reconsider
> > costs of using 64bit everywhere, it would streamline debugging of edge
> > cases.
>
> [Intel hat on]
> I don't mind this. All Intel processors are 64-bit, all Linux workloads are
> 64-bit.
>
> [Qt maintainer hat on]
> Sorry, that would mean 32-bit builds use a larger-than-a-register type which
> means extra carry operations and slower multiplications. That's probably
> not a good idea.
I know. What I do not know is if that really matters for users, especially as
we can use 64-bit values in API but 32-bit values in the implementation. Nah,
I have to admit, not the greatest idea :-)

> > If I have to I would also pick option 4, but:
> > 1. qsizetype can _not_ be a built in metatype in Qt5 (at best an alias).
>
> It's not going to be. It's a typedef to either int or long long, depending
> on your pointer's size.
Good

> > 2. qsizetype needs to have QDataStream operators so it doesn't fallback
> > to
> > a platform dependent size
>
> Since it's a typedef, that can't happen. But you've got a point: we need to
> serialise in explicit 64-bit widths.
Yes that was the thing I tried to express :-)

> > 3. QDataStream stream autotests of class that uses the type would need to
> > be>
> > extended.
> >
> > 4. I bet there is more, issues to solve, caused by pointer arithmetic ...
>
> No doubt. That's why I wanted to start now.
>
> > I would not try to trick qdoc to not see it. The consequences of size
> > changing "randomly" are far too important, to be hidden.
>
> So you want users to know now, in Qt 5, that this particular type in the API
> will become qsizetype in Qt 6?
Yes, or to turn it around, why not? It sounds like an information, which
doesn't obscure generated documentation and which would implicitly document
porting Qt5->Qt6. On the other hand it is a really minor detail, both
approaches are valid :-).

Cheers,
Jędrek
Continue reading on narkive:
Loading...