[Development] 16-bit PCRE landed in upstream master

[Stephen Kelly]

Just FYI.

This is due for release in mid-January, so Qt 5.0 could depend on the 16 bit
API already.

I'd recommend that the people who know about QRegExp take a look.

Thanks,

------- You are receiving this mail because: -------
You are on the CC list for the bug.
http://bugs.exim.org/show_bug.cgi?id=1049
--- Comment #44 from Zoltan Herczeg
---

... but please note that the documentation has not been touched at all.
That is my next big job.

True, but the library itself is ready to try, and we would be really happy
if you would give us feedback about the library (especially from those who
plan to use it).
Let me give you a short guide about the 16 bit PCRE library.
When you checkout PCRE you need to create the configure script first. The
svn co svn://vcs.exim.org/pcre/code/trunk pcre
libtoolize -c -f && aclocal && autoheader && automake -a -c && autoconf
--enable-pcre8 / --disable-pcre8 (enabled by default)
--enable-pcre16 / --disable-pcre16 (disabled by default)
--enable-utf / --disable-utf (disabled by default)
replaces --enable-utf8 which is obsolote from
now on, although it is kept for compatibility for some time.
The value set for --enable/disable-utf8 is simply
copied to --enable/disable-utf
With --enable-pcre8 the usual libpcre is created. Same performance, same
binary size, it is not really changed. We hope this keep users of the 8
bit library happy. However, with --enable-pcre16 a new, libpcre16 library
is created, which contains the 16 bit functions.
We still have a single pcre.h, which contains the forward declarations of
both 8 and 16 bit functions. This is not an issue in C, just don't use
functions from a library which is not linked to your application (with
-lpcre or -lpcre16).
The API itself is pretty simple: every function has a 16 bit counterpart,
starting with pcre16_ prefix. I.e: pcre_compile, pcre16_compile. That's
all. They have the same arguments, except some char* pointers are replaced
to short* when appropriate.
PCRE_EXP_DECL pcre *pcre_compile(const char *, int, const char **, int *,
const unsigned char *);
PCRE_EXP_DECL pcre *pcre16_compile(PCRE_SPTR16, int, const char **, int *,
const unsigned char *);
PCRE_SPTR16 is const short *
Warning: do not mix 8 and 16 bit API! Example: value returned by
pcre16_study must be freed by pcre16_free_study! Segfaults will occure if
you use pcre_free_study! Also, use pcre16_free for freeing the compiled
regex returned by pcre16_compile, not pcre_free. Keep in mind that all
functions and static variables are duplicated!
One more thing: --disable-pcre8 also disable posix API, pcregrep and
pcrecpp. These utilities have no 16 bit counterpart, and not planned at
the moment until someone need them (especially for posix API, since posix
standard has no 16 bit API for regex).
However --enable-utf --enable-unicode-properties and --enable-jit is
supported by both 8 and 16 bit libraries as well.
Updating the CMAKE build system is still an open task, we would be really
happy if someone could help us!

[Giuseppe D'Angelo]

Post by Stephen Kelly
This is due for release in mid-January, so Qt 5.0 could depend on the 16 bit
API already.

I'd like to bring the discussion we had on IRC yesterday on the
mailing list. What are the people's opinions about whether it's "good"
(for any value of good) for Qt 5.0 to depend on the 16 bit PCRE?

[Jonas M. Gastal]

Post by Giuseppe D'Angelo

Post by Stephen Kelly
This is due for release in mid-January, so Qt 5.0 could depend on the 16
bit API already.

I'd like to bring the discussion we had on IRC yesterday on the
mailing list. What are the people's opinions about whether it's "good"
(for any value of good) for Qt 5.0 to depend on the 16 bit PCRE?

I only caught a part of the discussion, but a concern that was raised was that
depending on bleeding edge is not good, but I disagree. Qt 5.0 is bleeding
edge, no reason not to depend on it.

Gastal

[Robin Burchell]

Post by Giuseppe D'Angelo

Post by Stephen Kelly
This is due for release in mid-January, so Qt 5.0 could depend on the 16 bit
API already.

I'd like to bring the discussion we had on IRC yesterday on the
mailing list. What are the people's opinions about whether it's "good"
(for any value of good) for Qt 5.0 to depend on the 16 bit PCRE?

Reiterating my views from that discussion...

This feature in PCRE is bleeding edge.
Qt 5 is also bleeding edge.

In my opinion, it is ok to depend on it, as for Qt-in-distro targets,
we expect that PCRE will most likely release, and be available, long
before Qt 5 will be, so the case of Qt 5 by default in distros will
happen after PCRE is already available. This is not the case for LTS
releases, etc, and never will be, because they don't like to bump
versions. On the other hand, Qt 5 won't be available there as a
default either, for similar reasons.

As we'll be bundling a copy of PCRE somehow anyway (for platforms
where it is not provided), it should be easy enough to fallback to the
bundled copy should the system copy not be sufficient. The costs (for
us) are low, the costs for users of old distros are small - an
additional library on disk and in-memory for Qt 5 using apps, and it
means less code (and more optimal code) in Qt itself.

I'd say it's worth it.

[Thiago Macieira]

Post by Giuseppe D'Angelo

Post by Stephen Kelly
This is due for release in mid-January, so Qt 5.0 could depend on the 16
bit API already.

I'd like to bring the discussion we had on IRC yesterday on the
mailing list. What are the people's opinions about whether it's "good"
(for any value of good) for Qt 5.0 to depend on the 16 bit PCRE?

I think we should import PCRE into src/3rdparty (or equivalent "qtsupport"
module) and keep it there basically forever. While Linux distributions may
update soon to this version of PCRE, Mac OS X isn't likely to get a new
version until 10.8 in 2013 and Windows users will never have it.

At the same time, we need configure to support -system-pcre, with the
appropriate checks for the UTF-16 API.