Discussion:
Multitouch on top of XInput 2.2
(too old to reply)
Christian Spielberger
2012-06-28 11:03:02 UTC
Permalink
Hi,

I got multitouch on top of XInput 2.2 to work.

See
!

Repo:
See https://gitorious.org/~cspiel/qt/cspiels-qt-4_8-with-xinput2_1 !

Branch:
4.8-XI2.2-touch

Maybe the branch can be the base for the multitouch integration for X11.

How the branch came about:
1. Applied changes of branch
git://gitorious.org/+qt-developers/qt/multipoint-touch.git which is
based on qt 4.7.x on qt 4.8.

2. Changed multi touch handling to XI2.2.

3. Tested successfully on XOrgR7.7 with atmel_mxt_ts touch screen.

regards,
chris.
Shawn Rutledge
2012-06-28 15:09:41 UTC
Permalink
On 28 June 2012 13:03, Christian Spielberger
Post by Christian Spielberger
Hi,
I got multitouch on top of XInput 2.2 to work.
See http://youtu.be/sRSC4CEYmIg !
See https://gitorious.org/~cspiel/qt/cspiels-qt-4_8-with-xinput2_1 !
4.8-XI2.2-touch
Maybe the branch can be the base for the multitouch integration for X11.
Excellent! I was planning to work on Qt 5 support for XI 2.2 soon, so
that should be helpful.
Shawn Rutledge
2012-07-09 17:42:02 UTC
Permalink
Post by Shawn Rutledge
On 28 June 2012 13:03, Christian Spielberger
Post by Christian Spielberger
Hi,
I got multitouch on top of XInput 2.2 to work.
See http://youtu.be/sRSC4CEYmIg !
See https://gitorious.org/~cspiel/qt/cspiels-qt-4_8-with-xinput2_1 !
4.8-XI2.2-touch
Maybe the branch can be the base for the multitouch integration for X11.
Excellent! I was planning to work on Qt 5 support for XI 2.2 soon, so
that should be helpful.
However due to the way the code has been rearranged, the patch will be in
different files anyway. Would you like to convert your changes into a patch
on gerrit so it can be reviewed to go into 4.8.x? I can also test it
on a couple of devices.

My patch for Qt 5 is here:

https://codereview.qt-project.org/#change,30469
--
MVH, Shawn Rutledge ❖ "ecloud" on IRC
Shawn Rutledge
2012-07-10 13:42:46 UTC
Permalink
Post by Shawn Rutledge
Post by Shawn Rutledge
On 28 June 2012 13:03, Christian Spielberger
Post by Christian Spielberger
Hi,
I got multitouch on top of XInput 2.2 to work.
See http://youtu.be/sRSC4CEYmIg !
See https://gitorious.org/~cspiel/qt/cspiels-qt-4_8-with-xinput2_1 !
4.8-XI2.2-touch
Maybe the branch can be the base for the multitouch integration for X11.
Excellent! I was planning to work on Qt 5 support for XI 2.2 soon, so
that should be helpful.
However due to the way the code has been rearranged, the patch will be in
different files anyway.
To be more specific: due to the introduction of QPA it's quite different; in
Qt 5 the platform XCB plugin generates touchpoints of type struct TouchPoint
to put into a QWindowSystemInterfacePrivate::TouchEvent, whereas in your patch
you can generate a QTouchEvent::TouchPoint directly. However from reading
your code I learned of the existence of the pre-converted coordinates:

touchPoint.setPos( QPointF(xideviceevent->event_x, xideviceevent->event_y) );
touchPoint.setScreenPos( QPointF(xideviceevent->root_x, xideviceevent->root_y)
);

as opposed to having to get them from the valuators and convert them manually.
But there are some odd things about it: you used XLib so you get them as
doubles, but in xcb they have 16.16 fixed-point format, so I have convert them
to qreals.

Another oddity is that when testing a touchscreen, each xXIDeviceEvent from
xcb has the correct coordinates already, whereas with an Apple Trackpad, both
of them will be at the same point where the cursor is. Maybe that's because
the touchpad is a relative device rather than absolute, or maybe it's an xcb
bug? Because what I think I want is that the first touchpoint should be at
the cursor position, and the second one should be offset the same distance
from the first as my second finger is offset on the trackpad. But with the
current version of my patch, if I run the fingerpaint example, all the fingers
are superimposed, so you just get different ellipses in different colors
painted on top of each other. (At least the major/minor axes are unique
depending on pressure.) So I'm thinking of calculating the offset from the
converted raw point to the given root_x and root_y for the first point, and
then somehow offsetting the second and subsequent points. But I'm not sure if
it's right for Qt to do that; if it's a bug or oversight in xcb (or somewhere
else), maybe it will get fixed there eventually.
Post by Shawn Rutledge
Would you like to convert your changes into a patch
on gerrit so it can be reviewed to go into 4.8.x? I can also test it on a
couple of devices.
I noticed you deleted the code to deal with the valuators completely, so that
means in your branch there is no support for pressure or major/minor axes,
right? When I run fingerpaint against your branch, it is indeed insensitive
to pressure, I just get circles all the same size. I don't think you should
be ignoring all the valuators, even though you are correctly making use of the
given points from the event itself.

The axes are supposed to describe the size and shape of the elliptical contact
patch of your finger against the touch device. I have tested my Qt 5 patch on
the following devices:

Lenovo S10-3t: has neither pressure nor axes; 2 points max

Apple Bluetooth trackpad: has major and minor axes plus (bogus?) orientation,
5 points max; points from the XI2 event are all at the same position as the
cursor

Apple trackpad on a Macbook Pro (17" from 2009 or so): has major and minor
axes plus ABS Pressure (NOT Abs MT Pressure), 5 points max; forgot whether it
has orientation; points from the XI2 event are all at the same position as the
cursor

ExoPC: no axes, has Abs MT Pressure but sometimes I have seen it go to the
minimum possible value if there are 2 fingers. 10 points max.

What I mean by "bogus orientation" on the trackpad is that instead of having
per-finger orientation, it exists only if 2 or more fingers are pressed, so it
would seem to be related to the angle between them; but it's quite jumpy, and
I'm not sure what a range from -31 to +32 is supposed to mean anyway. High-
end Wacom tablets apparently give 3 angles: the rotation of the stylus, and
two vertical angles w.r.t. the Wacom surface. I think the orientation is
supposed to be like the stylus rotation angle, and should really be on a per-
finger basis, so I guess we should call it rotation to be consistent, so that
we are ready in case future touch devices report more than one angle as well.
If the trackpad has a high density grid of capacitive sensors, then I think
the idea is that trackpad's processor should be fitting an ellipse to each
blob, and reporting the width, height and rotation of the ellipse. (Or maybe
the driver does it?) If rotation was really not reported from each of the
curve-fit ellipses, I wonder why.

The data coming from the BT trackpad looks like this

XI2 event type 19 seq 333 detail 400 pos 0x3D00000,0x28F0000
976.000000,655.000000 root pos 2896.000000,679.000000
valuator class label 636 value 71.000000 from range -2909.000000 ->
3167.000000 name Abs MT Position X
valuator class label 637 value -14.000000 from range -2456.000000 ->
2565.000000 name Abs MT Position Y
valuator class label 638 value 128.000000 from range 0.000000 -> 255.000000
name Abs MT Touch Major
valuator class label 639 value 168.000000 from range 0.000000 -> 255.000000
name Abs MT Touch Minor
tp 400 state 2 pos norm QPointF(0.490454, 0.486357) area
QRectF(2886.81,666.686 18.3749x24.6289) angle 0 pressure -1

but with even larger values for the axes on the Macbook Pro. So I decided to
convert the value such that that maximum value corresponds to max finger width
in mm (it's #defined to 10mm - perhaps a bit much) and convert that to pixels
based on the actual screen resolution.
--
MVH, Shawn Rutledge ❖ "ecloud" on IRC
Christian Spielberger
2012-07-19 13:14:53 UTC
Permalink
Post by Shawn Rutledge
I noticed you deleted the code to deal with the valuators completely, so that
means in your branch there is no support for pressure or major/minor axes,
right? When I run fingerpaint against your branch, it is indeed insensitive
to pressure, I just get circles all the same size. I don't think you should
be ignoring all the valuators, even though you are correctly making use of the
given points from the event itself.
I re-inserted the valuator evaluation for pressure, major- and minor
axes. The major axis valuator (AbsMTTouchMajor) has random outliers,
which does not come from the kernel modul. The evtest tool reports
correct values. So maybe worth to be further investigated by the Xorg
maintainer(s). My device is an "Atmel maXTouch Touchscreen".
Post by Shawn Rutledge
The axes are supposed to describe the size and shape of the elliptical contact
patch of your finger against the touch device. I have tested my Qt 5 patch on
Lenovo S10-3t: has neither pressure nor axes; 2 points max
Apple Bluetooth trackpad: has major and minor axes plus (bogus?) orientation,
5 points max; points from the XI2 event are all at the same position as the
cursor
Apple trackpad on a Macbook Pro (17" from 2009 or so): has major and minor
axes plus ABS Pressure (NOT Abs MT Pressure), 5 points max; forgot whether it
has orientation; points from the XI2 event are all at the same position as the
cursor
ExoPC: no axes, has Abs MT Pressure but sometimes I have seen it go to the
minimum possible value if there are 2 fingers. 10 points max.
What I mean by "bogus orientation" on the trackpad is that instead of having
per-finger orientation, it exists only if 2 or more fingers are pressed, so it
would seem to be related to the angle between them; but it's quite jumpy, and
I'm not sure what a range from -31 to +32 is supposed to mean anyway. High-
end Wacom tablets apparently give 3 angles: the rotation of the stylus, and
two vertical angles w.r.t. the Wacom surface. I think the orientation is
supposed to be like the stylus rotation angle, and should really be on a per-
finger basis, so I guess we should call it rotation to be consistent, so that
we are ready in case future touch devices report more than one angle as well.
If the trackpad has a high density grid of capacitive sensors, then I think
the idea is that trackpad's processor should be fitting an ellipse to each
blob, and reporting the width, height and rotation of the ellipse. (Or maybe
the driver does it?) If rotation was really not reported from each of the
curve-fit ellipses, I wonder why.
The data coming from the BT trackpad looks like this
XI2 event type 19 seq 333 detail 400 pos 0x3D00000,0x28F0000
976.000000,655.000000 root pos 2896.000000,679.000000
valuator class label 636 value 71.000000 from range -2909.000000 ->
3167.000000 name Abs MT Position X
valuator class label 637 value -14.000000 from range -2456.000000 ->
2565.000000 name Abs MT Position Y
valuator class label 638 value 128.000000 from range 0.000000 -> 255.000000
name Abs MT Touch Major
valuator class label 639 value 168.000000 from range 0.000000 -> 255.000000
name Abs MT Touch Minor
tp 400 state 2 pos norm QPointF(0.490454, 0.486357) area
QRectF(2886.81,666.686 18.3749x24.6289) angle 0 pressure -1
but with even larger values for the axes on the Macbook Pro. So I decided to
convert the value such that that maximum value corresponds to max finger width
in mm (it's #defined to 10mm - perhaps a bit much) and convert that to pixels
based on the actual screen resolution.
Maybe a good decision. I took the absolute values of AbsMTTouchMajor.
Christian Spielberger
2012-07-23 07:13:45 UTC
Permalink
Post by Christian Spielberger
I re-inserted the valuator evaluation for pressure, major- and minor
axes. The major axis valuator (AbsMTTouchMajor) has random outliers,
which does not come from the kernel modul. The evtest tool reports
correct values. So maybe worth to be further investigated by the Xorg
maintainer(s). My device is an "Atmel maXTouch Touchscreen".
record the device with evemu and send me the output, I can have a look.
http://people.freedesktop.org/~whot/evemu/
Cheers,
Peter
Hi Peter,

you find the records attached to this mail.

maXTouch.events is an up to five finger touch recording.
maXTouch.events2 only two fingers with varying pressure.

regards,
chris.
Christian Spielberger
2012-07-25 15:29:47 UTC
Permalink
Post by Shawn Rutledge
Post by Shawn Rutledge
On 28 June 2012 13:03, Christian Spielberger
Post by Christian Spielberger
Hi,
I got multitouch on top of XInput 2.2 to work.
See http://youtu.be/sRSC4CEYmIg !
See https://gitorious.org/~cspiel/qt/cspiels-qt-4_8-with-xinput2_1 !
4.8-XI2.2-touch
Maybe the branch can be the base for the multitouch integration for X11.
Excellent! I was planning to work on Qt 5 support for XI 2.2 soon, so
that should be helpful.
However due to the way the code has been rearranged, the patch will be in
different files anyway. Would you like to convert your changes into a patch
on gerrit so it can be reviewed to go into 4.8.x? I can also test it
on a couple of devices.
https://codereview.qt-project.org/#change,30469
Hi Shawn,

I tried to commit my branch to gerrit following this procedure.


1. Made a new branch from current 4.8 which rests here:
https://qt.gitorious.org/~cspiel/qt/cspiels-qt-4_8-with-xinput2_1/commits/4.8-XI2.2

2. Merged with
git://gitorious.org/+qt-developers/qt/multipoint-touch.git branch
refs/heads/master

3. Applied my changes with git cherry-pick.

4. Set up the gerrit account. In my .git/config I have:
[remote "gerrit"]
url = ***@codereview.qt-project.org:qt/qt
fetch = +refs/heads/*:refs/remotes/gerrit/*
[branch "gerrit_4.8"]
remote = gerrit
merge = refs/for/4.8

5. Changed to gerrit_4.8 locally and merged with 4.8-XI2.2

6. git push gives:

$ git push
Counting objects: 422, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (133/133), done.
Writing objects: 100% (377/377), 62.46 KiB, done.
Total 377 (delta 317), reused 269 (delta 244)
remote: Resolving deltas: 14% (45/317)
remote:
remote: ERROR: In commit 408021665e7a6b393b9aad3b911411ca02f7186b
remote: ERROR: committer email address ***@nokia.com
remote: ERROR: does not match your user account.
remote: ERROR:
remote: ERROR: The following addresses are currently registered:
remote: ERROR: ***@bct-electronic.com
remote: ERROR:
remote: ERROR: To register an email address, please visit:
remote: ERROR: https://codereview.qt-project.org/#settings,contact
remote:
remote:
To ***@codereview.qt-project.org:qt/qt
! [remote rejected] gerrit_4.8 -> refs/for/4.8 (invalid committer)
error: failed to push some refs to '***@codereview.qt-project.org:qt/qt'

Which makes sense since the commit 408021665 is originally from repo
multipoint-touch.git. This commit is from Bradley T. Hughes
<***@nokia.com>. So the email addr ***@nokia.com
makes no sense, but it makes sense that I can not submit this to gerrit.

What I want is to suggest the changes in the multipoint-touch.git repo
plus my changes to be reviewed/merged into 4.8 mainline. How such a
situation should be handled (with gerrit)?


regards,
chris.
Thiago Macieira
2012-07-25 16:32:56 UTC
Permalink
Post by Christian Spielberger
What I want is to suggest the changes in the multipoint-touch.git repo
plus my changes to be reviewed/merged into 4.8 mainline. How such a
situation should be handled (with gerrit)?
It cannot be. Qt 4.8 will not support XInput 2.2, since that would be a new
feature.

You're welcome to keep your branch elsewhere and link to it from the website,
so people know to find it. But they need to understand that it's not official and
the Qt Project cannot support it.
--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Intel Sweden AB - Registration Number: 556189-6027
Knarrarnäsgatan 15, 164 40 Kista, Stockholm, Sweden
Loading...