RawInput WinAPI: GetRawInputBuffer() and message handling

Question

Why
I'm trying to get input from a barcode scanner to my (visual) application. I would like to ignore input from other devices and get the input even if the application loses focus. I found the RawInput API recommended on SO and also elsewhere, to achieve this.

I've focused on GetRawInputBuffer() to read the input, as I'm expecting ~2 scans per second and ~700 events (key down / key up) triggered for each scan (assuming the scanner is acting as a keyboard). The documentation mentions to use GetRawInputBuffer() "for devices that can produce large amounts of raw input". I don't know whether the above actually qualifies...

Problem
I've successfully received input data - but there is something I must be doing wrong (possibly fundamentally...) as I can't figure out a good way to get consistent results. The raw data seems to 'disappear' very quickly and I often get no data back. There are similar existing questions on SO about GetRawInputBuffer() but they have only gotten me so far... Some notes:

I use 'unRawInput.pas' from https://github.com/lhengen/RawInput
I create a window handle using AllocateHwnd() and register it for raw input

(edit) Question
How/when should I (correctly) call GetRawInputBuffer() in a visual application to get consistent results, meaning e.g. all key events since the last call? Or: How/why do events seem to get 'discarded' between calls and how can I prevent it?

Code
The below code is a 64bit console application showcasing 3 approaches I've tried so far, and their problems (uncomment / comment-out approaches as described in code comments of the main begin-end.-block).

approach #1: Sleep() while input is happening, then reading the buffer right away. I got the idea to Sleep() from the learn.microsoft.com sample code - and it works very well in that it seems to get all the input, but I don't think this is practical as my application needs to remain responsive.
approach #2: use GetMessage() - usually, this yields no data, unless you type very quickly (like, mash keys) and even then, it's maybe 50% of input, tops.
approach #3: use PeekMessage() and PM_NOREMOVE - this seems to get input very consistently but maxes out the thread.

program readrawbuffer;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  WinAPI.Windows,
  WinAPI.Messages,
  System.Classes,
  System.SysUtils,
  URawInput in '..\URawInput.pas';       // from: https://github.com/lhengen/RawInput

type
  TGetInput = class
  strict private
    fRawInputStructureSize: UINT;
    fRawInputHeaderSize: UINT;
    fRawInputBufferSize: Cardinal;
    fRawInputDevice: RAWINPUTDEVICE;
    fRawInputBuffer: PRAWINPUT;
    procedure RawInputWndProc(var aMsg: TMessage);
  public
    fRawInputWindowHnd: HWND;
    function ReadInputBuffer(): String;
    constructor Create();
    destructor Destroy(); override;
  end;

  constructor TGetInput.Create();
  begin
    inherited;
    fRawInputStructureSize := SizeOf(RAWINPUT);
    fRawInputHeaderSize := SizeOf(RAWINPUTHEADER);
    // create buffer
    fRawInputBufferSize := 40 * 16;
    GetMem(fRawInputBuffer, fRawInputBufferSize);
    // create handle and register for raw (keyboard) input
    fRawInputWindowHnd := AllocateHWnd(RawInputWndProc);
    fRawInputDevice.usUsagePage := $1;
    fRawInputDevice.usUsage := $6;
    fRawInputDevice.dwFlags := RIDEV_INPUTSINK;
    fRawInputDevice.hwndTarget := fRawInputWindowHnd;
    if RegisterRawInputDevices(@fRawInputDevice, 1, SizeOf(RAWINPUTDEVICE)) then
      WriteLn('device(s) registered; start typing...')
    else
      WriteLn('error registering device(s): ' + GetLastError().ToString());
  end;

  destructor TGetInput.Destroy();
  begin
    if Assigned(fRawInputBuffer) then
      FreeMem(fRawInputBuffer);

    DeallocateHWnd(fRawInputWindowHnd);
    inherited;
  end;

  function TGetInput.ReadInputBuffer(): String;
  var
    pcbSize, pcbSizeT: UINT;
    numberOfStructs: UINT;
    pRI: PRAWINPUT;

  begin
    Result := String.Empty;
    pcbSize := 0;
    pcbSizeT := 0;

    numberOfStructs := GetRawInputBuffer(nil, pcbSize, fRawInputHeaderSize);
    if (numberOfStructs = 0) then
    begin
      // learn.microsoft.com says for 'nil'-call: "minimum required buffer, in bytes, is returned in *pcbSize"
      // though probably redundant, I guess it can't hurt to check:
      if (fRawInputBufferSize < pcbSize) then
      begin
        fRawInputBufferSize := pcbSize * 16;
        ReallocMem(fRawInputBuffer, fRawInputBufferSize);
      end;

      repeat
        pcbSizeT := fRawInputBufferSize;
        numberOfStructs := GetRawInputBuffer(fRawInputBuffer, pcbSizeT, fRawInputHeaderSize);
        if ((numberOfStructs > 0) and (numberOfStructs < 900000)) then
        begin
          {$POINTERMATH ON}
          pRI := fRawInputBuffer;

          for var i := 0 to (numberOfStructs - 1) do
          begin
            if (pRI.keyboard.Flags = RI_KEY_MAKE) then
              Result := Result + pRI.keyboard.VKey.ToHexString() + #32;

            pRI := NEXTRAWINPUTBLOCK(pRI);
          end;
          {$POINTERMATH OFF}
          // DefRawInputProc();   // doesn't do anything? http://blog.airesoft.co.uk/2014/04/defrawinputproc-rastinating-away/
        end
        else
          Break;
      until False;

    end
  end;

  procedure TGetInput.RawInputWndProc(var aMsg: TMessage);
  begin
    // comment-out case block for Sleep() approach; leave last DefWindowProc() line
    // leave case block for GetMessage() / PeekMessage() -approaches; comment-out last DefWindowProc() line
//    case aMsg.Msg of
//      WM_INPUT:
//        begin
//          Write(ReadInputBuffer(), '-');
//          aMsg.Result := 0;
//        end
//    else
//      aMsg.Result := DefWindowProc(fRawInputWindowHnd, aMsg.Msg, aMsg.WParam, aMsg.LParam);
//    end;

    // comment-out for GetMessage() / PeekMessage() -approaches
    aMsg.Result := DefWindowProc(fRawInputWindowHnd, aMsg.Msg, aMsg.WParam, aMsg.LParam);
  end;


var
  getInput: TGetInput;
  lpMsg: tagMSG;

begin
  getInput := TGetInput.Create();


////////////////////////////////////////////////////////////////////////////////
// approach #1: Sleep()
// >> comment-out other aproaches; comment-out case block in RawInputWndProc(), leave last DefWindowProc() line

  repeat
    WriteLn('sleeping, type now...');
    Sleep(3000);
    WriteLn('VKeys read: ', getInput.ReadInputBuffer());
  until False;


////////////////////////////////////////////////////////////////////////////////
// approach #2: GetMessage()
// >> comment-out other approaches; comment-out last DefWindowProc() line in RawInputWndProc(), leave case block

//  repeat
//    // learn.microsoft.com: "Use WM_INPUT here and in wMsgFilterMax to specify only the WM_INPUT messages."
//    if GetMessage(lpMsg, getInput.fRawInputWindowHnd, WM_INPUT, WM_INPUT) then
//      DispatchMessage(lpMsg);
//  until False;


////////////////////////////////////////////////////////////////////////////////
// approach #3: PeekMessage()
// >> comment-out other approaches; comment-out last DefWindowProc() line in RawInputWndProc(), leave case block

//  repeat
//    if PeekMessage(lpMsg, getInput.fRawInputWindowHnd, WM_INPUT, WM_INPUT, PM_NOREMOVE) then
//      DispatchMessage(lpMsg);
//
//    if PeekMessage(lpMsg, 0, 0, 0, PM_REMOVE) then
//      DispatchMessage(lpMsg);
//  until False;

  getInput.Free();
end.

@Sertac Akyuz: I like it - I have almost no experience with multi-threading though (and heard it's not trivial...) so I may wait for some alternatives first — emno
– emno, Commented Jan 19, 2020 at 15:58
I wonder if you could do this with the HID API though I'm not sure you can mark the input from the scanner device to no longer get processed into the input queue... — Stijn Sanders
– Stijn Sanders, Commented Jan 19, 2020 at 18:15

emno · Accepted Answer · 2020-02-10 17:25:00Z

2

I've overhauled this 'answer' based on the exchange in the comments below and involved testing. It does not necessarily answer my question but represents my current level of understanding and outlines the approach I ended up taking (and which seems to be working so far)

RawInput seems to be sent through WM_INPUT window messages in any case; whether when using GetRawInputData() or GetRawInputBuffer()
This means some kind of window is needed to which the messages can be sent to. This can be a hidden window. Using CreateWindowEx(0, PChar('Message'), nil, 0, 0, 0, 0, 0, HWND_MESSAGE, 0, 0, nil); works very well for me so far
This also means that there needs to be a message loop of some kind so messages can be worked off (and don't pile up).
The difference to GetRawInputData() seems to be that Windows will 'queue up' WM_INPUT messages and GetRawInputBuffer() gets and removes (from the queue) multiple messages at once. And I think the single advantage there is that input can be 'received in' quicker (higher throughput) this way than having to 'deal with every WM_INPUT message individually'.
What's tricky is that it seems like for GetRawInputBuffer() to work, it's paramount that messages except WM_INPUT are handled by regular means - and then GetRawInputBuffer() gets called regularly, which deals with the queued-up WM_INPUT messages. Any approach I took which in some way 'looked' at WM_INPUT messages ultimately caused me to get inconsistent / incomplete results from GetRawInputBuffer()

Below is my message loop, which is largely inspired by this SO answer and runs in a separate thread

repeat
  TThread.Sleep(10);

  while True do
  begin
    if (Not PeekMessage(lpMsg, 0, 0, WM_INPUT - 1, PM_NOYIELD or PM_REMOVE)) then System.Break;
    DefWindowProc(lpMsg.hwnd, lpMsg.message, lpMsg.wParam, lpMsg.lParam);
  end;

  while True do
  begin
    if (Not PeekMessage(lpMsg, 0, WM_INPUT + 1, High(Cardinal), PM_NOYIELD or PM_REMOVE)) then System.Break;
    DefWindowProc(lpMsg.hwnd, lpMsg.message, lpMsg.wParam, lpMsg.lParam);
  end;

  ReadRawInputBuffer();     // shown below; essentially reads out all queued-up input
until SomeCondition;

Reading the buffer (largely inspired by the sample code on learn.microsoft.com):

procedure ReadInputBuffer();
var
  // ...

begin
  // this returns the minimum required buffer size in ```pcbSize```
  numberOfStructs := GetRawInputBuffer(nil, pcbSize, rawInputHeaderSize);
  if (numberOfStructs = 0) then
  begin
    // read out all queued-up data
    repeat
      // ... allocate pBuffer as needed
      numberOfStructs := GetRawInputBuffer(pBuffer, pcbSize, rawInputHeaderSize);
      if ((numberOfStructs > 0) and (numberOfStructs < 900000)) then
        // do something with pBuffer / its data
        // I use a TThreadedQueue<T>; the items/data is worked off outside this thread
      else
        System.Break;
    until False;
  end
end;

(In tests of over 10 minutes, reading in > 700'000 key events doesn't seem to have lost me a single one (if my numbers don't lie). Using TStopWatch and starting/stopping at the start of the message loop (after TThread.Sleep(10)) and stopping at the end after having exhausted the input queue, in one test reading about 12k events in 15 seconds (that's close to 800 events per second), the slowest run measured... 0ms.)

edited Feb 10, 2020 at 17:25

answered Jan 24, 2020 at 19:06

emno

3112 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

21 Comments

Sertac Akyuz Over a year ago

This looks wrong. You are retrieving the same WM_INPUT over and over and over again, because you never remove any message from the queue. That's probably why the message queue fills up, and it may be the reason why you don't see any other message than WM_INPUT.

Sertac Akyuz Over a year ago

Additionally you seem to mix the concepts of buffered and unbuffered reads. If you are processing WM_INPUT, read unbuffered because a single message has information of one RAWINPUT. I'm not familiar with buffered read but I don't think it is used with a message loop.

emno Over a year ago

@SertacAkyuz Thanks for your inputs. I agree it looks wrong. What's odd is that this 'works'. I get well past the ~10K event limit that I encountered when I ignored all messaging. The thing is, when I register for raw input, I 'need' to provide a window handle in the RAWINPUTDEVICE structure - it can be NULL: "A handle to the target window. If NULL it follows the keyboard focus." ...which seems like something I don't want. As handle I provide the window handle I get from AllocateHWnd().

emno Over a year ago

To AllocateHWnd() I also need to provide a window procedure, which for me currently is simply DefWindowProc(). What I mean by this is that I don't think I can prevent Windows from sending me WM_INPUT messages. When I register, I cannot indicate that I intend to read input using GetRawInputBuffer() (I think). So this then implies I need to handle the WM_INPUT messages somehow even though GetRawInputBuffer() doesn't require them. As you and this SO answer state, it is called outside a message loop.

emno Over a year ago

In contrast though, this SO answer implies there is a connection to WM_INPUT. And the msdn sample seems to call GetRawInputBuffer() inside a 'private message'. It further adds this odd call to DefRawInputProc(), which doesn't seem to do anything - but implies some cleanup needs to happen...? Edit: I've tried calling DefRawInputProc() but it didn't help..

|

Collectives™ on Stack Overflow

RawInput WinAPI: GetRawInputBuffer() and message handling

1 Answer 1

21 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

21 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related